I am writing to update you about recommendations AASUA has provided to the employer for implementation of the new student opinion survey tool replacing Universal Student Ratings of Instruction (USRI) — the so-called Student Perspectives of Teaching (SPOT).
The draft of SPOT was piloted last fall; key changes from USRI include new questions, and a shift from numerical rankings to more qualitative assessment (such as ‘strongly agree,’ ‘disagree,’ etc.). These aspects of SPOT were spurred by changes in our Collective Agreement from the last round of bargaining, where updates included new language acknowledging the bias inherent in these surveys. General Faculties Council (GFC) approved the changes included with SPOT during a Feb. 27 meeting.
AASUA’s position on SPOT, as with USRI, remains that as a tool biased along lines of gender and visible minorities, student opinion surveys should not be used to evaluate an instructors’ teaching. We are pleased to see SPOT places an emphasis on students’ perceptions of learning, clarifying that these surveys should be viewed as a data point to understanding student views of a particular course. However, AASUA remains concerned that SPOT and the employer’s testing around the new tool do not adequately address the issues of bias we have raised.
Accordingly, AASUA has provided four recommendations to the employer for the implementation of SPOT. These recommendations were developed in consultation with AASUA’s Academic Affairs Committee (AAC) and include:
- That SPOT be re-evaluated on an ongoing basis, as opposed to analyzed only in the event a specific issue arises;
- That data produced through this ongoing re-evaluation be used to test whether SPOT results differ systemically across gender, race, and other key demographics;
- That visualizations of test data (such as radar charts) not be produced for any purposes, as this treats SPOT data as numerical response data; and
- That clear orientation material be provided to Chairs and Deans outlining how SPOT differs from USRI, how the data should be used, and how it should not be used. We are recommending these guidelines also be shared with all faculty/instructors whose evaluation process incorporates SPOT data.
Over Reading Week, we received a response from the employer indicating the points we raised align with their plans for SPOT, and that they are committed to ongoing consultation with the AASUA. We are hopeful our recommendations will be implemented as SPOT rolls out, as they are key to moving forward in a way that is conscious of the harmful potential of survey tools.
AASUA members interested in learning more about SPOT and USRI can review materials shared at the General Faculties Committee on the Learning Environment (CLE) meeting here, which includes links to the employer’s SPOT study, SPOT questions, and additional context on USRI.