Students at Colorado State University have been taking end-of-semester course surveys, and it could be the last time some take paper surveys. The survey is undergoing a major make-over because of issues with bias, institutional misuse and waste.
Matthew Hickey is the chair of the committee spearheading the reformation of surveys, known as the Faculty Council Committee on Teaching and Learning (CoTL), a subcommittee of the Faculty Council. According to Hickey, the committee has created a three-pronged approach to reform; they are changing the questions asked, swapping paper for an online format and encouraging a cultural change in the survey’s use.
Course surveys were originally created by the Associated Students of CSU (ASCSU), the University’s student government, to be a tool for students to know about a class before taking it and provide feedback after taking it.
In Spring 2016, ASCSU cut funding to the course surveys under the leadership of then-President Jason Sydoriak.
Sydoriak originally intended to modernize the survey process when he came upon what he found to be a larger problem.
“When I started looking into it and doing my own research I came to the same conclusion that the committee had been coming to,” Sydoriak said. “That these course surveys are fairly arbitrary, and there is significant bias against females, minorities, and unfortunately, those who have to work in sub-par facilities.”
The surveys have continued to be the key tool used to evaluate instructor performance to make tenure and pay decisions. For Hickey, his committee, ASCSU and many other committees this is a problem – and CoTL, with input from ASCSU and other faculty members, has developed a plan to improve the survey and its use.
Step 1- New questions
The first piece of the surveys CoTL worked on is redesigning them to be less affected by inherent bias.
Hickey worked with Dr. Zinta Byrne, a psychology professor and the committee’s survey design expert, to create a better set of questions – one core set that applies to all courses and another set that is more particular to the course. The questions will also be more centered around the kind of environment instructors provide.
Gwen Gorzelsky, the executive director of TILT, serves as ex officio, a non-voting position, on CoTL. She looks forward to the questions in the survey being more focused on the learning environment.
“Right now the course survey questions at CSU tend to be more focused on the faculty person as an individual (or their) characteristics,” said Gorzelsky.
Gorzelsky said course surveys at other institutions also face this problem. Questions about fairness and approachability of the instructors focus more on the character of the faculty member and less on the learning environment they are creating.
Tyler Siri, the ASCSU representative for CoTL, looks forward to the new set of questions being more applicable for student use by providing more useful questions.
Hickey emphasizes that the questions will be reformed, but not perfect.
“I wouldn’t want to have any information out (there) that suggests that the perfect survey can overcome implicit bias, it’s not going to happen,” said Hickey. “Bias even appears in peer evaluation.”
The new questions may be beta tested as soon as the upcoming summer semester.
Step 2 – Switch to online format
CoTL has found a survey vender, Qualtrics, to switch the survey to an online format. According to Hickey, it will save on cost and allow more flexibility.
“The beauty of (online) then becomes this core set of questions we develop, and (the course-specific questions), become much more nimble,” Hickey said. “It’s conceivable we go a couple semesters into this, decide that those core questions are not quite getting what we thought they would and change those immediately.”
The online format may also make answers more easily available to students. According to Siri, students should eventually be able to find previous survey results on a platform they already understand, like Canvas or RamWeb.
Step 3 – Shift how the surveys are used
The final, and perhaps most ambitious, step to course survey reform is a shift in how the results are treated.
“Ultimately the goal is to shift the emphasis on this away from a tool for evaluation teaching effectiveness, which is a misapplication of this tool,” Hickey said. “This tool allows students to provide insight only they can provide to faculty members about the learning landscape in a classroom.”
Members of CoTL want the survey to be used in conjunction with other measurements of teaching effectiveness. Hickey said the results of the survey, as it looks now, are not helpful for measuring an instructor’s ability. In some cases, the survey instructors, who are rated poorly because of course difficulty, are often prompting the most student growth. When this happens, the survey can inversely measure instructor effectiveness, according to Hickey.
Hickey believes the survey could be used to help track how professors respond to student feedback when administrators are considering instructors for tenure.
“It’s an issue of cultural and institution change, and that kind of change takes time, but I think it’s very worth pursuing,” said Gorzelsky.
The survey could start to be distributed on Qualtrics as soon as fall semester 2017. Next spring, the new 2018-2019 ASCSU administration will have to decide if ASCSU should fund the online version of the survey for the 2018-2019 school year.
“It’s going to be fantastic for everyone, it’s going to be a much more useful tool for students, faculty and administrators,” said Siri.
Collegian Reporter Tatiana Talesnick-Parafiniuk can be reached at firstname.lastname@example.org or on Twitter @TatianaSophiaPT.