The research on course evaluations

On today’s show, Betsy Barre joins me to share about the research on course evaluations.

Guest: Betsy Barre

Assistant Director of the Center for Teaching Excellence at Rice University

After making the move to Rice in 2012, she was able to pursue her interest in undergraduate pedagogy by working with students and faculty in Rice’s newly developed Program in Writing and Communication. In this role, she taught a series of disciplinary-based first-year seminars and contributed to the PWC’s faculty development programming for those teaching first-year writing courses. And in July of 2014, she began her current position as Assistant Director of Rice’s newly established Center for Teaching Excellence. More

Quotes

One of the biggest complaints faculty have about student evaluations is that it’s not a reflection of teaching effectiveness.
–Betsy Barre

Just because a student likes a class doesn’t necessarily mean they’re learning.
–Betsy Barre

It turns out that the harder your course is, the higher evaluations you get.
–Betsy Barre

If students think the work is valuable and something that’s helping them learn, you can give up to twenty extra hours a week of work outside of class and students will still give you higher evaluations.
–Betsy Barre

When we want to know if students have learned, one of the best things to do is just ask them if they’ve learned.
–Betsy Barre

Part of the movement in student evaluations now is to ask questions about learning, rather than questions about what the faculty members are doing.
–Betsy Barre

Notes

• Article: Do Student Evaluations of Teaching Really Get an “F”?
• Screencast: Student Ratings of Instruction: A Literature Review
• RateMyProfessor Analysis: Gendered Language in Teaching Evaluations

Betsy’s Six Most Surprising Insights about Course Evaluations

Taken from her article “Do Student Evaluations of Teaching Really Get an “F”?”

1. Yes, there are studies that have shown no correlation (or even inverse correlations) between the results of student evaluations and student learning. Yet, there are just as many, and in fact many more, that show just the opposite.
2. As with all social science, this research question is incredibly complex. And insofar as the research literature reflects this complexity, there are few straightforward answers to any questions. If you read anything that suggests otherwise (in either direction), be suspicious.
3. Despite this complexity, there is wide agreement that a number of independent factors, easily but rarely controlled for, will bias the numerical results of an evaluation. These include, but are not limited to, student motivation, student effort, class size, and discipline (note that gender, grades, and workload are NOT included in this list).
4. Even when we control for these known biases, the relationship between scores and student learning is not 1 to 1. Most studies have found correlations of around .5. This is a relatively strong positive correlation in the social sciences, but it is important to understand that it means there are still many factors influencing the outcome that we don’t yet understand. Put differently, student evaluations of teaching effectiveness are a useful, but ultimately imperfect, measure of teaching effectiveness.
5. Despite this recognition, we have not yet been able to find an alternative measure of teaching effectiveness that correlates as strongly with student learning. In other words, they may be imperfect measures, but they are also our best measures.
6. Finally, if scholars of evaluations agree on anything, they agree that however useful student evaluations might be, they will be made more useful when used in conjunction with other measures of teaching effectiveness.

Recommendations

Bonni

• Think about how you administer the student evaluations.
• Check out her Betsy’s screencast (see above).

Betsy

• Design your own evaluation instrument and distribute it yourself, especially at the mid-point of the source.
• Take advantage of the teaching center on your campus for student interviews and classroom observations.

Are You Enjoying the Show?

1. Rate/review the show. Please consider rating or leaving a review for the Teaching in Higher Ed podcast on whatever service you use to listen to it on (iTunes, Stitcher, etc.). It is the best way to help others discover the show.
2. Give feedback. As always, I welcome suggestions for future topics or guests.
3. Subscribe. If you have yet to subscribe to the weekly update, you can receive a single email each week with the show notes (including all the links we talk about on the episode), as well as an article on either teaching or productivity.