Compiled by Orlando Archibeque, Auraria Library, March 2015
This paper was prepared at the request of the Minority Affairs Committee or MAC (a committee of the Faculty Assembly of the University of Colorado Denver). It is not meant to be a comprehensive literature review of student evaluation of teacher performance. Rather, it is intended to provide some background information for those who are planning to attend the MAC-sponsored “Multiple Means of Evaluation? Identifying Best Practices with FCQs” workshop, April 8, 2015, 12:00 – 1:00 pm, Lawrence Street Center 1150).
“FCQ” or “Faculty Course Questionnaire” is the term used to refer to the survey instrument that University of Colorado Denver students complete at the conclusion of each course. The term FCQ rarely appears in the scholarly literature of teaching/learning. Instead, most researchers use the terms “student evaluation of teaching” (or SET) or “student ratings of teaching” (or SRT). It is impossible (nor would it be very helpful) to provide a comprehensive literature review in this short paper because of the sheer volume of research done in this area. One researcher (Benton, 2012) identified 2,875 references in the ERIC (Educational Resources Information Center) database on this topic through the year 2010.
Readers needing or wanting a more comprehensive perspective of student evaluation of teaching may find the following literature reviews helpful:
- Benton, S.L., Cashin, W.E. (2012). IDEA Paper #50: Student ratings of teaching: a summary of research and literature. Manhattan, KS: The IDEA Center. Full text: http://www.ideaedu.org/sites/default/files/idea-paper_50.pdf.
- Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: the state of the art. Review of Educational Research, 83(4), 598-642. Full text: http://0-rer.sagepub.com.skyline.ucdenver.edu/content/early/2013/08/12/0034654313496870.full.pdf+html
- Clayson, D. (2009). Student evaluations of teaching: are they related to what students learn? A meta-analysis and review of the literature. Journal of Marketing Education, 31(1), 16-30. Full text: http://0-jmd.sagepub.com.skyline.ucdenver.edu/content/31/1/16.full.pdf+html
- Palermo, J. (2011). Linking student evaluations to institutional goals: a change story. Assessment & Evaluation in Higher Education, 38(2), 211-223. Full text: http://0-www.tandfonline.com.skyline.ucdenver.edu/doi/pdf/10.1080/02602938.2011.618880
While there is an extensive and complex literature on student evaluation of teaching, there is very little pertaining to best practices on this topic, and even less on end-of-course questionnaires such as FCQs. Several universities have developed and implemented best practices, including the following:
In Fall 2007, the ISU Center for Excellence in Learning and Teaching (CELT) Advisory Board formed a committee to explore best practices related to student evaluation of teaching (SET). The CELT categorized these guidelines into four areas:
- use of SET
University of Michigan
Generally, students are able to report on the extent to which a teacher appears prepared for class sessions, communicates clearly, stimulates interest, and demonstrates enthusiasm and respect for students; research shows that student responses on these dimensions are valid and reliable. Generally, students are less able to judge the knowledge of the instructor or scholarly content and currency of a course.
When using student ratings for personnel decisions, keep the following guidelines in mind:
- Student ratings results should not be used as the only source of data on the quality of an instructor’s teaching.
- Questions about instructors and courses should be relevant. They should fit the instructors and courses being evaluated.
- Multiple sets of ratings of faculty courses over time should be considered; personnel decisions should be influenced only by ratings from several courses over several terms.
- Because global ratings of the teacher or course tend to correlate higher with student learning than do more specific items, personnel decisions should rely more on global items (e.g., "Overall, this is an excellent course." "Overall, the instructor is an excellent teacher.").
- Do not over emphasize small differences in ratings results. Especially for personnel decisions, three broad categories are sufficient (“excellent,” to help identify possible nominees for teaching honors; “satisfactory”; needs improvement).
- Comparative data (such as departmental, school, or institutional norms) allow individual evaluations to be interpreted within a meaningful context. For example, information about course characteristics (e.g., disciplinary field, class size, required/elective, lower division/upper division, etc.) should be considered when reviewing evaluation results.
- For additional context, departments can provide opportunities for instructors to comment on their ratings, either in a teaching statement or in a separate document. In particular, such comments allow instructors to offer their own perspective on student ratings results and they can also provide context on any special circumstances surrounding a given course (e.g., new courses or innovations in teaching, a shift from an elective to a required course, changes in departmental grading standards, student resistance to certain types of material).
- Student rating results should be considered in personnel decisions only when at least 10 students in a given class respond and only when the majority of the students in a class have completed the surveys.
- The use of optional items chosen by the instructor customizes the forms and makes them more useful for teaching improvement purposes.
- While written comments are particularly helpful in improving classroom performance, they are not recommended for use in personnel decisions.
Vanderbilt’s Center for Teaching has an extensive webpage which provides guidelines and best practices concerning student evaluations, including the following:
- Tips for Making Sense of Student Evaluation Feedback
- Resources on Interpreting Student Evaluations
- Summaries of Research on Student Evaluations
California State University System
Report on Student Evaluations of Teaching. (2008). Joint Committee: The California State University, California Faculty Association, Academic Senate CSU. http://www.calstate.edu/AcadSen/Records/Reports/documents/Report_on_Student_Evaluations_of_Teaching.pdf This 15-page report provides recommendations on how to improve end-of-semester student evaluations of teaching at the CSU institutions.
One major issue with end-of-semester student evaluations that researchers have identified is the question of student honesty. The following studies, for example, have found that students often misrepresent the truth in such questionnaires. See the following:
- Clayson, D. E., & Haley, D. A. (2011). Are students telling us the truth? A critical look at the student evaluation of teaching. Marketing Education Review, 21(Summer), 101-112. Full text: http://0-www.tandfonline.com.skyline.ucdenver.edu/doi/pdf/10.2753/MER1052-8008210201
- Sproule, R. (2002). The underdetermination of instructor performance by data from the student evaluation of teaching. Economics of Education Review, 21, 287-295. Full text: http://tinyurl.com/nmmaols
- Stanfel, L. E. (1995). Measuring the accuracy of student evaluations of teaching. Journal of Instructional Psychology, 22(2), 117-125. Full text: http://0-search.ebscohost.com.skyline.ucdenver.edu/login.aspx?direct=true&db=pbh&AN=9508012009
- A second major issue is the problem of bias in student evaluations of courses taught by minority, female, and international faculty members. See Bias in Student Evaluations of Minority Faculty for some recent publications on this matter.
For these reasons many researchers have called into question the practice of using end-of-semester evaluations for any personnel decisions.