+1 Recommend
1 collections
    • Record: found
    • Abstract: found
    • Article: found
    Is Open Access

    Student Evaluations of Teaching (Mostly) Do Not Measure Teaching Effectiveness

    ScienceOpen Research


    The article is currently in production. The preliminary preview version will be replaced with the processed article soon.

    This work has been published open access under Creative Commons Attribution License CC BY 4.0, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Conditions, terms of use and publishing policy can be found at


    permutation tests, gender bias, disparate impact, nonparametric statistics

    Read Bookmark
        There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.


        Student evaluations of teaching (SET) are widely used in academic personnel decisions as a measure of teaching effectiveness. We show:

        • SET are biased against female instructors by an amount that is large and statistically significant
        • the bias affects how students rate even putatively objective aspects of teaching, such as how promptly assignments are graded
        • the bias varies by discipline and by student gender, among other things
        • it is not possible to adjust for the bias, because it depends on so many factors
        • SET are more sensitive to students' gender bias and grade expectations than they are to teaching effectiveness
        • gender biases can be large enough to cause more effective instructors to get lower SET than less effective instructors.

        These findings are based on nonparametric statistical tests applied to two datasets: 23,001 SET of 379 instructors by 4,423 students in six mandatory first-year courses in a five-year natural experiment at a French university, and 43 SET for four sections of an online course in a randomized, controlled, blind experiment at a US university.

        Related collections

        Most cited references 24

        • Record: found
        • Abstract: not found
        • Article: not found

        On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9

          • Record: found
          • Abstract: found
          • Article: not found

          Hot or not: do professors perceived as physically attractive receive higher student evaluations?

          Previous research investigating the influence of perceived physical attractiveness on student evaluations of college professors has been limited to a handful of studies. In this study, the authors used naturally occurring data obtained from the publicly available Web site The data suggested that professors perceived as attractive received higher student evaluations when compared with those of a nonattractive control group (matched for department and gender). Results were consistent across 4 separate universities. Professors perceived as attractive received student evaluations about 0.8 of a point higher on a 5-point scale. Exploratory analyses indicated benefits of perceived attractiveness for both male and female professors. Although this study has all the limitations of naturalistic research, it adds a study with ecological validity to the limited literature.
            • Record: found
            • Abstract: not found
            • Article: not found

            Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors

              ScienceOpen disciplines:


              Comment on this article

              Register to benefit from advanced discovery features on more than 38,000,000 articles

              Already registered?