Student evaluations of teaching (mostly) do not measure teaching effectiveness

Abstract

Student evaluations of teaching (SET) are widely used in academic personnel decisions as a measure of teaching effectiveness. We show:

• SET are biased against female instructors by an amount that is large and statistically significant.

• The bias affects how students rate even putatively objective aspects of teaching, such as how promptly assignments are graded.

• The bias varies by discipline and by student gender, among other things.

• It is not possible to adjust for the bias, because it depends on so many factors.

• SET are more sensitive to students’ gender bias and grade expectations than they are to teaching effectiveness.

• Gender biases can be large enough to cause more effective instructors to get lower SET than less effective instructors.

These findings are based on nonparametric statistical tests applied to two datasets: 23,001 SET of 379 instructors by 4,423 students in six mandatory first-year courses in a five-year natural experiment at a French university, and 43 SET for four sections of an online course in a randomized, controlled, blind experiment at a US university.

Affiliations

[1 ]OFCE, SciencesPo, Paris, France

[2 ]PSL, Université Paris-Dauphine, LEDa, UMR DIAL, Paris, France

[3 ]Department of Statistics, University of California, Berkeley, CA, USA

Author notes

[* ]Corresponding author's e-mail address: pbstark@ 123456berkeley.edu

Contributors

Anne Boring: (View ORCID Profile)

Kellie Ottoboni: (View ORCID Profile)

Philip B. Stark: (View ORCID Profile)

Journal

Journal ID (publisher-id): SOR-EDU

Title: ScienceOpen Research

Publisher: ScienceOpen

ISSN (Electronic): 2199-1006

Publication date 07 January 2016

Volume: 0 (ID: 818d8ec0-5908-47d8-86b4-5dc38f04b23e )

Issue: 0

Pages: 1-11

Publisher ID: 3507:XE

DOI: 10.14293/S2199-1006.1.SOR-EDU.AETBZC.v1

Copyright statement: © 2016 Boring et al.

License:

This work has been published open access under Creative Commons Attribution License CC BY 4.0 , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Conditions, terms of use and publishing policy can be found at www.scienceopen.com .

Counts

Figures: 0, Tables: 11, References: 23, Pages: 11

Product

Categories

Subject: Original article