Defining and Adopting Clinical Performance Measures in Graduate Medical Education : Where Are We Now and Where Are We Going?

Patient outcomes have been used to assess the performance of hospitals and physicians; in contrast, residency programs have been compared based on nonclinical measures. To assess whether obstetrics and gynecology residency programs can be evaluated by the quality of care their alumni deliver. A retrospective analysis of all Florida and New York obstetrical hospital discharges between 1992 and 2007, representing 4 906 169 deliveries performed by 4124 obstetricians from 107 US residency programs. Nine measures of maternal complications from vaginal and cesarean births reflecting laceration, hemorrhage, and all other complications after vaginal delivery; hemorrhage, infection, and all other complications after cesarean delivery; and composites for vaginal and cesarean deliveries and for all deliveries regardless of mode. Obstetricians' residency program was associated with substantial variation in maternal complication rates. Women treated by obstetricians trained in residency programs in the bottom quintile for risk-standardized major maternal complication rates had an adjusted complication rate of 13.6%, approximately one-third higher than the 10.3% adjusted rate for women treated by obstetricians from programs in the top quintile (absolute difference, 3.3%; 95% confidence interval, 2.8%-3.8%). The rankings of residency programs based on each of the 9 measures were similar. Adjustment for medical licensure examination scores did not substantially alter the program ranking. Obstetrics and gynecology training programs can be ranked by the maternal complication rates of their graduates' patients. These rankings are stable across individual types of complications and are not associated with residents' licensing examination scores.

The public is calling for the U.S. health care and medical education system to be accountable for ensuring high-quality, safe, effective, patient-centered care. As medical education shifts to a competency-based training paradigm, clinician educators' assessment of and feedback to trainees about their developing clinical skills becomes paramount. However, there is substantial variability in the accuracy, reliability, and validity of the assessments faculty make when they directly observe trainees with patients. These difficulties have been treated primarily as a rater cognition problem focusing on the inability of the assessor to make reliable and valid assessments of the trainee.The authors' purpose is to reconceptualize the rater cognition problem as both an educational and clinical care problem. The variable quality of faculty assessments is not just a psychometric predicament but also an issue that has implications for decisions regarding trainee supervision and the delivery of quality patient care. The authors suggest that the frame of reference for rating performance during workplace-based assessments be the ability to provide safe, effective, patient-centered care. The authors developed the Accountable Assessment for Quality Care and Supervision equation to remind faculty that supervision is a dynamic, complex process essential for patients to receive high-quality care. This fundamental shift in how assessment is conceptualized requires new models of faculty development and emphasizes the essential and irreplaceable importance of the clinician educator in trainee assessment.

Researchers in medical education have been placing increased emphasis on "outcomes research," or the observable impact of educational interventions on patient care. However, although patient outcomes are obviously important, they should not be the sole focus of attention in medical education research. The purpose of this perspective is both to highlight the limitations of outcomes research in medical education and to offer suggestions to facilitate a proper balance between learner-centered and patient-centered assessments. The authors cite five challenges to research using patient outcomes in medical education, namely (1) dilution (the progressively attenuated impact of education as filtered through other health care providers and systems), (2) inadequate sample size, (3) failure to establish a causal link, (4) potentially biased outcome selection, and (5) teaching to the test. Additionally, nonpatient outcomes continue to hold value, particularly in theory-building research and in the evaluation of program implementation. To educators selecting outcomes and instruments in medical education research, the authors offer suggestions including to clarify the study objective and conceptual framework before selecting outcomes, and to consider the development and use of behavioral and other intermediary outcomes. Deliberately weighing the available options will facilitate informed choices during the design of research that, in turn, informs the art and science of medical education.