The effects of taking a practice examination on scores in the Qualifying Examination of the Medical Council of Canada.

Abstract

A practice examination has been prepared annually at the University of Toronto for use by those who will take the Qualifying Examination of the Medical College of Canada. Passing the Qualifying Examination is an important step in gaining the Licentiate of the Medical College of Canada (the LMCC) and thus in becoming licensed. The effect of taking the practice examination on scores on the LMCC exam was investigated. The practice examination is a smaller replica of the LMCC exam, with questions spanning major medical specialties presented in both multiple-choice and patient management problem (PMP) format. It was conjectured that the effect of practice would be greater on the PMP score than on the multiple-choice score. Analysis of covariance was used to compare the LMCC examination scores of senior students, who had taken the practice examination, with the scores of those who had not. The comparison was performed for two graduating classes, separately, and involved the use of a score summarizing each student's performance in the third year of the curriculum as a covariate. In each graduating class, the practice effect on PMP scores is statistically significant. In neither class alone is the practice effect on multiple-choice scores statistically significant, although a combination of results in the two classes is statistically significant. The presence of practice effects can be taken to indicate a flaw in the testing process. The elimination of practice effects, through experiences like the practice examination, would make the entire testing process more valid.