+1 Recommend
0 collections
      • Record: found
      • Abstract: found
      • Article: not found

      Detecting cheating in written medical examinations by statistical analysis of similarity of answers: pilot study.

      BMJ : British Medical Journal

      Probability, Deception, Education, Medical, Graduate, statistics & numerical data, Educational Measurement, England, Humans, Pediatrics, education, Pilot Projects

      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.


          To assess whether a computer program using a variant of Angoff's method can detect anomalous behaviour indicative of cheating in multiple choice medical examinations. Statistical analysis of 11 examinations held by the Royal College of Paediatrics and Child Health. UK postgraduate medical examination. Examination candidates. Detection of anomalous candidate pairs by regression of similarity of correct answers in all possible pairs of candidates on the overall proportion of correct answers. Anomalous pairs were subsequently assessed in terms of examination centres and the seating plan of candidates, to assess adjacency. The 11 examinations were taken by a total of 11,518 candidates, and Acinonyx examined 6,178,628 pairs of candidates. Two examinations showed no anomalies, and one examination found an anomaly resulting from a scanning error. The other eight examinations showed 13 anomalies compatible with cheating, and in each pair the two candidates had sat the examination at the same centre, and for six examinations with seating plans, the candidates in the anomalous pairs had been seated side by side. The raw probabilities of the anomalies varied from 3.9x10(-11) to 9.3x10(-30) (median = 1.1x10(-17)), with Bonferroni-corrected probabilities in the range 2.4x10(-5) to 4.1x10(-24) (median = 1.6x10(-11)). This suggests that one anomalous pair is found for every 1000 or so candidates taking this postgraduate examination. This statistical technique identified a small proportion of candidates who had very similar patterns of correctly answered questions. The likelihood is that one candidate has copied from the other, or that there was collusion, or that a technical error occurred in the exams department (as happened in a single case). Analysis of similarities can be used to identify cheating and as part of the quality assurance process of postgraduate medical examinations.

          Related collections

          Author and article information



          Comment on this article