Students’ performance in written and viva-voce components of final summative pharmacology examination in MBBS curriculum: A critical insight

Abstract

Sir, Assessment of learning has always been a difficult, yet an essential component of an educational program. In the undergraduate medical education system in India, curricular guidelines of Medical Council of India lay emphasis on methods of assessment of knowledge and skills in pharmacology.[1] Although continuous formative assessment constitutes an integral part in the curriculum, the ‘pass’ and the ‘fail’ certificates are based to a great extent on students’ performance in the final summative examination. The final examination consists of written papers, viva-voce sessions and practical exercises. Written examination consists of two papers. Each paper has the maximum marks of 40 and contains structured essay type questions and short notes. In viva-voce examination, each student is assessed by five examiners; two of them external examiners and the others internal examiners. In order to pass, a candidate must obtain 50% marks in aggregate with a minimum of 50% marks in written and viva together and a minimum of 50% marks in practical examination.[1] The written examination is a useful evaluation format that not only tests students’ ability to recall facts, but also can assess higher-order cognitive functions, such as interpretation of data and problem solving skills. The viva-voce examination on the other hand is a general encounter between a candidate and one or more examiners.[2] Viva-voce examinations are less reliable as they are essentially subjective in nature, afflicted with ‘halo effects’, errors of central tendency, a general tendency toward leniency, and errors of contrast.[2] Examiners mostly indulge in over-marking in viva-voce examinations in order to make an otherwise undeserving candidate ‘pass’. We explored this recently in a small questionnaire-based interview among examiners of pharmacology in one university and the examiners admitted showing such ‘leniency’. Against this backdrop, we planned the present study to compare students’ performance in written and viva-voce components of the final summative pharmacology examination in MBBS curriculum in order to have a critical insight into the two modes of evaluation, the way they are practiced. This was a record-based observational study done in a medical college in India that also served as an examination centre for second professional MBBS examination in pharmacology for four consecutive years, from 2008 to 2011. The performance of students was assessed. Permission for access to the students’ score sheets was obtained from appropriate authority and confidentiality of individual student's score was maintained. Percentage of marks obtained by four batches of students (n=589), in consecutive years (2008-11), in written and viva-voce components of the final summative examination in pharmacology were reviewed: Batch 1 (Jan 2011 Exam, n=159), Batch 2 (Jan 2010 Exam, n=139), Batch 3 (Jan 2009 Exam, n=148), Batch 4 (Jan 2008 Exam, n=143). Based on their performance in terms of percentage of marks in aggregate, all students in a batch were classified into four categories viz., ‘failed’(F) – <50%, ‘borderline passed’ (BP) – 50-57%, ‘passed’ (P) – >57% to <75% and ‘passed with distinction’ (PD) – ≥75%. Correlation was assessed between the percentage of marks obtained by students in these categories in written vis-a-vis viva-voce examination. Highly significant association was observed in marks obtained by students in P and PD categories in all four batches in viva-voce and written examination (P<0.001). However, no significant association was observed in marks obtained by students in F and BP categories in all four batches in viva-voce and written examination (P>0.05). The results are shown in Table 1. Interestingly, no student in F category got 50% marks in written examination, but most of them scored satisfactorily in the viva-voce. Among all the students in F category, three students in 2008, five in 2009, four in 2010 and none in 2011 failed in practical examination. The number of students in each category (e.g., F, BP, P and PD) when compared among the 4 years (2008-11) did not show any significant difference [Figure 1]. Table 1 Written and viva-voce marks of students in F, BP, P and PD categories in the final summative pharmacology examination Figure 1 Number of students in F, BP, P and PD category in 4 years. Data were analyzed by ANOVA. P=0.998 Our study showed that there was highly significant association between written and viva-voce marks of students in the PD and P categories (P<0.001). We interpret this as the ‘true’ reflection of knowledge and competence of the students in this category. Our study also revealed that there was a lack of significant association in performance in written and viva-voce examination among students in F and BP categories (P>0.05). Marks obtained by students in viva-voce were higher with respect to those in written examination in these two categories. Rather , the poorer the performance in written examination, the higher the marks obtained in the viva-voce. Such trend is most prominent in the F category of students very consistently , in all the four batches in successive years . These findings tend to establish the low reliability of viva-voce examinations. Examiners’ general tendency toward ‘leniency’ favoring particularly the weaker students may explain this trend. The higher marks in viva-voce examination compared to those in written papers, we believe, owes to the inherent subjective nature of viva-voce. Contrary to the written or to some extent, the practical components, archival evidence of student's performance is characteristically missing in the viva-voce examination. Further, the students’ performance was very uniform and consistent across the four consecutive years (2008-11) with little difference (P=0.998) in the number of students in F, BP, P and PD categories [Figure 1]. This further raises doubt against the reliability of the evaluation. The viva-voce examination forms an essential component of the final summative examination in medical colleges of India. The observations of the present work re-establish the subjective nature of these examinations.[2–6] Such trend maybe universal and not just limited to one university. There is, however, a relative paucity of published literature in this context. There is a need for having a relook at the strategies followed in the existing assessment system.