Blog
About

42
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper

      , 1 , 2 , 2

      BMC Medical Education

      BioMed Central

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Background

          Reliable and valid written tests of higher cognitive function are difficult to produce, particularly for the assessment of clinical problem solving. Modified Essay Questions (MEQs) are often used to assess these higher order abilities in preference to other forms of assessment, including multiple-choice questions (MCQs). MEQs often form a vital component of end-of-course assessments in higher education. It is not clear how effectively these questions assess higher order cognitive skills. This study was designed to assess the effectiveness of the MEQ to measure higher-order cognitive skills in an undergraduate institution.

          Methods

          An analysis of multiple-choice questions and modified essay questions (MEQs) used for summative assessment in a clinical undergraduate curriculum was undertaken. A total of 50 MCQs and 139 stages of MEQs were examined, which came from three exams run over two years. The effectiveness of the questions was determined by two assessors and was defined by the questions ability to measure higher cognitive skills, as determined by a modification of Bloom's taxonomy, and its quality as determined by the presence of item writing flaws.

          Results

          Over 50% of all of the MEQs tested factual recall. This was similar to the percentage of MCQs testing factual recall. The modified essay question failed in its role of consistently assessing higher cognitive skills whereas the MCQ frequently tested more than mere recall of knowledge.

          Conclusion

          Construction of MEQs, which will assess higher order cognitive skills cannot be assumed to be a simple task. Well-constructed MCQs should be considered a satisfactory replacement for MEQs if the MEQs cannot be designed to adequately test higher order skills. Such MCQs are capable of withstanding the intellectual and statistical scrutiny imposed by a high stakes exit examination.

          Related collections

          Most cited references 21

          • Record: found
          • Abstract: not found
          • Article: not found

          The Impact of Classroom Evaluation Practices on Students

           T. Crooks (1988)
            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment

              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Comprehensive undergraduate medical assessments improve prediction of clinical performance.

              This study aimed to compare an essay-style undergraduate medical assessment with modified essay, multiple-choice question (MCQ) and objective structured clinical examination (OSCE) undergraduate medical assessments in predicting students' clinical performance (predictive validity), and to determine the relative contributions of the written (modified essay and MCQ) assessment and OSCE to predictive validity. Before and after cohort study. One medical school running a 6-year undergraduate course. Study participants included 137 Year 5 medical students followed into their trainee intern year. Aggregated global ratings by senior doctors, junior doctors and nurses as well as comprehensive structured assessments of performance in the trainee intern year. Students' scores in the new examinations predicted performance significantly better than scores in the old examinations, with correlation coefficients increasing from 0.05-0.44 to 0.41-0.81. The OSCE was a stronger predictor of subsequent performance than the written assessments but combining assessments had the strongest predictive validity. Using more comprehensive, more reliable and more authentic undergraduate assessment methods substantially increases predictive validity.
                Bookmark

                Author and article information

                Journal
                BMC Med Educ
                BMC Medical Education
                BioMed Central
                1472-6920
                2007
                28 November 2007
                : 7
                : 49
                Affiliations
                [1 ]Centre for Learning and Professional Development, University of Adelaide, Adelaide, Australia
                [2 ]Dept of Surgery, University of Adelaide, Adelaide, Australia
                Article
                1472-6920-7-49
                10.1186/1472-6920-7-49
                2148038
                18045500
                Copyright © 2007 Palmer and Devitt; licensee BioMed Central Ltd.

                This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

                Categories
                Research Article

                Education

                Comments

                Comment on this article