For Publishers
DiscoveryMetadataPeer reviewHostingPublishing
For Researchers
JoinPublishReviewCollect
Register
Blog
About
Search
Advanced search
My ScienceOpen
Search
For Publishers
For Researchers
Blog
About
29
views
17
references
 Top references
cited by
3
    
0 reviews
 Review
0
comments
 Comment
0
recommends
+1 Recommend
0 collections
    0
    shares
      60
      similar
       All similar
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Identification of technical item flaws leads to improvement of the quality of single best Multiple Choice Questions

      research-article

      Read this article at

      ScienceOpenPMC
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Objective: The purpose of the study was to identify technical item flaws in the multiple choice questions submitted for the final exams for the years 2009, 2010 and 2011.

          Methods: This descriptive analytical study was carried out in Islamic International Medical College (IIMC). The Data was collected from the MCQ’s submitted by the faculty for the final exams for the year 2009, 2010 and 2011. The data was compiled and evaluated by a three member assessment committee. The data was analyzed for frequency and percentages the categorical data was analyzed by chi-square test.

          Results: Overall percentage of flawed item was 67% for the year 2009 of which 21% were for testwiseness and 40% were for irrelevant difficulty. In year 2010 the total item flaws were 36% and 11% testwiseness and 22% were for irrelevant difficulty. The year 2011 data showed decreased overall flaws of 21%. The flaws of testwisness were 7%, irrelevant difficulty were 11%.

          Conclusion: Technical item flaws are frequently encountered during MCQ construction, and the identification of flaws leads to improved quality of the single best MCQ’s.

          Related collections

          Most cited references17

          • Record: found
          • Abstract: not found
          • Article: not found

          The Assessment of Professional Competence: Developments, Research and Practical Implications

          C Van der Vleuten, CP Van der Vleuten, CPM Van Der Vleuten … (1996)
          Article 0 comments Cited 44 times – based on 0 reviews     Review now
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Multiple choice questions: a literature review on the optimal number of options.

            Rashmi Vyas, Avinash Supe (2015)
            Single, best response, multiple choice questions (MCQs) with 4 options (3 distractors and 1 correct answer) or 5 options (4 distractors) have been widely used as an assessment tool in medical education in India and globally. Writing plausible distractors is time consuming and the most difficult part of preparing MCQs. If the number of options can be reduced to 3, it will make preparing MCQs less difficult and time consuming, thus reducing the likelihood of flaws in writing MCQs. We reviewed the literature to find out if the number of options in MCQ test items could be reduced to 3 without affecting the quality of the test. A systematic database search was done using the following question as a framework: How many options are optimal for multiple choice questions? Theoretical, analytical and empirical studies, which addressed this issue, were included in the review. There was no significant change in the psychometric properties of the 3 options test when compared with 4 and 5 options. MCQs with 3 options improved the efficiency of the test as well as its administration compared with 4- or 5-option MCQs. MCQs with 3 options had a higher efficiency because fewer distractors needed to be prepared, took up less space and required less reading time, decreased the time required to develop the items and the time to administer, and more items could be administered in a given time thus increasing the content sampled. Our review of the literature suggests that MCQs with 3 options provide a similar quality of test as that with 4- or 5-option MCQs. We suggest that MCQs with 3 options should be preferred.
            Article 0 comments Cited 19 times – based on 0 reviews     Review now
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments.

              Brett K. Hayes, Marie Tarrant, Aimee Knierim … (2006)
              Multiple-choice questions are a common assessment method in nursing examinations. Few nurse educators, however, have formal preparation in constructing multiple-choice questions. Consequently, questions used in baccalaureate nursing assessments often contain item-writing flaws, or violations to accepted item-writing guidelines. In one nursing department, 2770 MCQs were collected from tests and examinations administered over a five-year period from 2001 to 2005. Questions were evaluated for 19 frequently occurring item-writing flaws, for cognitive level, for question source, and for the distribution of correct answers. Results show that almost half (46.2%) of the questions contained violations of item-writing guidelines and over 90% were written at low cognitive levels. Only a small proportion of questions were teacher generated (14.1%), while 36.2% were taken from testbanks and almost half (49.4%) had no source identified. MCQs written at a lower cognitive level were significantly more likely to contain item-writing flaws. While there was no relationship between the source of the question and item-writing flaws, teacher-generated questions were more likely to be written at higher cognitive levels (p<0.001). Correct answers were evenly distributed across all four options and no bias was noted in the placement of correct options. Further training in item-writing is recommended for all faculty members who are responsible for developing tests. Pre-test review and quality assessment is also recommended to reduce the occurrence of item-writing flaws and to improve the quality of test questions.
              Article 0 comments Cited 18 times – based on 0 reviews     Review now
                Bookmark
                All references

                Author and article information

                Journal
                Journal ID (nlm-ta): Pak J Med Sci
                Journal ID (iso-abbrev): Pak J Med Sci
                Journal ID (publisher-id): PJMS
                Title: Pakistan Journal of Medical Sciences
                Publisher: Professional Medical Publicaitons (Karachi, Pakistan )
                ISSN (Print): 1682-024X
                ISSN (Electronic): 1681-715X
                Publication date (Print): May-Jun 2013
                Volume: 29
                Issue: 3
                Pages: 715-718
                Affiliations
                [1 ]Dr. Humaira Fayyaz Khan, MBBS, FCPS, Assistant Professor Physiology, Islamic International Medical College, Rawalpindi, Pakistan.
                [2 ]Dr. Khalid Farooq Danish, MBBS, FCPS, Professor and Head of Surgery, Islamic International Medical College, Rawalpindi, Pakistan.
                [3 ]Dr. Azra Saeed Awan, MBBS, FCPS, Professor of Gynaecology and Obstetrics, Islamic International Medical College, Rawalpindi, Pakistan.
                [4 ]Maj. Gen.(R) Masood Anwar, MBBS, FCPS, Principal, Islamic International Medical College, Rawalpindi, Pakistan.
                Author notes
                Correspondence: Dr. Humaira Fayyaz Khan, Ho No: 700-A, Street III, Chaklala Scheme III, Rawalpindi, Pakistan. E-mail: drhumairakamal@yahoo.com
                Article
                Publisher ID: pjms-29-715
                PMC ID: 3809311
                SO-VID: 0880e161-932e-4c69-a8f4-863aa0ec10bd
                License:

                This is an Open Access article distributed under the terms of the Creative Commons Attribution License, ( http://creativecommons.org/licenses/by/3.0/) which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

                History
                Date received : 27 September 2012
                Date revision received : 15 May 2013
                Date accepted : 18 May 2013
                Categories
                Subject: Original Article

                Keywords: frequency,item writing flaws,testwiseness
                Data availability:
                Keywords: frequency, item writing flaws, testwiseness

                Comments

                Comment on this article

                Similar content60

                See all similar

                Cited by3

                See all cited by

                Most referenced authors46

                See all reference authors