1
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Development and Test-Item Analysis of a Freely Available 1900-Item Question Bank for Rheumatology Trainees

      research-article
      1 , , 2 , 3
      ,
      Cureus
      Cureus
      test item analysis, graduate medical education, rheumatology, evaluation, medical education

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Background

          Tests composed of multiple-choice questions are an established tool to help evaluate knowledge of medical content. Within the field of rheumatology, there is an absence of free and easily-accessible sets of multiple-choice questions that have been rigorously evaluated and analyzed.

          Objective

          To develop a question bank composed of multiple-choice questions that evaluate trainee knowledge of rheumatology, as well as to investigate the psychometric properties (reliability, discrimination indices, difficulty indices) of items within the question bank.

          Methods

          Multiple-choice questions were drafted according to a strict methodology devised by the investigators. Between January and December 2020, questions were administered in sets of 20-25 questions to test-takers who were either current trainees or had recently graduated from training programs. Performance was evaluated through descriptive statistics (mean, median, range, standard deviation) and test-item statistics (difficulty index, discrimination index, reliability).

          Results

          Investigators drafted 1900 multiple choice questions within 45 sections each composed of 20 to 25 questions each. These questions were administered to 32 participants. The mean discrimination index was 0.57 (standard deviation: 0.22) and mean difficulty index was 0.38 (standard deviation: 0.23). Reliability indices for the 45 sections ranged from 0.45 to 0.85 (mean: 0.613, standard deviation: 0.09). The overall reliability index for the entire item bank was greater than 0.95.

          Conclusion

          The investigators developed a 1900-item question bank composed of items that have sufficient difficulty and discrimination indices to be used for low- and moderate-stakes settings. A rigorous methodology was employed to create the first freely-accessible reliable tool for the assessment of rheumatology knowledge. This tool can be purposed for both summative and formative evaluation in multiple settings and platforms.

          Related collections

          Most cited references13

          • Record: found
          • Abstract: found
          • Article: not found

          Improving the fairness of multiple-choice questions: a literature review.

          The ubiquity of multiple-choice questions (MCQs) results from their efficiency and hence reliability. Cognitive knowledge assessed by MCQ predicts and correlates well with overall competence and performance but examinees and examiners alike frequently perceive MCQ-based testing as 'unfair'. Fairness is akin to defensibility and is an increasingly important concept in testing. It is dependent on psychometric adequacy, diligence of construction, attention to consequential validity and appropriate standard setting. There is a wealth of evidence that extended matching questions are the fairest format but MCQs should always be combined with practical assessments, as written testing emphasizes learning from written sources.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Education techniques for lifelong learning: writing multiple-choice questions for continuing medical education activities and self-assessment modules.

            The multiple-choice question (MCQ) is the most commonly used type of test item in radiologic graduate medical and continuing medical education examinations. Now that radiologists are participating in the maintenance of certification process, there is an increased need for self-assessment modules that include MCQs and persons with test item-writing skills to develop such modules. Although principles of effective test item writing have been documented, violations of these principles are common in medical education. Guidelines for test construction are related to development of educational objectives, defining levels of learning for each objective, and writing effective MCQs that test that learning. Educational objectives should be written in observable, behavioral terms that allow for an accurate assessment of whether the learner has achieved the objectives. Learning occurs at many levels, from simple recall to problem solving. The educational objectives and the MCQs that accompany them should target all levels of learning appropriate for the given content. Characteristics of effective MCQs can be described in terms of the overall item, the stem, and the options. Flawed MCQs interfere with accurate and meaningful interpretation of test scores and negatively affect student pass rates. Therefore, to develop reliable and valid tests, items must be constructed that are free of such flaws. The article provides an overview of established guidelines for writing effective MCQs, a discussion of writing appropriate educational objectives and MCQs that match those objectives, and a brief review of item analysis. (c) RSNA, 2006.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Multiple choice questions: a literature review on the optimal number of options.

              Single, best response, multiple choice questions (MCQs) with 4 options (3 distractors and 1 correct answer) or 5 options (4 distractors) have been widely used as an assessment tool in medical education in India and globally. Writing plausible distractors is time consuming and the most difficult part of preparing MCQs. If the number of options can be reduced to 3, it will make preparing MCQs less difficult and time consuming, thus reducing the likelihood of flaws in writing MCQs. We reviewed the literature to find out if the number of options in MCQ test items could be reduced to 3 without affecting the quality of the test. A systematic database search was done using the following question as a framework: How many options are optimal for multiple choice questions? Theoretical, analytical and empirical studies, which addressed this issue, were included in the review. There was no significant change in the psychometric properties of the 3 options test when compared with 4 and 5 options. MCQs with 3 options improved the efficiency of the test as well as its administration compared with 4- or 5-option MCQs. MCQs with 3 options had a higher efficiency because fewer distractors needed to be prepared, took up less space and required less reading time, decreased the time required to develop the items and the time to administer, and more items could be administered in a given time thus increasing the content sampled. Our review of the literature suggests that MCQs with 3 options provide a similar quality of test as that with 4- or 5-option MCQs. We suggest that MCQs with 3 options should be preferred.
                Bookmark

                Author and article information

                Journal
                Cureus
                Cureus
                2168-8184
                Cureus
                Cureus (Palo Alto (CA) )
                2168-8184
                29 September 2021
                September 2021
                : 13
                : 9
                Affiliations
                [1 ] Rheumatology, University of Iowa Hospitals and Clinics, Iowa City, USA
                [2 ] Internal Medicine, University of Iowa Hospitals and Clinics, Iowa City, USA
                [3 ] Nephrology, University of Iowa Hospitals and Clinics, Iowa City, USA
                Author notes
                Article
                10.7759/cureus.18382
                8483413
                02558073-06b1-4790-b21f-36cbe5d7f413
                Copyright © 2021, Kumar et al.

                This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

                Categories
                Medical Education
                Rheumatology

                test item analysis,graduate medical education,rheumatology,evaluation,medical education

                Comments

                Comment on this article