Blog
About

46
views
0
recommends
+1 Recommend
0 collections
    8
    shares
      • Record: found
      • Abstract: not found
      • Article: not found
      Is Open Access

      Interrater reliability: the kappa statistic

      Biochemia Medica

      Croatian Society for Medical Biochemistry and Laboratory Medicine

      Read this article at

      ScienceOpenPublisher
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          The kappa statistic is frequently used to test interrater reliability. The importance of rater reliability lies in the fact that it represents the extent to which the data collected in the study are correct representations of the variables measured. Measurement of the extent to which data collectors (raters) assign the same score to the same variable is called interrater reliability. While there have been a variety of methods to measure interrater reliability, traditionally it was measured as percent agreement, calculated as the number of agreement scores divided by the total number of scores. In 1960, Jacob Cohen critiqued use of percent agreement due to its inability to account for chance agreement. He introduced the Cohen’s kappa, developed to account for the possibility that raters actually guess on at least some variables due to uncertainty. Like most correlation statistics, the kappa can range from −1 to +1. While the kappa is one of the most commonly used statistics to test interrater reliability, it has limitations. Judgments about what level of kappa should be acceptable for health research are questioned. Cohen’s suggested interpretation may be too lenient for health related studies because it implies that a score as low as 0.41 might be acceptable. Kappa and percent agreement are compared, and levels for both kappa and percent agreement that should be demanded in healthcare studies are suggested.

          Related collections

          Most cited references 13

          • Record: found
          • Abstract: not found
          • Article: not found

          A Coefficient of Agreement for Nominal Scales

            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            The Kappa Statistic: A Second Look

              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Meta-analysis of Pap test accuracy.

              A literature search identified 62 studies published by August 1992 comparing Papanicolaou (Pap) test results with histology. Critical appraisal revealed that 82% of these had potential for verification bias and that only 37% stated that cytology and histology were independently assessed. Estimates of sensitivity and specificity ranged from 11 to 99% and 14 to 97%, respectively, and were highly negatively correlated (r = -0.63). Meta-analysis was used to combine data from 59 studies to estimate the accuracy of the Pap test using a summary receiver operating characteristic curve and to examine the effect of study quality. The summary receiver operating characteristic curve suggests that the Pap test may be unable to achieve concurrently high sensitivity and specificity. For example, specificity in the 90-95% range corresponds to sensitivity in the 20-35% range. Pap test accuracy was not associated with reported study characteristics or dimensions of quality. Future primary studies should pay more attention to methodologic standards for the conduct and reporting of diagnostic test evaluations.
                Bookmark

                Author and article information

                Journal
                Biochemia Medica
                Biochem Med
                Croatian Society for Medical Biochemistry and Laboratory Medicine
                18467482
                2012
                2012
                : 276-282
                Article
                3900052
                23092060
                © 2012

                Comments

                Comment on this article