Common pitfalls in statistical analysis: Measures of agreement

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Agreement between measurements refers to the degree of concordance between two (or more) sets of measurements. Statistical methods to test agreement are used to assess inter-rater variability or to decide whether one technique for measuring a variable can substitute another. In this article, we look at statistical measures of agreement for different types of data and discuss the differences between these and those for assessing correlation.

Related collections

Most cited references 3

Record: found
Abstract: found
Article: found

Is Open Access

Accuracy and Reliability of Pallor for Detecting Anaemia: A Hospital-Based Diagnostic Accuracy Study

Ashwini Kalantri, Mandar Karambelkar, Rajnish Joshi … (2010)

Background Anaemia is a common disorder. Most health providers in resource poor settings rely on physical signs to diagnose anaemia. We aimed to determine the diagnostic accuracy of pallor for anaemia by using haemoglobin as the reference standard. Methodology/Principal Findings In May 2007, we enrolled consecutive patients over 12 years of age, able to consent and willing to participate and who had a haemoglobin measurement taken within a day of assessment of clinical pallor from outpatient and medicine inpatient department of a teaching hospital. We did a blind and independent comparison of physical signs (examination of conjunctivae, tongue, palms and nailbed for pallor) and the reference standard (haemoglobin estimation by an electronic cell counter). Diagnostic accuracy was measured by calculating likelihood ratio values and 95% confidence intervals (CI) at different haemoglobin thresholds and area under the receiver operating characteristic curve. Two observers examined a subset of patients (n = 128) to determine the inter-observer agreement, calculated by kappa statistics. We studied 390 patients (mean age 40.1 [SD 17.08] years); of whom 48% were women. The haemoglobin was <7 g/dL in 8% (95% confidence interval, 5, 10) patients; <9 g/dL in 21% (17, 26) patients and <12 g/dL in 64% (60, 70) patients. Among patients with haemoglobin <7 g/dL, presence of severe tongue pallor yielded a LR of 9.87 (2.81, 34.6) and its absence yielded a LR of 0. The tongue pallor outperformed other pallor sites and was also the best discriminator of anaemia at haemoglobin thresholds of 7 g/dL and 9 g/dL (area under the receiver operating characteristic curves (ROC area = 0.84 [0.77, 0.90] and 0.71[0.64, 0.76]) respectively. The agreement between the two observers for detection of anaemia was poor (kappa values = 0.07 for conjunctival pallor and 0.20 for tongue pallor). Conclusions/Significance Clinical assessment of pallor can rule out and modestly rule in severe anaemia.

0 comments Cited 25 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: found

Is Open Access

Gleason’s Grading of Prostatic Adenocarcinoma: Inter-Observer Variation Among Seven Pathologists at a Tertiary Care Center in Oman

Asim Qureshi, Ritu Lakhtakia, Maiya AL Bahri … (2016)

Prostatic adenocarcinoma is the commonest solid malignancy seen in Omani elderly males 60-80 years of age. The Gleason grade is the most widely used grading system for prostatic carcinoma and is recommended by the World Health Organization. A peer review was carried out at the Pathology Department of Sultan Qaboos University Hospital (SQUH), Oman, to assess the quality of reporting at the center. The aim of this study was to determine inter-observer variation among 7 pathologists working at a tertiary care center in Oman. A total of 47 consecutive prostatic biopsies were interdependently reviewed by seven pathologists and the results obtained were compared with each other and the original diagnosis. This peer review indicated a fair inter-observer agreement (0.482) among 7 pathologists in the department, with fair to moderate agreement when the results were compared to the reported diagnosis, comparable to the published literature. Dual and sub-specialty reporting are being instituted to improve the performance in this vital aspect of pathology.

0 comments Cited 3 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Scoring of Hand Eczema: Good Reliability of Hand Eczema Extent Score (HEES).

Annica Carlsson, Åke Svensson, Chris D. Anderson … (2017)

There is good agreement between dermatological staff and patients using the Hand Eczema Extent Score (HEES). The aim of this study was to assess inter- and intra-observer reliability of the HEES in dermatologists and intra-observer reliability of the HEES in patients with hand eczema. Six dermatologists assessed 18 patients twice. Only the hands of the patients were visible to the assessors. Patients performed a self-assessment twice. Inter- and intra-observer reliability was tested with intraclass correlation coefficient (ICC). The mean HEES score for all dermatologists' assessments was 21.0 (range 3.6-46.3). The corresponding mean scores for all patients' own assessments were 24.9 (range 4.0-54.0). Inter-observer reliability in the dermatologists' observations ICC classification was very good, median value 0.82 (range 0.56-0.92). The overall intra-observer reliability for the 6 dermatologists' ICC classification was very good (range 0.88-0.94). Intra-observer reliability in the patients' 2 self-assessments ICC classification was very good (ICC 0.95). In conclusion, HEES is a reliable tool for both dermatologists and patients to grade the extent of hand eczema.

0 comments Cited 2 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Journal

Journal ID (nlm-ta): Perspect Clin Res

Journal ID (iso-abbrev): Perspect Clin Res

Journal ID (publisher-id): PCR

Title: Perspectives in Clinical Research

Publisher: Medknow Publications & Media Pvt Ltd (India )

ISSN (Print): 2229-3485

ISSN (Electronic): 2229-5488

Publication date (Print): Oct-Dec 2017

Volume: 8

Issue: 4

Pages: 187-191

Affiliations

[1]Department of Anaesthesiology, Tata Memorial Centre, Mumbai, Maharashtra, India

[1 ]Department of Surgical Oncology, Tata Memorial Centre, Mumbai, Maharashtra, India

[2 ]Department of Gastroenterology, Sanjay Gandhi Postgraduate Institute of Medical Sciences, Lucknow, Uttar Pradesh, India

Author notes

Address for correspondence: Dr. Priya Ranganathan, Department of Anaesthesiology, Tata Memorial Centre, Ernest Borges Road, Parel, Mumbai - 400 012, Maharashtra, India. E-mail: drpriyaranganathan@ 123456gmail.com

Article

Publisher ID: PCR-8-187

DOI: 10.4103/picr.PICR_123_17

PMC ID: 5654219

PubMed ID: 29109937

SO-VID: 44150e93-4486-43a6-b133-87c65612b483

License:

This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as the author is credited and the new creations are licensed under the identical terms.

Common pitfalls in statistical analysis: Measures of agreement

Read this article at

Abstract

Related collections

Core Readings in Statistical Mediation Analysis

Most cited references 3

Accuracy and Reliability of Pallor for Detecting Anaemia: A Hospital-Based Diagnostic Accuracy Study

Gleason’s Grading of Prostatic Adenocarcinoma: Inter-Observer Variation Among Seven Pathologists at a Tertiary Care Center in Oman

Scoring of Hand Eczema: Good Reliability of Hand Eczema Extent Score (HEES).

Author and article information

Journal

Affiliations

Author notes

Article

History

Categories

Comments

Comment on this article

Similar content 392

Cited by 114

Most referenced authors 42