There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.
Abstract
The literature contains a large number of potential biases in the evaluation of diagnostic
tests. Strict application of appropriate methodological criteria would invalidate
the clinical application of most study results.
To empirically determine the quantitative effect of study design shortcomings on estimates
of diagnostic accuracy.
Observational study of the methodological features of 184 original studies evaluating
218 diagnostic tests. Meta-analyses on diagnostic tests were identified through a
systematic search of the literature using MEDLINE, EMBASE, and DARE databases and
the Cochrane Library (1996-1997). Associations between study characteristics and estimates
of diagnostic accuracy were evaluated with a regression model.
Relative diagnostic odds ratio (RDOR), which compared the diagnostic odds ratios of
studies of a given test that lacked a particular methodological feature with those
without the corresponding shortcomings in design.
Fifteen (6.8%) of 218 evaluations met all 8 criteria; 64 (30%) met 6 or more. Studies
evaluating tests in a diseased population and a separate control group overestimated
the diagnostic performance compared with studies that used a clinical population (RDOR,
3.0; 95% confidence interval [CI], 2.0-4.5). Studies in which different reference
tests were used for positive and negative results of the test under study overestimated
the diagnostic performance compared with studies using a single reference test for
all patients (RDOR, 2.2; 95% CI, 1.5-3.3). Diagnostic performance was also overestimated
when the reference test was interpreted with knowledge of the test result (RDOR, 1.3;
95% CI, 1.0-1.9), when no criteria for the test were described (RDOR, 1.7; 95% CI,
1.1-2.5), and when no description of the population under study was provided (RDOR,
1.4; 95% CI, 1.1-1.7).
These data provide empirical evidence that diagnostic studies with methodological
shortcomings may overestimate the accuracy of a diagnostic test, particularly those
including nonrepresentative patients or applying different reference standards.