The choice of methods in determining the optimal cut-off value for quantitative diagnostic test evaluation

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Introduction The choice of criteria in determining optimal cut-off value is a matter of concern in quantitative diagnostic tests. Several indexes such as Youden's index, Euclidean index, product of sensitivity and specificity in receiver operator characteristic space and diagnostic odds ratio have been used in clinical practices but their advantages and limitations are not well understood by clinicians. This study aimed to compare these methods in determining optimal cut-off values for quantitative diagnostic test. Methods The different configurations of binormal and bilogistic distributions with equal and unequal variances for nondiseased and diseased subjects were examined. The cut-off values with increment of 0.1 in Z-scale were varied. Then, the Youden's index, Euclidean index, product of sensitivity and specificity, and diagnostic odds ratio were calculated over various cut-off values under distributional assumptions with confirmed parameters. Results According to the obtained data from binormal model and equal variances, the optimal cut-off values derived from Youden's index, Euclidean index, and product method were similar but the diagnostic odds ratio yielded either extremely low or extremely high optimal cut-off value. For bilogistic pair distributions with equal variances, the Youden's, Euclidean indexes and product method resulted in an identical cut-off value but the diagnostic odds ratio was constant over various cut-points. By both binormal and bilogistic data with more variations in nondiseased population, the Youden's index produced a higher sensitive optimal cut-off value; but with more variation for diseased distribution, the Euclidean index showed a more sensitive optimal cut-off. For bilogistic data with unequal variance, the log(diagnostic odds ratio) had a straight line relationship over cut-off values with either positive or negative slope. Conclusion As a measure of association, diagnostic odds ratio cannot be informative in determining an optimal cut-off value. The advantage of receiver operator characteristic analysis to obtain the optimal cut-off value is to use Youden's index, Euclidean index, or product index which is recommended. The choice between them depends on variability of test results in diseased and nondiseased subjects and the desired sensitivity.

Related collections

Most cited references 14

Record: found
Abstract: found
Article: not found

The meaning and use of the area under a receiver operating characteristic (ROC) curve.

J A Hanley, B J McNeil, Marnix van Holsbeeck (1982)

A representation and interpretation of the area under a receiver operating characteristic (ROC) curve obtained by the "rating" method, or by mathematical predictions based on patient characteristics, is presented. It is shown that in such a setting the area represents the probability that a randomly chosen diseased subject is (correctly) rated or ranked with greater suspicion than a randomly chosen non-diseased subject. Moreover, this probability of a correct ranking is the same quantity that is estimated by the already well-studied nonparametric Wilcoxon statistic. These two relationships are exploited to (a) provide rapid closed-form expressions for the approximate magnitude of the sampling variability, i.e., standard error that one uses to accompany the area under a smoothed ROC curve, (b) guide in determining the size of the sample required to provide a sufficiently reliable estimate of this area, and (c) determine how large sample sizes should be to ensure that one can statistically detect differences in the accuracy of diagnostic techniques.

0 comments Cited 3884 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests.

M. Greiner, D. Pfeiffer, R.D. Smith (2000)

We review the principles and practical application of receiver-operating characteristic (ROC) analysis for diagnostic tests. ROC analysis can be used for diagnostic tests with outcomes measured on ordinal, interval or ratio scales. The dependence of the diagnostic sensitivity and specificity on the selected cut-off value must be considered for a full test evaluation and for test comparison. All possible combinations of sensitivity and specificity that can be achieved by changing the test's cut-off value can be summarised using a single parameter; the area under the ROC curve. The ROC technique can also be used to optimise cut-off values with regard to a given prevalence in the target population and cost ratio of false-positive and false-negative results. However, plots of optimisation parameters against the selected cut-off value provide a more-direct method for cut-off selection. Candidates for such optimisation parameters are linear combinations of sensitivity and specificity (with weights selected to reflect the decision-making situation), odds ratio, chance-corrected measures of association (e. g. kappa) and likelihood ratios. We discuss some recent developments in ROC analysis, including meta-analysis of diagnostic tests, correlated ROC curves (paired-sample design) and chance- and prevalence-corrected ROC curves.

0 comments Cited 445 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker.

M. S. Pepe (2004)

A marker strongly associated with outcome (or disease) is often assumed to be effective for classifying persons according to their current or future outcome. However, for this assumption to be true, the associated odds ratio must be of a magnitude rarely seen in epidemiologic studies. In this paper, an illustration of the relation between odds ratios and receiver operating characteristic curves shows, for example, that a marker with an odds ratio of as high as 3 is in fact a very poor classification tool. If a marker identifies 10% of controls as positive (false positives) and has an odds ratio of 3, then it will correctly identify only 25% of cases as positive (true positives). The authors illustrate that a single measure of association such as an odds ratio does not meaningfully describe a marker's ability to classify subjects. Appropriate statistical methods for assessing and reporting the classification power of a marker are described. In addition, the serious pitfalls of using more traditional methods based on parameters in logistic regression models are illustrated.