Assessing the Value of Diagnostic Tests in the Coronavirus Disease 2019 Pandemic

Abstract

Editor: Until recently, we lived in a relatively quiet world where diagnostic tests had a known and pretty stable performance for diagnosing or excluding a given disease. By their definition, the sensitivity and specificity of the diagnostic test are independent from disease prevalence. This allows for a practical use of the Bayes theorem, with a test likelihood ratio that is used to translate the pretest probability to posttest probability (1). However, in practice, the sensitivity and specificity of a test may vary along with the disease prevalence (2). The concomitant variation of prevalence and diagnostic performance can occur through clinical factors (eg, disease spectrum, referral filter, and reader expectation) and artificial factors (eg, selection of patients, verification bias, and reference standard misclassification) (3). These effects are not easy to predict; higher (or lower) prevalence does not necessarily lead to higher (or lower) sensitivity and specificity. Positive and negative predictive values of a test are known to change with prevalence. However, the clinical setting for the physician is usually known and predictable. Epidemiologists have been able to predict effectiveness of screening programs by considering prevalence as a function of risk factors in the population. The coronavirus disease 2019 (COVID-19) pandemic outbreak has opened a window on an unexpected world. The prevalence of COVID-19 has moved from 0% to an unknown but surely high proportion of the population. This prevalence is also changing over time in different parts of the world. Indeed, we are now faced with a highly heterogeneous situation, with some countries having few cases and countries having hundreds of thousands ( https://coronavirus.jhu.edu/map.html ). Moreover, the prevalence is highly heteronomous even within a single country. Because the severe acute respiratory syndrome coronavirus 2 virus was previously unknown, we are moving in an almost blind fashion for both diagnosis and treatment. On the diagnostic side, we are faced with new questions and challenges. When to test? Whom to test? How often to test? How do we interpret the test results? In this continuously changing scenario, clinicians need to quickly adapt to their own clinical setting. For diagnostic testing of COVID-19, the thresholds that define a positive test need to be adapted as well, by integrating prevalence of the specific clinical setting. The most common test for COVID-19 is the reverse transcription polymerase chain reaction, or RT-PCR, that uses swabs taken from the nasopharynx and/or oropharynx. For patients with pneumonia, lower respiratory tract secretions may also be tested. The detection rate for each of these sites varies and may change during the course of illness. Yang et al (4) reported a sensitivity of throat samples of 60% at initial patient presentation, whereas the sputum sample had a higher sensitivity in patients with both severe (89%) and mild (82%) symptoms. However, the prevalence was unknown because the authors evaluated only confirmed cases. Chest CT is used to support COVID-19 diagnosis. The most common CT findings reported in COVID-19 pneumonia are bilateral subpleural areas of ground-glass opacity with or without consolidations affecting the lower lobes (5). In the intermediate phase of infection (4–14 days from symptom onset), a so-called “crazy-paving pattern” is seen. Peak radiologic abnormalities occur at around day 10, followed by gradual regression starting 2 weeks after symptom onset (6). In a retrospective study that appeared online in Radiology in February 2020 that used throat swab tests as the reference standard in 1014 consecutive Chinese patients suspected of having COVID-19 (mean age, 51 years ± 15), Dr Ai and colleagues (5) reported a 97% sensitivity and a 25% specificity for chest CT. The disease prevalence was notably high (59%), and had this same study been conducted in Italy, with its older population and greater spectrum of comorbid conditions (7), the prevalence and therefore the CT accuracy would have been different. Similarly, Dr Ai and colleagues would have obtained different results if sputum samples were used as a reference standard instead of the throat swab. These factors tend to undermine the external validity and generalizability of the performance of diagnostic tests in the context of the COVID-19 outbreak. In a quiet world, clinicians know how to use diagnostic tests. Decades of relatively slow research on well-known diseases allows for publication of clinical practice guidelines for patient management after a complex and long evidence-based process. However, the COVID-19 pandemic has been an ever-changing scenario. We need to learn from the start when to perform a test, how to combine results from different tests, and in which sequence to perform them. The scientific community is working hard to mitigate the outbreak and limit the number of deaths. By April 25, 2020, about 4 months from the beginning of this pandemic, more than 6200 studies on COVID-19 appeared on PubMed. Highly respected journals are certainly receiving hundreds of COVID-19–related submissions, trying to react with new and efficient review processes (8). But even the best available published research may be full of uncertainty and unknowns (9). Hope et al (10) argued that the article by Dr Ai and colleagues (5) (and others on the same topic) is flawed by suboptimal design, likely biased patient cohorts, and lack of a valid reference standard, which limit the generalizability of the results. Because findings at chest CT are not specific, CT may not allow for differentiation of COVID-19 from other forms of viral or nonviral atypical pneumonia (9). Thus, the authors suggest, as their title states, “Don’t rush the science.” Whereas we understand this point of view, we cannot forget that we are in an exceptional situation and some compromises must be accepted (9). In the context of a medical catastrophe, priority must be given to mitigate the pandemic consequences, even while knowing that solid and validated scientific evidence is not yet available. The 1986 story of the Challenger explosion, when information regarding the effect of a lower temperature was discarded because it was qualitative and incomplete, not quantitative and based on a large data set (11), is a lesson not to be forgotten. We do not know whether and when we will come back to our old, quiet world. However, in these current difficult times, a famous aphorism of uncertain origin (12) seems to apply: “In theory there is no difference between theory and practice but in practice there is.”