Antibody tests for identification of current and past infection with SARS‐CoV‐2

Abstract

Background

The severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) virus and resulting COVID‐19 pandemic present important diagnostic challenges. Several diagnostic strategies are available to identify current infection, rule out infection, identify people in need of care escalation, or to test for past infection and immune response. Serology tests to detect the presence of antibodies to SARS‐CoV‐2 aim to identify previous SARS‐CoV‐2 infection, and may help to confirm the presence of current infection.

Objectives

To assess the diagnostic accuracy of antibody tests to determine if a person presenting in the community or in primary or secondary care has SARS‐CoV‐2 infection, or has previously had SARS‐CoV‐2 infection, and the accuracy of antibody tests for use in seroprevalence surveys.

Search methods

We undertook electronic searches in the Cochrane COVID‐19 Study Register and the COVID‐19 Living Evidence Database from the University of Bern, which is updated daily with published articles from PubMed and Embase and with preprints from medRxiv and bioRxiv. In addition, we checked repositories of COVID‐19 publications. We did not apply any language restrictions. We conducted searches for this review iteration up to 27 April 2020.

Selection criteria

We included test accuracy studies of any design that evaluated antibody tests (including enzyme‐linked immunosorbent assays, chemiluminescence immunoassays, and lateral flow assays) in people suspected of current or previous SARS‐CoV‐2 infection, or where tests were used to screen for infection. We also included studies of people either known to have, or not to have SARS‐CoV‐2 infection. We included all reference standards to define the presence or absence of SARS‐CoV‐2 (including reverse transcription polymerase chain reaction tests (RT‐PCR) and clinical diagnostic criteria).

Data collection and analysis

We assessed possible bias and applicability of the studies using the QUADAS‐2 tool. We extracted 2x2 contingency table data and present sensitivity and specificity for each antibody (or combination of antibodies) using paired forest plots. We pooled data using random‐effects logistic regression where appropriate, stratifying by time since post‐symptom onset. We tabulated available data by test manufacturer. We have presented uncertainty in estimates of sensitivity and specificity using 95% confidence intervals (CIs).

Main results

We included 57 publications reporting on a total of 54 study cohorts with 15,976 samples, of which 8526 were from cases of SARS‐CoV‐2 infection. Studies were conducted in Asia (n = 38), Europe (n = 15), and the USA and China (n = 1). We identified data from 25 commercial tests and numerous in‐house assays, a small fraction of the 279 antibody assays listed by the Foundation for Innovative Diagnostics. More than half (n = 28) of the studies included were only available as preprints.

We had concerns about risk of bias and applicability. Common issues were use of multi‐group designs (n = 29), inclusion of only COVID‐19 cases (n = 19), lack of blinding of the index test (n = 49) and reference standard (n = 29), differential verification (n = 22), and the lack of clarity about participant numbers, characteristics and study exclusions (n = 47). Most studies (n = 44) only included people hospitalised due to suspected or confirmed COVID‐19 infection. There were no studies exclusively in asymptomatic participants. Two‐thirds of the studies (n = 33) defined COVID‐19 cases based on RT‐PCR results alone, ignoring the potential for false‐negative RT‐PCR results. We observed evidence of selective publication of study findings through omission of the identity of tests (n = 5).

We observed substantial heterogeneity in sensitivities of IgA, IgM and IgG antibodies, or combinations thereof, for results aggregated across different time periods post‐symptom onset (range 0% to 100% for all target antibodies). We thus based the main results of the review on the 38 studies that stratified results by time since symptom onset. The numbers of individuals contributing data within each study each week are small and are usually not based on tracking the same groups of patients over time.

Pooled results for IgG, IgM, IgA, total antibodies and IgG/IgM all showed low sensitivity during the first week since onset of symptoms (all less than 30.1%), rising in the second week and reaching their highest values in the third week. The combination of IgG/IgM had a sensitivity of 30.1% (95% CI 21.4 to 40.7) for 1 to 7 days, 72.2% (95% CI 63.5 to 79.5) for 8 to 14 days, 91.4% (95% CI 87.0 to 94.4) for 15 to 21 days. Estimates of accuracy beyond three weeks are based on smaller sample sizes and fewer studies. For 21 to 35 days, pooled sensitivities for IgG/IgM were 96.0% (95% CI 90.6 to 98.3). There are insufficient studies to estimate sensitivity of tests beyond 35 days post‐symptom onset. Summary specificities (provided in 35 studies) exceeded 98% for all target antibodies with confidence intervals no more than 2 percentage points wide. False‐positive results were more common where COVID‐19 had been suspected and ruled out, but numbers were small and the difference was within the range expected by chance.

Assuming a prevalence of 50%, a value considered possible in healthcare workers who have suffered respiratory symptoms, we would anticipate that 43 (28 to 65) would be missed and 7 (3 to 14) would be falsely positive in 1000 people undergoing IgG/IgM testing at days 15 to 21 post‐symptom onset. At a prevalence of 20%, a likely value in surveys in high‐risk settings, 17 (11 to 26) would be missed per 1000 people tested and 10 (5 to 22) would be falsely positive. At a lower prevalence of 5%, a likely value in national surveys, 4 (3 to 7) would be missed per 1000 tested, and 12 (6 to 27) would be falsely positive.

Analyses showed small differences in sensitivity between assay type, but methodological concerns and sparse data prevent comparisons between test brands.

Authors' conclusions

The sensitivity of antibody tests is too low in the first week since symptom onset to have a primary role for the diagnosis of COVID‐19, but they may still have a role complementing other testing in individuals presenting later, when RT‐PCR tests are negative, or are not done. Antibody tests are likely to have a useful role for detecting previous SARS‐CoV‐2 infection if used 15 or more days after the onset of symptoms. However, the duration of antibody rises is currently unknown, and we found very little data beyond 35 days post‐symptom onset. We are therefore uncertain about the utility of these tests for seroprevalence surveys for public health management purposes. Concerns about high risk of bias and applicability make it likely that the accuracy of tests when used in clinical care will be lower than reported in the included studies. Sensitivity has mainly been evaluated in hospitalised patients, so it is unclear whether the tests are able to detect lower antibody levels likely seen with milder and asymptomatic COVID‐19 disease.

The design, execution and reporting of studies of the accuracy of COVID‐19 tests requires considerable improvement. Studies must report data on sensitivity disaggregated by time since onset of symptoms. COVID‐19‐positive cases who are RT‐PCR‐negative should be included as well as those confirmed RT‐PCR, in accordance with the World Health Organization (WHO) and China National Health Commission of the People's Republic of China (CDC) case definitions. We were only able to obtain data from a small proportion of available tests, and action is needed to ensure that all results of test evaluations are available in the public domain to prevent selective reporting. This is a fast‐moving field and we plan ongoing updates of this living systematic review.

Plain language summary

What is the diagnostic accuracy of antibody tests for the detection of infection with the COVID‐19 virus?

Background

COVID‐19 is an infectious disease caused by the SARS‐CoV‐2 virus that spreads easily between people in a similar way to the common cold or ‘flu. Most people with COVID‐19 have a mild to moderate respiratory illness, and some may have no symptoms (asymptomatic infection). Others experience severe symptoms and need specialist treatment and intensive care.

The immune system of people who have COVID‐19 responds to infection by developing proteins that can attack the virus (antibodies) in their blood. Tests to detect antibodies in peoples' blood might show whether they currently have COVID‐19 or have had it previously.

Why are accurate tests important?

Accurate testing allows identification of people who might need treatment, or who need to isolate themselves to prevent the spread of infection. Failure to detect people with COVID‐19 when it is present (a false negative result) may delay treatment and risk further spread of infection to others. Incorrect identification of COVID‐19 when it is not present (a false positive result) may lead to unnecessary further testing, treatment, and isolation of the person and close contacts. Correct identification of people who have previously had COVID‐19 is important in measuring disease spread, assessing the success of public health interventions (like isolation), and potentially in identifying individuals with immunity (should antibodies in the future be shown to indicate immunity).

To identify false negative and false positive results, antibody test results are compared in people known to have COVID‐19 and known not to have COVID‐19. Study participants are classified as to whether they are known or not known to have COVID‐19 based on criteria known as the ‘reference standard’. Many studies use samples taken from the nose and throat to identify people with COVID‐19. The samples undergo a test called reverse transcriptase polymerase chain reaction (RT‐PCR). This testing process can sometimes miss infection (false negative result), but additional tests can identify COVID‐19 infection in people with a negative RT‐PCR result. These include measuring clinical symptoms, like coughing or high temperature, or ‘imaging’ tests like chest X‐rays. People known not to have COVID‐19 are sometimes identified from stored blood samples taken before COVID‐19 existed, or from patients with respiratory symptoms found to be caused by other diseases.

What did the review study?

The studies looked at three types of antibody, IgA, IgG and IgM. Most tests measure both IgG and IgM, but some measure a single antibody or combinations of the three antibodies.

Levels of antibodies rise and fall at different times after infection. IgG is the last to rise but lasts longest. Levels of antibodies are usually highest a few weeks after infection.

Some antibody tests need specialist laboratory equipment. Others use disposable devices, similar to pregnancy tests. These tests can be used in laboratories or wherever the patient is (point‐of‐care), in hospital or at home.

We wanted to find out whether antibody tests:

‐ are accurate enough to diagnose infection in people with or without symptoms of COVID‐19, and

‐ can be used to find out if someone has already had COVID‐19.

What did we do?

We looked for studies that measured the accuracy of antibody tests compared with reference standard criteria to detect current or past COVID‐19 infection. Studies could assess any antibody test compared with any reference standard. People could be tested in hospital or the community. Studies could test people known to have – or not to have – or be suspected of having COVID‐19.

Study characteristics

We found 54 relevant studies. Studies took place in Asia (38), Europe (15), and in both USA and China (1).

Forty‐six studies included people who were in hospital with suspected or confirmed COVID‐19 infection only. Twenty‐nine studies compared test results in people with COVID‐19 with test results in healthy people or people with other diseases.

Not all studies provided details about participants’ age and gender. Often, we could not tell whether studies were evaluating current or past infection, as few reported whether participants were recovering. We did not find any studies that tested only asymptomatic people.

Main results

Our findings come mainly from 38 studies that provided results based on the time since people first noticed symptoms.

Antibody tests one week after first symptoms only detected 30% of people who had COVID‐19. Accuracy increased in week 2 with 70% detected, and was highest in week 3 (more than 90% detected). Little evidence was available after week 3. Tests gave false positive results in 2% of those without COVID‐19.

Results from IgG/IgM tests three weeks after symptoms started suggested that if 1000 people had antibody tests, and 50 (5%) of them really had COVID‐19 (as we might expect in a national screening survey):

‐ 58 people would test positive for COVID‐19. Of these, 12 people (21%) would not have COVID‐19 (false positive result).

‐ 942 people would test negative for COVID‐19. Of these, 4 people (0.4%) would actually have COVID‐19 (false negative result).

If we tested 1000 healthcare workers (in a high‐risk setting) who had had symptoms, and 500 (50%) of them really had COVID‐19:

‐ 464 people would test positive for COVID‐19. Of these, 7 people (2%) would not have COVID‐19 (false positive result).

‐ 537 people would test negative for COVID‐19. Of these, 43 (8%) would actually have COVID‐19 (false negative result).

We did not find convincing differences in accuracy for different types of antibody test.

How reliable were the results of the studies of this review?

Our confidence in the evidence is limited for several reasons. In general, studies were small, did not use the most reliable methods and did not report their results fully. Often, they did not include patients with COVID‐19 who may have had a false negative result on PCR, and took their data for people without COVID‐19 from records of tests done before COVID‐19 arose. This may have affected test accuracy, but it is impossible to identify by how much.

Who do the results of this review apply to?

Most participants were in hospital with COVID‐19, so were likely to have more severe disease than people with mild symptoms who were not hospitalised. This means that we don't know how accurate antibody tests are for people with milder disease or no symptoms.

More than half of the studies assessed tests they had developed themselves, most of which are not available to buy. Many studies were published quickly online as ‘preprints’. Preprints do not undergo the normal rigorous checks of published studies, so we are not certain how reliable they are.

As most studies took place in Asia, we don't know whether test results would be similar elsewhere in the world.

What are the implications of this review?

The review shows that antibody tests could have a useful role in detecting if someone has had COVID‐19, but the timing of when the tests are used is important. Antibody tests may help to confirm COVID‐19 infection in people who have had symptoms for more than two weeks and do not have a RT‐PCR test, or have negative RT‐PCR test results. The tests are better at detecting COVID‐19 in people two or more weeks after their symptoms started, but we do not know how well they work more than five weeks after symptoms started. We do not know how well the tests work for people who have milder disease or no symptoms, because the studies in the review were mainly done in people who were in hospital. In time, we will learn whether having previously had COVID‐19 provides individuals with immunity to future infection.

Further research is needed into the use of antibody tests in people recovering from COVID‐19 infection, and in people who have experienced mild symptoms or who never experienced symptoms.

How up‐to‐date is this review?

This review includes evidence published up to 27 April 2020. Because a lot of new research is being published in this field, we will update this review frequently.