How do you tell if an antibody test is effective? Shouldn’t this be a simple process?
In medical testing, it’s not always black and white, positive or negative: a test’s ability to read specific pieces of information can vary. The latest place where this becomes important is in the antibody or blood (serology) testing for presence of antibodies to coronavirus.
Two phrases are key to understanding a large part of how testing results may not give straightforward answers: specificity and sensitivity. Many different types of medical tests, not just coronavirus tests, use this terminology, and it’s key to help doctors and epidemiologists understand how patients and populations are being infected.
First, there’s sensitivity. If a test is highly sensitive, that means that it is very good at reading positive results. A test with a 100% sensitivity rate means that if you get a positive result, doctors can be confident that you’ve been infected.
Then, there’s specificity, which is a test’s ability to measure negative results. A test with a 100% specificity rate tells doctors that any negative results are very reliable.
Polymerase chain reaction (PCR) tests, which are used to analyze active coronavirus infections with nasal swabs, can also be measured in terms of sensitivity and specificity. Those tests are considered “the gold standard” for detecting viruses, and are more sensitive than the blood testing used to detect antibodies. Experts say inaccurate sample collection and handling can increase the margin of error for these tests.
Looking at the Food and Drug Administration page for different tests approved by the agency, you can begin to see how important these two terms are for figuring out why some tests can be more reliable for some sets of results and not others.
Take the test developed by Mount Sinai Hospital in New York. According to the FDA page, the test has a 100 percent specificity rate. If you get a negative result on this test, doctors can be confident you don’t have antibodies. But the test only has a 92.5% sensitivity rate, meaning that there’s a significant chance of false positives.
For coronavirus, any false positives when it comes to antibody tests are extremely dangerous – imagine if someone is told that they have antibodies with a false positive test and then chooses not to not wear a mask or stay at home.
Understanding these gradients of results is important when it comes to comparing different brands of tests as well. For instance, the antibody test made by the pharmaceutical giant Abbott has a 100 percent sensitivity rate and a 99.6 percent specificity rate – a pretty accurate test by that measure in both directions, although there is some slight possibility for false positives to slip through. But the Cellex test only has a 93.8% sensitivity rate and a 96% percent specificity rate, meaning both false positives and false negatives can arise from this test.
Quartz has developed a handy simulator for telling how sensitivity and specificity can affect specific populations – and how one little percentage point can make a huge difference.
If 5 percent of the population is infected, per Quartz’s model, the Cellex test will correctly identify 918 people out of 1,000 that don’t have antibodies (and incorrectly tell 5 people with antibodies that they don’t have them). More alarmingly, the Cellex model correctly identifies 45 people with antibodies – but gave 32 false positives. In real life, of the 77 people the test gave positive results to, more than 41 percent would walk away thinking they were immune when they weren’t.
“It is critical to remember that serosurveys are population-level surveys,” Natalie Dean, who studies biostatistics at the University of Florida, wrote on Twitter. “They are intended to inform our broader understanding of the disease, not to tell individuals whether or not they have been infected. The test is still too unreliable for the latter.”
Resources
If you’re interested in learning more about the accuracy of antibody tests, here’s a couple helpful lectures, explainers, and tip sheets:
- A video lecture from MIT professor Michael Cima
- An explainer from ProPublica
- A panel webinar from Columbia University; the part about testing starts around minute 31.
- The Quartz simulator mentioned above
- A crowdsourced list of where to get antibody testing and donate plasma in New York from Reddit
