(Guide) The disappointing truth about antibody testing

[Karlston]

The disappointing truth about antibody testing

There’s still a lot we don’t know about COVID-19

Illustrations by Alex Parkin and Grayson Blackmon

DidDid you get seriously sick this year? Have you, like many others, been wondering if it was COVID-19? “I get emails every day,” says George Rutherford, a University of California San Francisco epidemiologist. They say things like, “I think I might have had it. I really want to know,” he says.

Maybe you’ve been tempted by the wave of antibody tests that recently hit the market — but don’t think that just because you can buy a test, it will be accurate.

 

There are lots of tests available, but it’s not entirely clear which ones are best. Tests can go wrong in several places: by choosing the wrong proteins to look for, by using the wrong chemicals in the test itself, or by making the test badly. For instance, the UK paid $20 million for antibody tests that didn’t work.

 

The US Food and Drug Administration allowed test makers to market their antibody tests, even if they hadn’t undergone review by the agency. The Association of Public Health Laboratories (APHL) has raised concerns about this: “We now have at least 90 tests on the market, and we don’t know about the accuracy of the results,” Kelly Wroblewski, APHL’s director of infectious disease programs, told USA Today.

 

“It’s kind of like the wild West,” says Janko Nikolich-Zugich, the head of the University of Arizona College of Medicine-Tucson’s department of immunobiology. One of the reasons the University of Arizona wound up making its own antibody test was because it wasn’t clear how reliable other ones might be, he says.

 

“It’s very attractive to think about, ‘I’m going to go to a drive-through and get a finger prick and I’ll get an answer,’” says Patricia Slev, the section chief for immunology at ARUP Laboratories, a national reference lab. But most of these rapid tests don’t have independent validation that the test works the way the manufacturer says. Consumers who want to make sure they’re getting good antibody tests should look for ones that have outside testing to confirm the test is accurate.

 

That relaxed policy has since been revised. On May 4th, the FDA said that if test makers didn’t submit data showing their accuracy within 10 days, the agency would yank the tests from the market. “We unfortunately see unscrupulous actors marketing fraudulent test kits and using the pandemic as an opportunity to take advantage of Americans’ anxiety,” the FDA statement read.

 

Several labs, including Slev’s ARUP, are now doing independent validation of antibody tests. Already, there are some early results: a manuscript that hasn’t yet been peer-reviewed or published from a group of scientists led by the University of California San Francisco and the University of California, Berkeley. Only three of the 14 tests the group evaluated were reliable. Four of the tests had false positive rates that ranged from 11 to 16 percent. “Those numbers are just unacceptable,” Scott Hensley, a microbiologist at the University of Pennsylvania, told The New York Times. “If your kit has 14 percent false positive, it’s useless.”

 

“I am waiting for the ultimate good test,” says Robert Gallo, the co-founder and director of the Institute of Human Virology at the University School of Medicine and the co-founder of the Global Virus Network. Gallo helped develop the antibody test for HIV.

 

And if you’ve been pinning your hopes for getting back to normal on antibody testing that would “certify” people as safe to work or travel if they’ve already been sick, I have bad news: The World Health Organization (WHO) has warned against “immunity passports.” Even if the tests were 100 percent accurate, we still don’t know whether antibodies to the new coronavirus will protect people from being reinfected. If they do confer protection, we don’t know whether it’s full or partial or how long it’ll last. Even skipping the process questions — What authority would issue these, and to whom? — this seems like a bad idea.

 

Chile is preparing to put out “release certificates” for people who’ve recovered from COVID-19, despite the WHO’s warning. Other countries, such as the UK, have also expressed an interest — as has Los Angeles’ mayor.

 

Some experts have expressed concerns that people will rush to get “immunity passports” without understanding the limitations of antibody testing. “What I worry about is that employers will misuse this,” Rutherford says. It may be true that accurate antibody testing could let us know who’s protected from COVID-19, but that hasn’t been proven yet, Rutherford says.

 

To understand why these tests may not be the solution everyone’s hoping for, I spoke to some experts about how antibodies work, why false positives are important, and what real uses for antibody testing might be.

What is an antibody test?

An antibody test is a way of seeing if a person’s immune system has responded to something. In the case of COVID-19, it’s a way of seeing who’s had the new coronavirus, which is different from the tests used to diagnose the illness. Though antibody tests can be used to diagnose diseases, in the case of COVID-19, they’re not very helpful for diagnoses. That’s because people don’t make antibodies to fight off the virus until at least a week — or even two weeks — after showing symptoms. Tests used for diagnosis look for virus fragments and can spot infections much earlier.

How do antibodies work?

Antibodies are part of how your body responds to threats: viruses, bacteria, and other things the body decides are harmful. Imagine someone coughs on you and infects you with the new coronavirus. The virus can’t reproduce on its own, so it invades your cells to make copies of itself. Eventually, your immune system cottons on. Here’s what happens:

⤷ Some specialized immune cells shred the virus and present its fragments to a kind of white blood cell called a B cell.

⤷ When the fragments hit the B cell, it clones itself and transforms into an antibody factory.

⤷ The antibodies the B cell creates are then released into the bloodstream.

⤷ The antibodies vary in quality: some may glom on to the virus and inactivate it.

⤷ Some may stick to the virus and signal other immune cells to come kill the virus.

⤷ Some will do nothing. And sometimes, antibodies can make things worse. How effective antibodies are will vary by person and disease.

 

In HIV, for instance, antibodies are just a marker of infection. Because the virus attacks the immune system, the antibodies don’t do a great job of fighting off the virus, says Rutherford. The majority of antibodies for most illnesses are neither good nor bad for you, says Gallo. They’re just a marker that you’ve been exposed to an illness. “A good majority of them are irrelevant for protection of you, and irrelevant for how the disease is progressing,” Gallo says.

 

It’s also possible that they can inactivate the virus — for instance, by gumming up the spike the virus uses to insert itself into your cells; these antibodies are called “neutralizing antibodies” and are the most important kind for protection. They “are what you need for vaccines to work,” says Rutherford. If we do produce neutralizing antibodies when we’re infected by the new coronavirus, it might be possible to pick one that works really well and make a version in labs that can be used to treat the illness.

 

It’s possible to test for neutralizing antibodies, but it’s expensive and difficult to do at a mass scale. “It’s a very labor-intensive laboratory thing,” Rutherford says. The commercially available tests don’t do this. Most of them just give people a “yes” or “no” on whether the antibodies exist.

 

Neutralizing antibodies aren’t the only way antibodies might help us, Gallo says. Some antibodies serve as signal flares by grabbing on to a different part of the virus and alerting the immune system’s killer cells. But in some cases, such as dengue, antibodies can actually make a later infection worse.

 

Unfortunately, it’s not yet clear what antibodies do in COVID-19. For instance, the prevalence of antibodies doesn’t mean that the amount of virus in a person’s system declines, according to a study of German patients. And the less virus in your body, the better.

If I have antibodies to the new coronavirus, does that mean I won’t get sick again?

We don’t know. The virus is too new. The question of reinfection is “an unknown,” according to Mike Ryan, executive director of WHO’s emergencies programs.

 

Some Chinese, South Korean, and Japanese patients who left the hospital after testing negative for the virus were later readmitted and tested positive for the virus again. We don’t yet know why, but other coronaviruses usually can’t reinfect people for months or even years. It’s possible that this is the result of false negatives on their tests. It’s also possible that people continue to excrete viral fragments that show up in testing after the disease has passed. Rectal swabs and stool culture remain positive for about a month, “and that has nothing to do with transmission,” Rutherford says. “It’s just pieces of digested virus.”

 

An early draft of an article supports the idea that these patients weren’t reinfected. Chinese scientists infected two rhesus monkeys with COVID-19, then let them recover. Thirty days later, the scientists tried to infect the monkeys again. The monkeys didn’t get sick a second time. This article hasn’t been finalized, though; it’s been posted to let other scientists critique it. The final version may change.

 

There are some encouraging signs that antibodies may confer protection. One is serum therapy where parts of the blood that include antibodies are spun out from survivors of COVID-19 infections and given to people who are sick. In a study of 10 patients, being given survivors’ serum “significantly improved” those patients’ symptoms. That study is too small to be definitive, but it does suggest that larger trials of serum therapy might be a good idea.

Does it matter how many antibodies you have?

Again, we don’t know. It’s still too early to know what might serve as signposts for protection. It’s also too early to know what level of antibodies might signal a person who’s recovered from COVID-19 would be a good serum donor.

 

The total amount of antibodies to COVID-19 in patients’ blood varies widely, according to an early report from Fudan University in Shanghai. (The paper was also put online for the research community to comment on and is not yet finalized.) Younger patients had fewer antibodies, and in 10 young patients who’d tested positive for the virus and then recovered, researchers didn’t detect any neutralizing antibodies at all. About a third of patients showed low levels of antibodies. It’s possible that’s because other parts of the immune system attacked and defeated the coronavirus — but antibodies and the cells that produce them are the keys to remaining protected. The low levels of antibodies here also may make it difficult to interpret population-level data; if someone was infected but didn’t produce antibodies, will we get an accurate count of cases?

 

In other coronaviruses — the ones that cause the common cold — the way people form antibodies seems to vary by age: people over the age of 60 are better at forming antibodies that neutralize the virus as well as ones that signal to the immune system’s killer cells.

 

There’s another way antibody testing might be useful, but it’s pretty far off. We know most vaccines work by generating an antibody response, says Nikolich-Zugich. Antibody testing might help drugmakers figure out whether their vaccines against COVID-19 work. In fact, we already do this with the flu vaccine. We know that a certain level of antibodies means a flu shot is effective. For this to be useful in coronavirus, though, we’d have to have a reliable test that indicates a protective immune response and some sense of what that looks like. Learning those things will take time, and we don’t know in advance how long it will be.

How long does protection by antibodies last?

As a reminder: we don’t know yet whether COVID-19 antibodies confer protection. If protection exists, we also don’t know how long it might last. The virus is too new for us to have long-term data. We know, in some diseases — such as HIV — the most-protective antibodies are also the ones that disappear the quickest, says Gallo.

 

Sometimes, the body forms antibodies to a disease that last a lifetime, like it does with measles. But other times, the immune system seems to “forget” diseases. Take SARS, another coronavirus disease: three years after infection, about a quarter of people didn’t have any detectable antibodies in their blood. Six years after infection, most people didn’t have any. In a small study of patients who’d recovered from MERS — also caused by a coronavirus — people who got very sick had detectable antibodies to the disease two years later, but people with milder illnesses didn’t.

 

The limited antibody response may be because coronaviruses aren’t in the bloodstream, Rutherford says. “They’re not really exposed to the full force of the immune system. It’s a slightly different kettle of fish.” Instead, the virus attacks the lungs, liver, kidneys, and some other organs.

 

The coronaviruses we know best — the ones that cause the common cold — can reinfect people after a certain period of time. A study from the 1970s showed that six people who’d had colds after being exposed to one strain of coronavirus were immune when they were reexposed a year later. But another 12 people in the same study who were exposed to a slightly different strain were only partially protected. A tiny study from the 1990s suggests that coronavirus reinfections are milder than the original infection. The coronaviruses that are closest to the one that causes COVID-19 only create immune protection for about a year, according to a modeling study. That may be a clue to how the new coronavirus behaves.

 

But Slev cautioned against assuming too much about this coronavirus based on other coronaviruses. “There are assumptions [you can make] by looking at other coronaviruses, but really, you want to know the answer for this specific one,” she says. “So really, it’s important to wait to know what the answer is for this virus and we hope to have that answer soon. But right now, we do not have that particular answer for COVID-19.”

Why are antibody tests useful?

The first known death from COVID-19 in the US was on February 6th, in Santa Clara County, California, but the connection to the virus wasn’t made until April. Overall deaths were up 20 percent in March in Santa Clara County from the previous year. California coroners are now examining deaths from as early as December to see if there were more missed cases.

 

The evidence from California suggests COVID-19 was circulating in the US well before we realized it — and that some number of people may have been sick with the disease without knowing it. We don’t know how widely it was spreading or for how long because we didn’t start testing for the virus early enough. Testing people to see if they have antibodies to the virus may give health officials a better picture of how far the pandemic spread.

 

Right now, most people who receive diagnostic testing are allowed to because they feel sick. But we know some infections from the novel coronavirus don’t come with symptoms — and those people are being missed by most of our testing efforts. “Nobody knows what the rate of infection is,” Slev says. “An immunologic study could give us an idea on the true rates of infection in the community.”

 

Even after the testing criteria changed, though, there still weren’t enough tests available. Widespread testing of the population for the virus hasn’t occurred. Tests have been reserved, instead, for some of the sickest patients. We also know some infections by the new coronavirus are asymptomatic. All that means our confirmed case counts are low, but we don’t know exactly how low. Reliable antibody testing might give us a better picture of how much of the population has already been infected, which might give us a better sense of how quickly the virus spreads.

 

This kind of work can help epidemiologists figure out how much of the population remains vulnerable to COVID-19, how often asymptomatic cases occur, and exactly how deadly the disease is. “We have suspected that the number of undocumented cases is much, much higher than what has been documented by testing for the virus itself,” says Nikolich-Zugich. Antibodies to the virus don’t fade as quickly as the virus does in the body, and they might give us more accurate information on how widely the disease has spread.

Has this kind of antibody testing been done already?

Yes, in a few locations — though doctors have reservations about how accurate the tests are. In a Boston suburb, a sample of people tested on the street showed that about a third of the 64 people who were tested had been exposed to the virus. In a hard-hit German town, about 15 percent of the 200 people tested had antibodies. Those findings are unusual, though; most tests show that less than 5 percent of the population has antibodies.

 

For instance, early estimates from Santa Clara County, California, suggest that about 1.5 percent of the population had antibodies. If that’s right, the infection rate is 50 to 85 times higher than the official count for coronavirus cases. In Los Angeles County, there might be 28 to 55 times more people infected if those antibody tests are right. In New York, the antibody testing suggests 10 times more people were infected than the testing numbers reflect. These attempts to figure out how many people were sick used different methods and different tests.

 

Both the California studies relied on tests from Premier Biotech, which may have a false positive rate as high as 1.7 percent. That’s a problem since Santa Clara’s study found antibodies in 1.5 percent of the people they surveyed. “Literally every single one could be a false positive,” Marm Kilpatrick, an infectious disease expert at the University of California at Santa Cruz, told BuzzFeed News. “No one thinks all of them were, but the problem is we can’t actually exclude the possibility.”

 

Experts have fewer reservations about the New York testing numbers, which also showed that about 20 percent of New York City’s residents had been exposed to COVID-19. Several experts had suggested that the confirmed case numbers were 2 to 20 times too low; these results seem to confirm that. The New York tests also used a different antibody test. And while this test will also produce errors, it’s been done in a place with a much higher rate of infections, making the results less likely to be skewed by false positives. There is a sampling problem here, though: since people were tested in grocery store parking lots, any parts of the population that don’t go out — for instance, because they are unusually vulnerable to serious illness from COVID-19 — have been missed.

 

It’s still too early to tell how many people had COVID-19 and didn’t get counted in the official numbers. You should expect to see a bunch more fights between scientists about which studies have good methodology and which ones don’t. (There is a lot of arguing in science, actually.) More reliable data will take time and will come from areas that have been hit harder by the disease.

 

There have been around 30 attempts at seeing how widespread the virus is, but despite the variations in methodology and fights about accuracy, they broadly show the same thing: most of the population is still vulnerable to COVID-19.

What is a false positive, and why does it matter?

A false positive is what it sounds like: a person who didn’t have a COVID-19 infection but whose test results said they did. In order to determine how likely a test is to produce a false positive, most laboratories use specimens collected from people who have a lot of virus, and people they know haven’t been exposed (for instance, because the sample was taken before December 2019). Test manufacturers often report their “analytic” sensitivity from these samples, which may be more clear-cut than testing in the wild. Independent third-party testing usually leads to a more reliable sense of error rates.

 

The tests also need to accurately distinguish between people who had the virus that causes COVID-19 and people who’ve had other kinds of coronaviruses. It’s possible that people who’ve recently had one of the four viruses that cause the common cold might test positive for COVID-19 when they’re actually negative, the WHO warns. It’s also possible that people who had SARS or MERS will trigger a positive result.

 

In the case of antibody testing, false positives matter partly because they’ll give us a wrong picture of how many people have been sick — which could make for bad public health policy. But they especially matter if a person who’s gotten a false positive believes they’re immune and chooses to disregard social distancing. That person is still at risk of getting sick, even if they don’t know it.

 

Imagine if we were using these tests to put people in groups based on their antibody status, says Rutherford. For simplicity’s sake, imagine a test where 4 percent of results are false positives, and a workforce of 100 people is in a hospital. “You have four people waltzing around saying, ‘Oh, I’m positive, I can’t get it, I don’t need to wear a mask in the ICU,’” Rutherford says. “Au contraire. That’s bad, dangerous information.”

 

“We can’t accept an extreme number of false positives,” Gallo says. He thinks a 5 percent false positive rate is “on the high side.” “Something is better than nothing right now, yes, but you can get misleading results.”

 

It’s also possible to get a false negative result, which suggests that you haven’t had COVID-19 even though you were infected. While this also might lead to bad public health decisions, it’s less likely to put a person at risk.

Is it possible to create an antibody test that looks for the right coronavirus?

There are some strategies to make sure that the antibody tests look for the right coronavirus, says Nikolich-Zugich. For instance, the spike protein that the new coronavirus uses to get into human cells “is quite unique in this particular coronavirus relative to the others,” he says.

 

Looking for antibodies to that protein could help discriminate between coronaviruses. (But if other coronaviruses do trip up testing, that might not be a bad thing clinically, says Nikolich-Zugich. It might mean that having a cold recently could protect you from COVID-19; after all, people who are vaccinated against smallpox are also protected against monkey pox, which is caused by a similar virus. In that sense, cross-reactivity would be “a welcome occurrence.”)

If there are all of these uncertainties around coronavirus and antibody testing, what is it good for?

Well, the uncertainties aren’t permanent! Again, we’re just starting to know this virus. Several months from now, we’ll have more information, and years from now, even more.

 

This is also true of antibody tests, antibodies to coronavirus, and immunity. If we are willing to tolerate this period of uncertainty, we’ll give scientists time to do their jobs. That may mean telling us which tests are useful and which ones aren’t. It might also mean that we keep track of how many people test positive for coronavirus antibodies at a mass scale, rather than as individuals.

 

The thing people are hoping for is called “herd immunity.” That means that enough people have formed an immune response that the virus doesn’t move through the population like a hot knife through butter. There have been various estimates about how much of the population that would be — some as low as 60 percent and some as high as 90 percent — but no one’s really sure yet because we don’t know how long people remain protected after an infection or even if they are protected at all.

At this point, even our contested antibody testing shows the same thing: we’re not close to herd immunity — not by a long shot. So that’s one immediate use.

Part of A guide to the COVID-19 pandemic

 

Source: The disappointing truth about antibody testing (The Verge)