For one thing, they’ll have different levels of protection overall. The Pfizer study covered 43,538 people, 38,955 of whom got the two doses required. And the company reported only 94 cases of COVID-19. (Presumably many more of the infected were in the placebo group that didn’t get the actual vaccine; the release didn’t give the exact numbers.) But it’s probably not the case that nine times as many non-vaccinated people got the disease as vaccinated people; those numbers vary based on how many people were in each group. And that math is supposed to include a statistical range of possibility called a confidence interval. It’s sort of the possible answers the same data could account for. The smaller the confidence interval, the more sure that researchers are about a number.
People are going to want those, because they’ll make it easier to compare vaccines. Moderna spokespeople have issued a statement saying they’re about to release their initial data too. What if they say their efficacy is higher than 90 percent? Without a confidence interval, it’ll seem like Moderna’s drug is better than Pfizer’s. But if the confidence intervals of the drugs overlap, people will know that they’re actually closer to each other in terms of effectiveness.
“If Moderna comes out and it’s a 93 percent, let’s say, now we have to unwind the messaging on this. Are these numbers the same? Are they different?” says Peter Bach, director of the Center for Health Policy and Outcomes and the Drug Pricing Lab at Memorial Sloan Kettering Cancer Center. “Sure, it’s a big effect. It’s unmistakable. But because the actual numbers are pretty small, they’re just very unstable.” Confidence intervals will make the public health messaging easier later, when other vaccines come out—especially since nobody’s actually comparing these vaccines head-to-head.
Trial and errors
Comparisons will only happen after the fact, almost inferentially. “It’s very hard to do head-to-head comparisons, in part because nobody’s interested. The companies, unless they see a commercial advantage, aren’t going to do it,” says Arnold Monto, an epidemiologist at the University of Michigan School of Public Health who chairs the FDA’s Vaccines and Related Biological Products Committee. “This information is going to come from observational studies after vaccines are being used.”
At an October meeting of the FDA vaccines committee that Monto chairs, he worried that a too-early reveal of data from any of the companies might induce participants in the study to bail out. When volunteers consent to be part of a trial, they often get the right to leave the study and find out whether they received the vaccine or the placebo. As soon as one seems to work, everyone in every study might, potentially, want to just go get that—torpedoing all the other work in progress. Monto now says he’s a little less worried about that. “It may all be moot, because the trials are large in size,” Monto says. “If people do drop out, it may not really matter in terms of the long-term study of both efficacy and safety.”
In fact, having a working vaccine could, in some interpretations, mean that ethically other companies can’t keep testing their own candidates—exposing people to COVID-19 when, nominally, an actual vaccine exists. That doesn’t seem to be happening either. The other trials are still ongoing. Vaccine makers know how to compare different vaccines against the same disease in “noninferiority” studies. They already do that for influenza shots. COVID-19 vaccines might have entirely different “correlates of protection,” the thing that’s actually conferring immunity. Some might induce more neutralizing antibodies, the immune system’s general infantry, while others might induce more T cells, a more specifically targeted fighter.
“Almost a miracle”
People need vaccines to do different things—to work better on old people or on kids, or to stave off different sets of symptoms. Pfizer’s end points, the things they were actually studying, were bad side effects (which they didn’t see) and a lack of mild symptoms or disease. That seems good, but it’s not the whole story.
“How does that correlate with emergence of symptoms severe enough that they require medical attention? Hospitalization? Mortality? Long-haul symptoms?” asks Llew Keltner, a drug development expert and CEO of Epistat, a health care strategy consultancy. “If we’re preventing hospitalization, preventing death, and preventing long-haul disease at 9 to 1? Oh my God, it’s almost a miracle.” Nobody knows if that’s what the Pfizer vaccine does.
For that matter, a vaccine that has lower efficacy but only requires one dose might be better. Getting two shots is no fun, and it requires an informational infrastructure to track who has gotten what and when. A single shot makes everything easier, even if it works slightly less well. “Maybe we give up a little bit in efficacy,” Bach says, “but true efficacy in the real world with Pfizer and Moderna is hampered, because it’s very hard to get people to return.”
So that’s yet another reason the Pfizer vaccine will encourage rather than block development of competitors. The market, so to speak, isn’t uniform. Two key pieces of information about any vaccine simply aren’t available yet—how long its protection lasts, and whether it prevents not only illness but actual transmission of the disease from person to person.
“I am not concerned about long-term protection initially, because we’ve got a pandemic to deal with. But I think, down the road, duration of protection is going to be very important, given the fact that the virus is not going to go away,” Monto says. “I’ve been around a number of rollouts of vaccines, and the question is always, is this lifelong protection? And we never know the answer until time passes. It’s going to be very difficult to find out, and it’s going to be different for different vaccines.”
We don’t know “longer-term health consequences”
If a vaccine can’t prevent transmission, that may likewise open a door to other candidates. A vaccine that’s 90 percent effective at preventing sickness, in conjunction with all the familiar public health “non-pharmaceutical interventions”—masks, distancing, testing, and contact tracing—might be enough to bring the US back to a new normal by late summer. But it wouldn’t be enough to allow US citizens to travel anywhere that doesn’t have the vaccine. In those places, a vaccinated person exposed to the disease and able to transmit it is just an asymptomatic superspreader.
“In terms of understanding the epidemiological consequences of the vaccine, we don’t know. The longer-term health consequences, we don’t know,” Scarpino says. “The degree to which this actually blocks transmission is going to dictate the kinds of non-pharmaceutical interventions we’ll have to layer with it.”
Even if Pfizer gets some kind of authorization to start giving people shots, as seems likely, the world will still need other COVID-19 vaccines, too. “That was the goal, to not put all our eggs in one basket. In many ways, that’s a good problem to be talking about—how the market will absorb many vaccine candidates. That’s not new to health care,” Dopp says. “Right now, there’s not enough information that would give any sort of picture to that. Right now, people are just wanting to vaccinate with an effective vaccine.” That ember of hope burns on, even with cold water raining down upon it.
This story originally appeared on wired.com.
