As of this writing, there are about 40 Coronavirus vaccines in the clinical trial process, plus another 135 in preclinical development. Russia has jumped the gun and “approved” a vaccine that has just begun Phase 3 trials, and, likewise, China has approved a pre-Phase 3 vaccine for use with their military. The vaccine approval process has entered the geopolitical realm, but understanding of the clinical trial process is limited, as shown by the way the New York Times mangled its description of the FDA vaccine approval bar:
“The FDA said that a coronavirus vaccine would have to protect at least 50% of vaccinated people to be considered effective.”
Of course, even as cumulative infection rates climb, they will be nowhere near 50% during the vaccine trial period. The current confirmed cumulative infection rate in the U.S. is 2.7%. So well over 50% of those taking the vaccine would be “protected” out of the gate, since they would never get Covid in the first place. What the Times meant to say was:
“For a vaccine to be approved, those vaccinated must get 50% fewer infections than those not vaccinated.”
Let’s look at the vaccine approval process and the different phases of clinical trials. Both are pretty consistent across different countries.
Preclinical Phase
The preclinical phase belongs to virologists and biochemists. Vaccine candidates are currently being developed from inactive viruses, genetically engineered viruses, and messenger RNA (mRNA) that delivers proteins to cells. The goal is to determine whether the vaccine candidate triggers an immune response. Testing is conducted on animals.
Phase 1 Trial
Candidate vaccines that produce immune responses in animals are then tested on a small group of human subjects to ascertain safety and to explore different dosages. The immune response must be established for different dosages.
There are currently 19 vaccine candidates in Phase 1 trials.
Phase 2 Trial
After the candidate passes the Phase 1 hurdles, it goes on to Phase 2, where it is tested with larger groups of human subjects, typically several hundred, to confirm safety and the generation of an immune response. The Phase 2 subject pool must adequately represent the groups the vaccine is targeting, and subgroups of interest are often upsampled.
There are currently 12 vaccine candidates in Phase 2 trials.
Phase 3 Trial
Phases 1 and 2 do not directly measure the effectiveness of the vaccine – they only establish that it generates an immune response. The real proof of effectiveness lies in whether vaccine recipients contract Coronavirus infections (and, secondarily, how serious they are). This is the role of large-scale Phase 3 trials.
There are currently 8 vaccine candidates in Phase 3 trials.
Phase 3 vaccine trials need large samples and extended timelines
Phase 3 vaccine trials need much larger sample sizes than drug trials, which can recruit sick patients at the start. Vaccine trials must recruit healthy subjects and wait for them to catch the virus. The current reported infection rate in the U.S. is only 2.7% of the population. You could recruit, say, 500 subjects and only 15 of them will get the virus. That is far too small a number to determine whether the 250 subjects in the treatment group will fare better than the 250 in the placebo group. You need numbers in the tens of thousands and, ideally, locations that have higher infection rates.
Skipping Phase 3 trials, like China and Russia have done, is a big risk, for two reasons:
- The sponsors still have no direct evidence that their vaccine works, only that it produces an immune response.
- The sponsors also lack the comprehensive safety information that a Phase 3 trial will provide.
The safety consideration is a big one. Suppose the vaccine is effective and reduces the infection rate from 4% to 2%. It will immediately benefit just 2% of the recipients. It will not take much of a change in the safety profile (say, adverse events rising from 2% to 5%) for the harm to rise to a level that rivals the benefit. The Phase 2 results, on a sample of just a few hundred, while needed to justify the next stage of testing, will not furnish the level of detailed safety information needed to balance benefit and safety for formal approval.
How soon will we have a vaccine?
The speed with which a vaccine can be developed and approved depends on several things:
- The base infection rate (the higher it is, the faster the Phase 3 trial will accumulate evidence)
- The true efficacy of the vaccine (the more effective, the sooner it will prove itself)
- A sound trial design and implementation
The latter point is important, for a couple of reasons.
Studies that require repeated looks at the data to see how well the treatment is doing are complex from a statistical perspective and must be well designed to avoid Type 1 error (mistakenly “finding” a positive result just by chance). Repeated examination of the data to test for a statistically-significant result that meets the 50% target heightens the risk of Type 1 error. The protocol for examining the data and the success threshold must be carefully specified in advance in a way that “preserves alpha,” the parameter that controls Type 1 error. Neither the trial sponsor nor the public will want to wait many months for a declaration of success if a vaccine appears successful in early days of a trial — hence the need for flexible and adaptive designs that allow early stopping.
Another factor is the large number of clinical sites that must be engaged in a Phase 3 trial; the Moderna trial will involve 89. The more sites, and the more diverse their locations and administrators, the greater the heterogeneity in the overall trial implementation. Heterogeneity is a problem for regulators; it means a longer review while they sort out whether all the different trial sites were doing the same thing and can effectively be lumped together.
Finally, an additional factor is the manufacturing lag. Gearing up to produce enough vaccines to cover the entire world is a huge undertaking. While drug and biologic sponsors can start manufacturing at any time, they are at risk if they invest in production of a drug or biologic that ultimately fails regulatory approval. They must at least wait until they are certain that they are producing the same drug or biologic that will be submitted for approval. In the Coronavirus case, a number of companies are moving ahead with production now, ahead of regulatory approval, so they can get a jump on distribution (assuming their vaccine is approved).
Here are some numbers on Phase 3 partnerships:
(drawn partly from this New York Times Coronavirus Vaccine Tracker)
Moderna (US) – US National Institutes of Health (NIH)
– Announced Phase 3 start July 27, target: 30,000 participants in 89 sites in the US. The US government will supply $1 billion in development funding and also has a $1.5 billion contract to purchase vaccine doses if it is approved.
BIONTECH (Germany) – Pfizer (US) – Fosun Pharma (China)
– Announced combined Phase 2/3 start July 27, target 30,000 participants in US, Brazil, Germany, Argentina. The US has agreed to a $1.9 billion contract to purchase doses.
CanSinoBio (China) – Chinese Academy of Military Medical Sciences
– Announced Phase 3 start August 9, target 5000 participants in Saudi Arabia, seeking more sites; Chinese military will begin using the vaccine candidate immediately based on Phase 2 results.
Astrazeneca (UK/Sweden) – University of Oxford
– Announced first Phase 3 trials in May (Phases 1 and 2 were combined), now targeting 50,000 participants in UK, South Africa, US. The U.S. has provided $1.2 billion in support, and the EU has a contract to purchase 400 million doses.
Wuhan Institute of Biological Products – Sinopharm (China)
– Announced in July Phase 3 trials in the UAE targeting 5000 participants.
Beijing Institute of Biological Products – Sinopharm (China)
– Announced in July Phase 3 trials in the UAE targeting 5000 participants, separate from the Wuhan Institute vaccine.
Sinovac Biotech (China)
– Announced July 6 Phase 3 trials in Brazil targeting 9000 health professionals, and Indonesia targeting 1620 participants
Murdoch Children’s Research Institute (Australia)
– Phase 3 trial of an existing tuberculosis vaccine to determine if it is protective against Coronavirus, targeting 10,000 participants. This is the largest of numerous Phase 3 studies for this vaccine, which has benefited from extensive prior study of safety and efficacy for tuberculosis.
Success Rates in Clinical Trials, Historically
A recent MIT study put the overall approval rate for drugs and biologics entering clinical trials generally at about 14%, while the success rate for vaccines was 33%. Obviously, the success rate for Phase 3 trials will be higher, since clearly-unsuccessful candidates will be weeded out at earlier stages. Still, even drugs that have passed all hurdles may ultimately prove unsafe or ineffective. Vioxx, an anti-inflammatory drug that was highly effective, passed its Phase 3 trials but had to be pulled from the market when additional studies to extend its scope for prescribing turned up hazards for particular subgroups.
For Coronavirus there is the additional factor of the extreme political and public pressure to find a vaccine quickly which further lowers the odds for success for any individual trial. Sponsors rush even their preliminary results to publication in non-peer-reviewed pre-press venues, which capture widespread attention and generate momentum. Despite assurances to the contrary, pharmaceutical and regulatory officials at all levels and at all stages will feel the need to tip closely balanced decisions in favor of rapid approval.
Multiple Testing
Pressure for rapid action raises the odds that an approved drug might ultimately prove less effective or safe than initially thought. A complicating factor is the unprecedented effort going into vaccines, which heightens the “multiple-testing” problem. There is always some probability that a vaccine declared a “success” in a study will ultimately not work in reality. With over a hundred vaccines in the works and dozens of trials with human subjects under way, the probability of this happening is heightened. Suppose an individual vaccine that is declared a success in a study actually has only a 90% probability of being a real success, and suppose that 15 vaccines have been declared by their regulators to be successes. Then the probability that all 15 will be true successes would only be 0.9^15, or about 20%. Put another way, there would be an 80% probability that one of the vaccines that was successful in a study will not really be a success when deployed.
Conclusion
A sense of how accelerated the Coronavirus vaccine hunt has become is gained from this New York Times article on vaccine timelines, which in the past have been measured more in decades than months. The article describes various shortcuts that can and have been taken, but the cost of these shortcuts is a heightened probability that vaccines labeled “successful” will not work very effectively or will be unsafe. The more a vaccine has been hurried up to be the first out of the gate, the greater this probability. Widespread distribution of ineffective or hazardous vaccines may do more harm than good, if it undermines public confidence in vaccines and retards the acceptance of better vaccines.