In a randomized experiment, the endpoint or outcome is a formal measure (statistic) of the result of the experiment. In a randomized clinical trial preparatory to regulatory submission, there is often more than one outcome, due to the time and expense involved in conducting a clinical trial. A good illustration is the Moderna Covid-19 vaccine Phase 3 clinical trial. It has one primary outcome for efficacy, and three for safety:
- Number of participants with a first occurrence of COVID-19 within two years
- Number of participants withdrawing from the study within two years due to adverse events (AEs)
- Number of participants with solicited Adverse Reactions (ARs) within a week after injection
- Number of Participants with Unsolicited AEs within four weeks of injection
It also has 10 secondary outcomes that track things such as the severity of any Covid disease, the presence of SARS-Cov2, and the presence and level of various antibodies.
It is important that outcomes of interest be specified in the study protocol, along with the metrics that will be used to measure them. Finding interesting things that were not specified in advance as being of interest carries little weight, due to the problem of “data dredging.” Data dredging is the process of looking through the data at great length in exploratory fashion. If you look at complex data long enough, you are sure to find something interesting or an odd pattern, so in a proper study you get to count interesting things only if you specify what you are looking for in advance.
The primary outcomes are typically limited in number, because the more you specify, the higher the bar of statistical significance for each (due to the problem of multiple testing, similar to the concerns with data dredging). For a clinical trial in support of a drug or medical device, it is these primary outcomes on which success or failure hinges. The secondary outcomes provide additional information of interest for further research or consideration; the NIH defines a secondary outcome as
“A planned outcome measure in the protocol that is not as important as the primary outcome measure, but is still of interest in evaluating the effect of an intervention.”
The Moderna trial targets 30,000 participants. Split into equal-sized treatment and placebo groups, this would yield 15,000 vaccine recipients. At a rate of infection of 2% (a reasonable estimate, but this rate is not known), there would be an expected 300 Covid cases in the placebo group.
Conclusion
How much better must the vaccine group do, for the vaccine to be declared a success and approved by regulatory authorities? The US Food and Drug Administration and the World Health Organization have said that for a new vaccine to be approved, trials must demonstrate a 50% reduction in the infection rate of vaccine-takers, compared to those who do not take it. (Note: this is often misinterpreted in the media as requiring that a vaccine must prevent the disease in 50% of the people who take it.)