Executive Summary

The paper presents a substantive critique of COVID-19 vaccine trials and subsequent public health communications. While the core arguments regarding trial design limitations and data interpretation are well-founded, several areas require refinement to strengthen the analysis and ensure precise academic discourse.

Strengths

The paper effectively identifies significant issues in the COVID-19 vaccine trials and provides actionable solutions that could have enhanced trial quality and public health communication accuracy. The authors' conclusions regarding the FDA's role in requiring clinically important outcomes and the need for more precise public health messaging are well-supported by the evidence presented.

Areas Requiring Refinement

Trial Design and Primary Outcome Selection

The authors' characterization of the trials as having "failures in design" requires important clarification. While their critique of primary outcome selection is valid, it should be distinguished from the overall trial design quality. The trials were, in fact, methodologically robust and well-designed to detect their chosen primary outcome of symptomatic infection. The authors should more precisely frame their critique by acknowledging this distinction: the limitation lies not in the trial design methodology itself, but rather in the selection of primary outcomes that were not optimally aligned with the most clinically meaningful endpoints.

The authors should revise their analysis to reflect that primary outcome selection, while an integral component of trial design, is just one element among many. The trials successfully achieved their stated objective of measuring vaccine efficacy against symptomatic infection through sound methodology, appropriate sample size calculations, and proper follow-up protocols. However, the authors make a valid point that for a disease primarily threatening through hospitalization and death, choosing symptomatic infection rather than these serious clinical outcomes as the primary endpoint represents a missed opportunity to gather more clinically relevant data. This more nuanced analysis would strengthen their argument while maintaining scientific accuracy about trial design quality.

Trial Interpretation versus Data Extrapolation

The paper's discussion of "interpretation failures" would be more precisely characterized as inappropriate extrapolation of trial data. This distinction is crucial as it suggests public health officials were aware of the data's limitations when making broader claims about vaccine efficacy. The paper should emphasize how claims exceeded what the trial data could support, rather than focusing on misinterpretation of the available evidence.

Current Evidence State: COVID-19 Vaccine Impact on Hospitalization and Death

A critical limitation of the paper is its insufficient emphasis on how our current understanding of vaccine impacts on hospitalization and death derives entirely from observational studies. Since the original randomized controlled trials were not designed to examine these crucial clinical outcomes, as the authors point out, the scientific community has had to rely on observational data, which is inherently unreliable for determining intervention efficacy. This reliance on observational studies, which are particularly susceptible to healthy vaccinee bias, means that the true impact of COVID-19 vaccines on hospitalization and death remains uncertain. Indeed, in observational studies it has been found that healthy user bias may have accounted for the entire reduction in COVID-19 death associated with vaccination, highlighting the unreliability of such data for determining vaccine effectiveness. The paper should explicitly acknowledge this uncertainty and explain how the original trial design limitations have left us without reliable evidence regarding these critical outcomes. This uncertainty extends to both COVID-specific and all-cause outcomes, a distinction that requires clear delineation in the analysis. For review of the evidence regarding this uncertainty see this non-peer reviewed paper I co-authored).

Transmission Prevention and Fundamental Vaccine Goals

The paper's abstract opens with the statement "For the Covid vaccines, the fundamental goal was not to prevent mild infections but to prevent deaths, hospitalizations, and transmission." This characterization requires correction, as it incorrectly positions transmission prevention as a fundamental goal of COVID-19 vaccination. While reducing deaths and hospitalizations are indeed fundamental vaccine goals, transmission reduction represents a potential secondary benefit rather than a primary objective. Some effective vaccines achieve their primary goal of preventing severe outcomes without reducing transmission, as demonstrated by the acellular pertussis vaccine, which successfully prevents severe disease while having minimal impact on transmission.

Mechanisms of Vaccine Protection

The paper presents a dichotomy between vaccines working "directly" by protecting the vaccinated person or "indirectly" by preventing viral spread. This framework needs refinement, as it does not accurately reflect how vaccines prevent hospitalization and death. A more precise dichotomy would distinguish between infection prevention and symptom severity reduction. Historically, vaccines have worked either through both direct and indirect mechanisms—preventing infection or reducing symptom severity (and reducing transmission)—or through direct protection alone by reducing disease severity without affecting transmission. Importantly, no vaccine has ever prevented hospitalization and death solely through indirect means of transmission reduction without providing direct protection to the vaccinated individual.

The relationship between symptom severity reduction and transmission deserves particular attention. When a vaccine reduces symptom severity, it can affect transmission in two opposing ways. It may decrease transmission by reducing symptoms that typically facilitate spread, such as coughing or sneezing. Conversely, as the authors correctly note, reducing symptom severity could potentially increase transmission by enabling more asymptomatic spread, as infected individuals may be unaware of their infection status and continue normal activities. This complex relationship between severity reduction and transmission further demonstrates why framing vaccine mechanisms in terms of infection prevention versus symptom severity reduction provides a more accurate model for understanding vaccine effectiveness and evaluating clinical trial endpoints.

Supporting Evidence for Transmission Effects

The paper's discussion of potential transmission increases due to symptom reduction warrants expansion with supporting evidence. The authors could strengthen their analysis by examining several corroborating observations. Israel's experience in August 2021 provides a compelling case study: despite achieving one of the world's highest vaccination rates, the country experienced one of the highest per capita COVID-19 infection rates globally. This outcome would be improbable if the vaccine significantly reduced transmission, and potentially suggests the opposite effect. Similarly, global trends throughout 2021 showed many highly vaccinated nations experiencing higher per capita infection rates than countries with lower vaccination rates, as documented by OurWorldInData.org. While these population-level observations remain circumstantial, they align with the hypothesis that symptom suppression might facilitate transmission. Furthermore, the authors could draw parallels with the pertussis acellular vaccine, which demonstrates a historical precedent for vaccines that effectively reduce disease severity while having limited impact on transmission has potentially resulted in the vaccine increasing asymptomatic transmission.

Safety Analysis Limitations

The paper should emphasize how the accelerated authorization process impacted safety monitoring. The trial's safety analysis period was deliberately shortened to expedite authorization, accepting an increased risk of missing serious safety signals. With a median follow-up of only two months after the second dose, half of the participants had less than two months of safety data, and thousands had not yet received their second dose. Even within this abbreviated monitoring period, a re-analysis of the original trial data published more than a year after authorization identified a statistically significant increase in the incidence of serious adverse events in the Pfizer vaccine group that was present in the original data but went unreported during the initial review. (Note I was a co-author of this paper).  In addition, in the same re-analysis it was demonstrated that the vaccine group experienced an increase in the rate of SAEs higher than the reduction in hospitalization the vaccine offered. Given that the trial was not powered to examine hospitalization, it remains unclear whether extending the trial duration or enrolling higher-risk participants would have demonstrated that the vaccine's benefits outweighed this increase in serious adverse events. These important points are relevant to the author's arguments and may deserve mention. 

All-Cause Outcomes and Safety Assessment

The paper's statement that "Unlike efficacy, randomized clinical trials are insufficient to determine whether a vaccine or drug is sufficiently safe to administer" requires revision. This assertion overlooks how a well-designed trial using all-cause hospitalization or death as its primary outcome could simultaneously demonstrate both efficacy and safety. A trial showing a reduction in all-cause adverse outcomes would inherently establish that an intervention's benefits outweigh its risks. Neither the Pfizer nor Moderna trials have published their all-cause hospitalization data, leaving this crucial assessment incomplete. Had the trials been designed with all-cause outcomes as primary endpoints, they could have provided clear evidence of net benefit while addressing safety concerns through a single, robust measure.

Use of "Unscientific"

The paper's use of the term "unscientific" in both the abstract and main text requires more precise language. When discussing recommendations regarding the COVID-19 vaccine, the term could be interpreted in multiple ways and may not effectively convey the authors' intended critique. A more precise characterization would be to state that these recommendations were not supported by high-quality evidence, providing a clearer and more specific criticism of the evidence base underlying these decisions.

Use of "Unethical"

The term "unethical" carries significant weight in academic discourse and requires careful justification, particularly in medical research and public health contexts. When authors characterize actions or decisions as unethical, they should provide clear reasoning based on established ethical frameworks and principles. The paper uses this term in several contexts that warrant more rigorous explanation.

Regarding vaccine mandates, the authors could strengthen their position by applying established medical ethics principles. A fundamental principle holds that forcing medical interventions on unwilling individuals violates individual autonomy and is generally considered unethical, even for potentially life-saving treatments. The only widely accepted exception occurs when an intervention provides substantial external benefits to others, such as vaccines that prevent disease transmission. Since COVID-19 vaccines have not demonstrated meaningful transmission reduction, they do not qualify for this exception, providing a clear ethical framework for opposing mandates.

The paper's characterization of early trial termination as unethical requires more nuanced discussion. This decision involved balancing emergency pandemic conditions against the value of continued data collection. The authors should clarify whether their ethical concerns stem from ending the trial for all participants or specifically for those not yet eligible for vaccination under initial authorization guidelines. Given the limited vaccine supply during early authorization, ending the trial for those not yet eligible for vaccination could have delayed access for higher-risk individuals, presenting a distinct ethical consideration.

In discussing global vaccine distribution, the paper invokes ethical concerns about vaccine access disparities between higher and lower-income nations. While equity in global health resource allocation raises important ethical considerations, presenting this solely as an ethical issue oversimplifies the complex interplay of practical, political, and moral factors affecting vaccine distribution. The authors should acknowledge these competing considerations while building their ethical argument.

Booster Trial Analysis

The paper's examination of booster trial limitations warrants expansion, particularly regarding two concerns. First, the authorization of boosters in September 2021 occurred in a context where documented COVID-19 cases in the United States exceeded 100 million, with actual infections likely substantially higher given limited testing availability. This epidemiological reality meant that approximately half the population had experienced prior infection, yet the clinical trials had not systematically evaluated vaccine safety or efficacy in this substantial demographic. The authorization of boosters for a population where a significant proportion had prior infection, without adequate testing in this subgroup, represents a departure from standard clinical trial practice that merits critical examination.

Second, the regulatory approach to surrogate endpoints requires scrutiny. The FDA's position on antibody testing was against the use of SARS-CoV-2 antibody test results to evaluate immunity or protection. However, this position was apparently contradicted when antibody testing in mice was accepted as a primary outcome measure for bivalent booster authorization. This inconsistency in regulatory standards, particularly for such a widely deployed intervention, deserves mention in the paper.

Historical Trial Comparison Analysis

The paper's comparison between the polio and COVID-19 vaccine trials requires more consistent analytical standards and greater precision in evaluating trial outcomes. While the comparative approach provides valuable insights, several aspects of the analysis need refinement to ensure equivalent evaluation of both trials.

The authors' statement that the polio vaccine trial "provided solid proof the vaccine worked" needs more precise characterization. In the immediate post-trial period, both the polio and COVID-19 vaccine trials demonstrated efficacy for their respective endpoints. The polio trial showed reduction in paralysis, while the COVID-19 trials showed reduction in symptomatic infection. The distinction lies not in whether the vaccines "worked," but in the clinical significance of the endpoints measured.

The paper's assertion that without the polio trial, "public health officials would have been shooting in the dark, giving advice to parents about the efficacy of the product without real data underlying it" requires clarification. This statement implies a false dichotomy between the polio trial design and potential alternatives. A more accurate comparison would note that the polio trial allowed officials to confidently communicate reduction in paralysis risk, while the COVID-19 trials only supported claims about reducing flu-like symptoms. This distinction emphasizes the importance of selecting clinically meaningful primary outcomes.

The authors' reference to polio being "now eradicated in the United States and most of the world" introduces information that extends beyond the trial comparison timeframe. The paper should maintain consistent temporal boundaries when comparing the trials, focusing on what was known in the immediate post-trial period. If the polio vaccine's impact on transmission was not demonstrated in the original trial but discovered later, this should be explicitly stated to maintain analytical consistency.

Causality in Public Trust Analysis

The paper makes several assertions about the relationship between various events and declining public trust in vaccines and health institutions. While these relationships may exist, the paper's causal claims require more rigorous substantiation. For instance, the statement "Consequently, trust in vaccines has plummeted" and cites (Siani and Tranter, 2022) a survey comparing vaccine hesitancy between 2019 and 2022, but this citation only demonstrates correlation, not causation. Similar causal claims regarding loss of trust in public health officials appear in the abstract and multiple sections without adequate supporting evidence. The authors should either provide empirical evidence demonstrating causality or modify these statements to acknowledge the potential complexity of factors influencing public trust and vaccine hesitancy.

Sympomatic COVID-19 Infection Definition

The paper would benefit if it offered the definition of symptomatic COVID-19 infection as used in the clinical trials. The FDA's official definition required both a positive COVID-19 PCR test and at least one qualifying symptom.

Citation and Reference Accuracy

The paper contains several statements requiring additional citation support or correction of existing references:

The claim that public health officials "promised that the vaccine would provide long-lasting protection against getting, spreading, and dying from Covid" needs specific citations to verify these statements.

The assertion regarding 1952 polio deaths requires a historical reference to support the specific number cited.

The paper incorrectly cites Moher et al. (1994) regarding the Salk polio vaccine trial size. While this reference discusses statistical power in randomized controlled trials generally, it does not specifically analyze the Salk trial. The authors should locate and cite primary sources about the Salk trial's design considerations.

Conclusion

While the paper presents valuable insights regarding COVID-19 vaccine trials and public health communication, implementing these refinements would strengthen its academic rigor and impact. The core arguments are sound, but greater precision in terminology, evidence presentation, and ethical argumentation would enhance its contribution to the scientific discourse.

It’s important to discuss what happened in order to figure out how to do better in the future. In this regard, the contribution from Drs. Bhattacharya and Kulldorff—and indeed the entire concept of the Journal of the Academy of Public Health— is a refreshing departure from business as usual. The success of the concept will ultimately depend on the credibility of the reviewers and the quality of the constructive scientific dialog that ensues. The following comments are offered in the spirit of furthering this discussion.

The authors do an excellent job reciting some of the public health failures during the COVID pandemic, which include misinterpretation not only of data from the original clinical trials, but also from observational studies. For example, the logic underlying vaccine mandates, which were ultimately struck down by the Supreme Court, was based on the false supposition that vaccines would dramatically decrease SARS-CoV-2 transmission.  Because mandates were considered difficult for unlicensed vaccines that were made available through Emergency Use Authorization, the plan to impose mandates appeared to be connected to a push to speed up the approval of one of the COVID vaccines.  The plan to authorize boosters for the general population was announced by the White House and public health authorities before the FDA had even started, much less completed, its review of these applications. Although observational data from Israel was considered a major factor in the initial recommendations for COVID vaccine boosters, the presence of “healthy vaccinee bias” (where people with access to boosters were in much better health and had access to better overall medical care) made it falsely appear that the original boosters were much more effective than they actually were (Høeg, 2023). In addition to the references cited by the authors, the CDC’s own data (León, 2022) showed that previous infection was more effective than vaccination in preventing COVID-associated hospitalization. This doesn’t mean that it would have been preferable for more people to get infected, because COVID was still deadly even in young adults.  But it does expose as misguided the policy of requiring previously infected people to be vaccinated, and as the authors point out, clothing fundamentally political solutions such as vaccine mandates and White House generated recommendations for boosters as public health imperatives very likely contributed to mistrust of public health authorities and of vaccines in general.

The authors also decry a shutting down of public debate on potential public health responses to the pandemic, and here too I also agree that a thoughtful public airing of facts can only contribute to positive outcomes.

Lessons learned: Public health officials and stringency of regulatory decision-making

In spite of my agreement with the authors on several key points, the authors have not made a persuasive case on the “lessons learned”. In one case, they do not go far enough, and in the other, they do not consider key facts and the need for flexibility in pandemic response. Both of these lessons are related to potential politicization of pandemic response.

The first of the authors’ two major “lessons learned” is that public health officials must be scrupulous in not extrapolating beyond what the randomized trial data say.  It’s hard to disagree with this; it’s what we should expect from all government employees, especially those charged with making decisions regarding public health.  However, political pressure and career aspirations can play a role as well, and some public health officials will inevitably be more competent than others. Thus, in addition to hoping for better public health officials, it will be important to strengthen the system that leads to public health recommendations by increasing its transparency and limiting the authority of any single individual to overrule the outcome of normal review processes.   

The authors’ other “lesson learned” is that the FDA should require more stringent trials of pandemic products. Here, it is important to distinguish the flexibility necessarily allowed during emergencies under Emergency Use Authorization (EUA) from the stronger FDA endorsement that is implied by full licensure (also known as “approval”). While caution must be exercised and transparency of decision-making is critically important, having flexibility to address emergencies is a key component of pandemic response because pandemics are by their nature unpredictable. In an emergency, an EUA may be issued after (among other conditions) the FDA determines “that, based on the totality of scientific evidence available to FDA, including data from adequate and well-controlled clinical trials, if available, it is reasonable to believe that: … the product may be effective in diagnosing, treating, or preventing …such disease or condition” and that “the known and potential benefits of the product, when used to diagnose, prevent, or treat such disease or condition, outweigh the known and potential risks of the product…”. Relative to full licensure, FDA points out that [t]he “may be effective" standard for EUAs provides for a lower level of evidence than the "effectiveness" standard that FDA uses for product approvals (Food and Drug Administration, 2017).

It is critical that the flexibility offered by the EUA standard be used responsibly and transparently, as indeed it was during the early COVID response.  Within this framework, FDA’s Office of Vaccines Research and Review (OVRR) insisted on reasonably stringent vaccine trials at the EUA stage and provided a high level of transparency into the process. 

To promote transparency and scientifically valid conclusions, OVRR began early discussions about what it should take to license or authorize a COVID vaccine. OVRR wrote and published two guidance documents—first, one released June 2020 for development and licensure of vaccines (Food and Drug Administration 2020a), and a follow-up written in August-September 2020, once it became clear that vaccines would likely initially be made available via Emergency Use Authorization (EUA) (Food and Drug Administration 2020b). A vaccine needed to show at least 50% efficacy in clinical trials with 95% statistical confidence that the efficacy was at least 30%. Without foreknowledge of how effective COVID vaccines might be, this set a reasonable target for vaccine efficacy that assured that ineffective vaccines would not be approved—a situation that could be disastrous in a pandemic if it interfered with evaluation or deployment of truly effective vaccines (Krause, 2020a). These criteria also assured that studies would be large enough to obtain a robust safety database. For licensure, safety monitoring of at least 6 months would be expected, while for EUA, total follow-up of a median of 2 months (meaning that half of participants would be followed for at least 2 months) would suffice. This would assure rapid availability of COVID vaccines desperately needed in the growing pandemic while also collecting the minimal safety and efficacy information needed to provide confidence in the vaccine’s evaluation (Krause, 2020b). These criteria were discussed at international meetings, including by FDA’s Vaccines and Related Biological Products Advisory Committee which endorsed the criteria as laid out in the EUA Guidance (Food and Drug Administration, 2020c).

 In making products available to the public, FDA is appropriately limited to its statutory authority to determine whether the indications and labeling proposed by the developer meet the applicable standards based on objective reviews of the supporting data. Data from additional studies may support additional indications, for example, efficacy against transmission, duration of immunity, or efficacy against serious outcomes—and the FDA will include this information in the prescribing information/fact sheet if the data support these indications or assertions. If public health authorities choose to make decisions without formal FDA review of these data, they may look to other sources, but the FDA does not have the authority to require submission of all of the data that might eventually be used to support public health decisions, nor would it be appropriate for FDA to decide to withhold an otherwise appropriate authorization or approval (e.g., for a safe and effective vaccine that would save lives) if it thought some potentially useful piece of data (e.g., its efficacy against viral transmission) were missing. The authors’ claim that FDA will cause harm by putting public health officials in a position where they are tempted to extrapolate communications beyond the evidence puts the responsibility in the wrong place. It is the FDA’s role to ensure that the evidence is completely and fairly presented, but it is a separate responsibility for public health officials and politicians to be honest about what FDA has concluded.

The COVID vaccine trials

The authors also provide a critique of the design of the COVID vaccine trials. A careful look at the record shows that the major failures were not in the design of the original trials or the criteria used by the FDA, but in the incuriousness of the public health establishment to obtain additional useful and reliable data about important vaccine parameters. Here, I have the following comments:

Serious outcomes and ages of trial participants

The authors suggest that the original studies should have been powered to detect serious outcomes such as hospitalization or death.  But we already know from experience that vaccines that are protective against mild outcomes will be at least as protective against serious outcomes. As the authors themselves point out, since symptomatic infection is a precondition for severe disease and death, it was likely that the Covid vaccines also prevented severe disease and death.  Because the outcomes are rarer, trials against serious outcomes need to be much larger, unnecessarily increasing the number of people and time needed before life-saving treatments can become available. 

Indeed, the authors appear to further contradict themselves as they point out, “Careful observational studies, such as the previously mentioned Qatari and Swedish studies, later verified both the efficacy against severe disease among the old ….” In addition, data from New York State (which collected high quality data in a large population over the course of the pandemic) showed that vaccination consistently protected from hospitalization at a rate of 89.5% or higher (Rosenberg, 2021).  Thus, the trials as originally designed did in fact provide the expected information regarding serious outcomes. 

Would it have been a good idea to change the age distribution of the original trials to focus on the elderly as the authors suggest? While the greatest risk was in the elderly, there was still substantial risk in younger adults. In 2020, before vaccines became widely available, over 10% of deaths in Americans over age 44 had COVID as an underlying cause, and over 5% of deaths in those aged 25-44 had COVID as an underlying cause (Ahmad et al).  Even the youngest adults risked dying of COVID (Williams, 2024). Through 2020, COVID had become the third most common cause of death in the US, behind heart disease and cancer. A trial, as proposed by the authors, that excluded younger adults would have justly been subject to substantial criticism. The authors allude to early fears that the elderly might be excluded from COVID vaccine efficacy trials (Helfand, et al 2020), but thanks to the foresight of FDA/OVRR, the studies that were performed (Baden 2020, Polack 2020, Heath 2021, and Sadoff 2021) all included substantial proportions of elderly participants and were large enough to show that the vaccines were highly effective regardless of age group.

The author’s statement that mRNA vaccines did not reduce all-cause mortality is based on a flawed interpretation of summarized outcomes against all-cause mortality from the original clinical trials (Benn 2023). These trials were too small to determine the vaccines’ effect on mortality, and the cited paper reported very wide confidence intervals. Larger studies have shown clear benefit of vaccination in reducing all-cause mortality (e.g., Dahl 2024).

Transmission

The authors suggest that a household exposure study to evaluate vaccine efficacy against transmission should have been embedded in the original COVID vaccine efficacy trials. While it is important to understand the impact of a vaccine on transmission, this is normally not the primary goal of the first vaccine trial.  A study similar to the authors’ proposed household exposure study demonstrated that the chickenpox vaccine reduced household transmission (Merck and Co., 1995), but this appropriately was a different study from the original efficacy trial. Nobody would study vaccine impact on transmission without first demonstrating an impact on disease. Thus, the failure was not in the design of the initial studies, but it was in the absence of follow-up to answer critical additional questions required to support downstream policy decisions. Even if a household transmission study had been done, other study designs would also have been useful because a vaccine that fails to prevent household transmission might still reduce community transmission, at least for some period of time, and once vaccine became generally available, it may have become more difficult to enroll households in which some members agreed to potentially be randomized to placebo.

Safety

The authors correctly point out that there is unavoidably less information available about safety when a vaccine is newly released than after more experience has been obtained.  Substantial improvements in the ability to study vaccine safety in recent years using large databases has permitted the evaluation of even very rare vaccine side-effects.  Along with other sources of information, the FDA and CDC use the “VAERS” database, which relies on spontaneous reporting to generate hypotheses about potential adverse events.  The databases are then used to determine more rigorously whether or not these potential adverse reactions truly occur in temporal proximity to vaccination. 

In the COVID pandemic, a substantial opportunity to collect additional highly reliable safety information was missed. FDA/OVRR reviewers pointed out that, since at the time vaccines were authorized, there was considerably more vaccine demand than supply, it would be ethical to distribute vaccines within priority groups not just on a first-come first served basis, but based on an arguably much more fair approach of random allocation.  Random allocation of vaccine appointment times would have allowed unbiased comparison of adverse events in far more people (even millions) between people who had and had not yet been vaccinated than were feasible to include in the original clinical trials, and might have led to much earlier detection of rare events after vaccination like myocarditis, Guillain-Barre syndrome, or Thrombosis with Thrombocytopenia Syndrome.  This idea was discussed in public at FDA’s VRBPAC meeting on 12/10/20 (Food and Drug Administration, 2020d), but vaccine distribution and implementation plans were considered too advanced to allow for randomized collection of this potentially important safety data.  It also has been noted that a similar approach could also be used in a future pandemic to collect additional efficacy data during vaccine roll-out, which could help to further define efficacy against rare outcomes like hospitalization or death.

Duration of protection and boosters

The authors unfairly criticize the FDA for allowing the original clinical trials to be terminated too early to collect longer term data on duration of vaccine efficacy. Before the vaccines became available, FDA/OVRR  stipulated that studies of long term efficacy and immunity should be performed (Food and Drug Administration, 2020a and Food and Drug Administration, 2020b), but once vaccine became available based on strong efficacy data, it was not possible to prevent placebo recipients from obtaining vaccine, which made it impossible to obtain randomized controlled data on long term efficacy.

In the absence of vaccine effect against transmission, the major justification for boosting should have been to protect vaccine recipients against severe disease, if necessary. Although the continued evolution of the SARS-CoV-2 virus into new variants caused the vaccines to lose efficacy against symptomatic COVID over time, protection from severe disease has been considerably more resilient, probably because the vaccines induce cell-mediated immunity against cellular immune epitopes that, to date, continue to be shared by variants.  However, some individuals, especially the elderly and immunocompromised, have higher risk because their cellular immune responses are weaker.  The original universal recommendation (Banco, 2021) for boosters did not adequately consider data showing that vaccine protection against severe disease was maintained over time (Krause, 2021), and made it appear that political considerations were at least as important as public health considerations in decision-making about boosters.

The fairly loose criterion, that the “product may be effective” and that the “benefits… may…outweigh the …risks” allowed even relatively ineffective products like convalescent plasma, hydroxychloroquine or COVID vaccine boosters (at least when applied to the general population) to meet the criteria to be made available under EUA. Indeed, the updated COVID vaccines (including boosters) received full licensure for adults and adolescents only in 2023 (Food and Drug Administration, 2023), and COVID vaccines still are available only under EUA for children under age 12 (Food and Drug Administration, 2024).  Because it generally isn’t made clear to the public what the reasons are for a given product to be made available under the flexible and broad EUA criteria instead of being fully licensed, the public has no way to interpret these apparent discrepancies. Also, all products that might be perceived to meet the EUA criteria do not necessarily need to be authorized, especially where better alternatives exist. Requiring the FDA to transparently make the reasons for these decisions more explicit (especially the reasons why a “authorized” product has not yet been “approved” and how a newly authorized product fits in the context of other available therapies and vaccines) could help place the data and conclusions being used by public health officials to support their recommendations into a more appropriate context.

To conclude, Drs. Bhattacharya and Kulldorff make a worthwhile contribution to the literature regarding COVID vaccine development and deployment. While some of their conclusions should be tempered by a closer examination of the underlying facts and realities, this is an important discussion that I hope will help to better prepare us all for the next pandemic.

References

1. Ahmad FB, Cisewski JA, Miniño A, Anderson RN. Provisional Mortality Data — United States, 2020. 2021. MMWR Morb Mortal Wkly Rep 70:519–522. DOI: http://dx.doi.org/10.15585/mmwr.mm7014e1 https://www.cdc.gov/mmwr/volumes/70/wr/mm7014e1.htm

2. Baden LR, El Sahly HM, Essink B., Kotloff K, Frey S, Novak R, Diemert D, et al. 2020. Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. N Engl J Med 2021;384:403-416. DOI: 10.1056/NEJMoa2035389 https://www.nejm.org/doi/full/10.1056/NEJMoa2035389

3. Banco E, Cancryn A. 2021. The U.S. plans to offer booster shots next month. Some health experts are wary. https://www.politico.com/news/2021/08/18/biden-recommends-covid-booster-shots-505911.

4. Benn CS, Schaltz-Buchholzer F, Nielsen S, Netea MG, Aaby P. 2023. Randomized clinical trials of COVID-19 vaccines: Do adenovirus-vector vaccines have beneficial non-specific effects? iScience. 26(5):106733. doi: 10.1016/j.isci.2023.106733. Epub 2023 Apr 25. PMID: 37163200; PMCID: PMC10125209. https://pubmed.ncbi.nlm.nih.gov/37163200/

5. Dahl J, Tapia G, Bøås H, Bakken IJL, Løvdal Gulseth HL. 2024.  COVID-19 mRNA-vaccination and all-cause mortality in the adult population in Norway during 2021-2023: a population-based cohort study. medRxiv 2024.12.15.24319058; doi: https://doi.org/10.1101/2024.12.15.24319058. https://www.medrxiv.org/content/10.1101/2024.12.15.24319058v1

6. Food and Drug Administration 2017.  Emergency Use Authorization of Medical Products and Related Authorities: Guidance for Industry and Other Stakeholders. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/emergency-use-authorization-medical-products-and-related-authorities

7. Food and Drug Administration. 2020a. Development and Licensure of Vaccines to Prevent COVID-19: Guidance for Industry https://web.archive.org/web/20200701070333/https://www.fda.gov/media/139638/download

8. Food and Drug Administration. 2020b. Development and Licensure of Vaccines to Prevent COVID-19: Guidance for Industry. https://web.archive.org/web/20201201002608/https://www.fda.gov/media/142749/download

9. Food and Drug Administration. 2020c. Vaccines and Related Biological Products Advisory Committee October 22, 2020 Meeting.  https://www.fda.gov/advisory-committees/advisory-committee-calendar/vaccines-and-related-biological-products-advisory-committee-october-22-2020-meeting-announcement#event-materials

10. Food and Drug Administration. 2020d. Vaccines and Related Biological Products Advisory Committee December 10, 2020 Meeting.  https://www.fda.gov/advisory-committees/advisory-committee-calendar/vaccines-and-related-biological-products-advisory-committee-december-10-2020-meeting-announcement#event-materials

11. Food and Drug Administration. 2023. FDA Takes Action on Updated mRNA COVID-19 Vaccines to Better Protect Against Currently Circulating Variants. https://www.fda.gov/news-events/press-announcements/fda-takes-action-updated-mrna-covid-19-vaccines-better-protect-against-currently-circulating

12. Food and Drug Administration. 2024. COVID-19 Vaccines for 2024-2025.  https://www.fda.gov/emergency-preparedness-and-response/coronavirus-disease-2019-covid-19/covid-19-vaccines-2024-2025

13. Heath PT, Galiza EP, Baxter DN, Boffito M, Browne D,  Burns F, Chadwick DR, et al. 2021. Safety and Efficacy of NVX-CoV2373 Covid-19 Vaccine. N Engl J Med 2021;385:1172-1183. DOI: 10.1056/NEJMoa2107659. https://www.nejm.org/doi/full/10.1056/NEJMoa2107659

14. Helfand BKI, Webb M, Gartaganis SL, Fuller L, Kwon CS, Inouye SK. 2020. The Exclusion of Older Persons From Vaccine and Treatment Trials for Coronavirus Disease 2019-Missing the Target. JAMA Intern Med. 180(11):1546-1549. doi: 10.1001/jamainternmed.2020.5084. PMID: 32986099; PMCID: PMC7522773. https://pmc.ncbi.nlm.nih.gov/articles/PMC7522773/

15. Høeg TB, Ram Duriseti R, Prasad V, M.D., M.P.H. 2023. Potential “Healthy Vaccinee Bias” in a Study of BNT162b2 Vaccine against Covid-19. N Engl J Med 389:284-286DOI: 10.1056/NEJMc2306683 https://www.nejm.org/doi/full/10.1056/NEJMc2306683

16. Krause PR, Fleming TR, Longini I, Henao-Restrepo AM, Peto R, et al. 2020a COVID-19 vaccine trials should seek worthwhile efficacy. Lancet. 2020 Aug 27;396(10253):741–743. doi: 10.1016/S0140-6736(20)31821-3. https://pmc.ncbi.nlm.nih.gov/articles/PMC7832749/

17. Krause PR, Gruber MF. 2020b. Emergency Use Authorization of Covid Vaccines — Safety and Efficacy Follow-up Considerations. N Engl J Med 2020;383: e107. DOI: 10.1056/NEJMp2031373. https://www.nejm.org/doi/full/10.1056/NEJMp2031373

18. Krause PR, Fleming TR, Peto R, Longini IM, Figueroa JP, Sterne JAC, Cravioto A, Rees H, Higgins JPT, Boutron I, Pan H, Gruber MF, Arora N, Kazi F, Gaspar R, Swaminathan S, Ryan MJ, Henao-Restrepo AM. 2021. Considerations in boosting COVID-19 vaccine immune responses. Lancet. 398: 1377–80. https://doi.org/10.1016/ S0140-6736(21)02046-8 https://www.thelancet.com/action/showPdf?pii=S0140-6736%2821%2902046-8

19. León TM, Dorabawila V, Nelson L, et al. 2022. COVID-19 Cases and Hospitalizations by COVID-19 Vaccination Status and Previous COVID-19 Diagnosis — California and New York, May–November 2021. 2022. MMWR Morb Mortal Wkly Rep 71:125–131. DOI: http://dx.doi.org/10.15585/mmwr.mm7104e1. https://www.cdc.gov/mmwr/volumes/71/wr/mm7104e1.htm

20. Merck and Co. 1995. Varivax Prescribing Information https://www.merck.com/product/usa/pi_circulars/v/varivax/varivax_pi.pdf

21. Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, Perez JL, et al. 2020. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. N Engl J Med 383:2603-2615. DOI: 10.1056/NEJMoa2034577. https://www.nejm.org/doi/full/10.1056/NEJMoa2034577

22. Rosenberg ES, Holtgrave DR, Dorabawila V, et al. 2021. New COVID-19 Cases and Hospitalizations Among Adults, by Vaccination Status — New York, May 3–July 25, 2021. MMWR Morb Mortal Wkly Rep 70:1306–1311. DOI: http://dx.doi.org/10.15585/mmwr.mm7037a7. https://www.cdc.gov/mmwr/volumes/70/wr/mm7037a7.htm

23. Sadoff J, Gray G, Vandebosch A, Cárdenas V, Shukarev G, Grinsztejn B, Goepfert PA, et al. 2021. Safety and Efficacy of Single-Dose Ad26.COV2.S Vaccine against Covid-19. N Engl J Med 384:2187-2201. DOI: 10.1056/NEJMoa2101544Sadoff et al 2021  https://www.nejm.org/doi/full/10.1056/NEJMoa2101544

24. Williams, Piasecki TM, Fiore MC, Conner KL, Slutske WS. 2024.  Hospital outcomes for young adults with COVID-19. Glob Epidemiol. 2024 Jul 5;8:100155. doi: 10.1016/j.gloepi.2024.100155. https://pmc.ncbi.nlm.nih.gov/articles/PMC11296004/