Main

Additional doses of messenger RNA (mRNA)-based vaccines are being widely deployed to counter waning immunity and broaden protection against emerging variants of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)1,2. However, head-to-head comparisons of the effectiveness of a third dose of different mRNA-based vaccines have been lacking.

In a previous head-to-head comparison of breakthrough COVID-19 outcomes after the first dose of the BNT162b2 (Pfizer-BioNTech) and mRNA-1273 (Moderna) vaccines among 439,684 US veterans aged ≥18 yr, we found a low risk of documented infection and severe COVID-19 outcomes (for example, hospitalization and death) in a period marked by SARS-CoV-2 Alpha-variant predominance in both vaccine groups, but lower risk for the mRNA-1273 vaccine (with a similar pattern for documented infection in a period marked by Delta-variant predominance)3. It is unclear whether the same findings apply to third doses and a range of COVID-19 outcomes in periods marked by other SARS-CoV-2 variants, including Omicron.

A report from Spain estimated a third dose of mRNA-1273 to be 13% more effective than a third dose of BNT162b2 for the prevention of documented SARS-CoV-2 infection during a period of Omicron-variant predominance4. In other reports, effectiveness of mRNA-based vaccines against symptomatic COVID-19 was analysed separately for each third dose compared with no vaccination5 (or with the primary vaccination series6). These comparisons provide indirect evidence for comparative effectiveness but only if the comparators from each analysis had similar characteristics. Additional studies are needed to compare the effectiveness of a third dose of these vaccines head-to-head for severe COVID-19 outcomes to inform the choice of vaccine in coordinated public health responses. Ideally, any comparative effectiveness study should include racially diverse groups, evaluate potential differences in effectiveness according to time since the completion of the primary vaccination series and separately address time frames that include predominance of different SARS-CoV-2 variants.

300x250x1

Here we analysed data from the national healthcare databases of the Department of Veterans Affairs (VA), the largest integrated healthcare system in the US, to compare the effectiveness of a third dose of either the BNT162b2 or the mRNA-1273 vaccine among US veterans who had completed an mRNA vaccine primary series and received a third dose between (1) 20 October 2021 and 8 February 2022 (a period spanning Delta- and Omicron-variant predominance) or (2) 1 January and 1 March 2022 (a period restricted to Omicron-variant predominance). Recipients of each vaccine were matched in a 1:1 ratio according to their risk factors. Comparative effectiveness was estimated over 16 weeks in the combined Delta-Omicron period and over 9 weeks in the Omicron period for five COVID-19 outcomes: documented SARS-CoV-2 infection, symptomatic COVID-19 and COVID-19-related hospitalization, admission to an intensive care unit (ICU) and death.

Results

Primary analyses during a period spanning SARS-CoV-2 Delta- and Omicron-variant predominance

Study population and follow-up

Among 147,553 eligible recipients of a BNT162b2 third dose and 214,728 eligible recipients of an mRNA-1273 third dose, 65,196 BNT162b2 recipients were matched to 65,196 mRNA-1273 recipients (Fig. 1). The baseline characteristics of this matched population (Table 1) were similar to those of the eligible population (Supplementary Table 1). The median age was 70 yr (interquartile range, 62–74 yr), 96% of individuals were male and 24% were Black. The two vaccine groups had similar distributions of demographics, coexisting conditions and markers of healthcare utilization (see also Extended Data Fig. 1).

Fig. 1: Selection of persons for the emulation of a target trial evaluating the comparative effectiveness of a third dose of BNT162b2 and mRNA-1273 vaccines.

Selection was conducted during a period spanning Delta- and Omicron-variant predominance (20 October 2021 to 8 February 2022).

Full size image
Table 1 Baseline characteristics of matched persons in the target trial emulation evaluating the comparative effectiveness of a third dose of BNT162b2 and mRNA-1273 vaccines during a period spanning Delta- and Omicron-variant predominance
Full size table

The median follow-up was 77 d (interquartile range, 61 to 97 d). Over a 16-week-follow-up, 2,994 SARS-CoV-2 infections were documented, of which 200 were detected as symptomatic COVID-19 within the VA healthcare system, 194 required hospitalization, 52 required ICU admission and 22 resulted in death.

Comparative effectiveness

Over a 16-week period spanning Delta- and Omicron-variant predominance, the estimated risk of documented infection was 353.9 (95% CI: 326.7, 373.2) events per 10,000 persons for the BNT162b2 third dose and 308.5 (95% CI: 276.8, 320.9) events per 10,000 persons for the mRNA-1273 third dose (Fig. 2). As expected, we found a nearly identical risk pattern in the two vaccine groups in the evaluations of two negative outcome controls7: symptomatic COVID-19 during the first 7 d after the third vaccine dose (Extended Data Fig. 2) and death from causes other than COVID-19 during follow-up (Extended Data Fig. 3).

Fig. 2: Cumulative incidence of Covid-19 outcomes during a period spanning Delta- and Omicron-variant predominance (20 October 2021 to 15 February 2022).
figure 2

a–e, Documented SARS-CoV-2 infection (a), symptomatic COVID-19 (b), COVID-19 hospitalization (c), COVID-19 ICU admission (d) and COVID-19 deaths (e) during a period spanning Delta- and Omicron-variant predominance (20 October 2021 to 15 February 2022). Solid blue line represents the risk curve for BNT162b2, dashed orange line represents the risk curve for mRNA-1273 and shaded areas represent pointwise 95% confidence intervals.

Source data

Full size image

The estimated 16-week risk ratios (95% CI) for recipients of a third dose of BNT162b2 as compared with mRNA-1273 were 1.15 (1.06, 1.30) for documented SARS-CoV-2 infection, 1.21 (1.12, 2.14) for symptomatic COVID-19, 1.64 (1.27, 2.79) for COVID-19 hospitalization, 1.37 (0.67, 3.14) for COVID-19 ICU admission and 1.08 (0.46, 6.39) for COVID-19 death (Table 2). The estimated risk differences (BNT162b2 minus mRNA-1273), expressed as events over 16 weeks per 10,000 persons (95% CI), were 45.4 (19.4, 84.7) for documented SARS-CoV-2 infection, 3.7 (2.2, 14.1) for symptomatic COVID-19, 10.6 (5.1, 19.7) for COVID-19 hospitalization, 2.0 (−3.1, 6.3) for COVID-19 ICU admission and 0.2 (−2.2, 4.0) for COVID-19 death.

Table 2 Estimated comparative effectiveness of a third dose of BNT162b2 and mRNA-1273 vaccines during a period spanning Delta- and Omicron-variant predominance (20 October 2021 to 15 February 2022)
Full size table

Estimates were similar across subgroups defined at baseline according to age, race, months since completion of COVID-19 vaccine primary series (Table 3) and by vaccine type of primary series (risk ratio for documented infection comparing a third dose of BNT162b2 vs mRNA-1273, 1.19 (0.98, 2.65) among recipients of a BNT162b2 primary series, 1.04 (0.84, 2.32) among recipients of an mRNA-1273 primary series). Estimates were also similar in sensitivity analyses that excluded eligible individuals with previously documented SARS-CoV-2 infection (6.3%), those who did not receive a homologous COVID-19 vaccine primary series (3.3%) and those whose third dose could not be identified as a booster dose (17.2%) (data not shown).

Table 3 Estimated comparative effectiveness of a third dose of BNT162b2 and mRNA-1273 vaccines in subgroups defined by baseline characteristics during a period spanning Delta- and Omicron-variant predominance (20 October 2021 to 15 February 2022)
Full size table

Secondary analyses during a period of SARS-CoV-2 Omicron-variant predominance

Study population and follow-up

Among 25,557 eligible recipients of a BNT162b2 third dose and 36,809 eligible recipients of an mRNA-1273 third dose, 7,894 BNT162b2 recipients were matched to 7,894 mRNA-1273 recipients. Compared with the eligible population, the matched population was generally similar with respect to baseline demographic and clinical characteristics but included a higher percentage of men, White individuals, individuals with an urban residence and individuals who had received no SARS-CoV-2 tests in the past year (Supplementary Table 1). Baseline characteristics of the matched persons are shown in Supplementary Table 2. All variables were well-balanced between the two vaccine groups (Extended Data Fig. 1).

As compared with the matched population during a period spanning Delta- and Omicron-variant predominance, the matched population during a period of Omicron-variant predominance was, on average, younger and included a higher percentage of individuals who had received no SARS-CoV-2 tests in the previous year and no influenza vaccinations in the previous 5 yr at a VA facility (Supplementary Table 1).

The median follow-up was 54 d (interquartile range, 40–60 d). Over a 9-week follow-up, 214 SARS-CoV-2 infections were documented.

Comparative effectiveness

Over a 9-week period of Omicron-variant predominance, the estimated risk of documented SARS-CoV-2 infection was also higher with a third dose of the BNT162b2 vaccine than with a third dose of the mRNA-1273 vaccine; the estimated risk ratio was 1.57 (1.12, 2.10) and the estimated risk difference, expressed as events over 9 weeks per 10,000 persons, was 63.2 (15.2, 100.7) (see also Fig. 3).

Fig. 3: Cumulative incidence of documented SARS-CoV-2 infection during a period of Omicron-variant predominance (1 January to 8 March 2022).
figure 3

Solid blue line represents the risk curve for BNT162b2, dashed orange line represents the risk curve for mRNA-1273 and shaded areas represent pointwise 95% confidence intervals.

Source data

Full size image

Discussion

We quantified the comparative effectiveness of a third dose of the BNT162b2 and mRNA-1273 vaccines for the prevention of COVID-19 outcomes in the largest integrated healthcare system in the United States. Although the risks of all five measured COVID-19 outcomes (documented infection, symptomatic COVID-19 and COVID-19-related hospitalization, ICU admission and death) over 16 weeks were low for both vaccines during a period spanning Delta- and Omicron-variant predominance (risks <4% for documented infection and <0.03% for death, within each vaccine group), recipients of a third dose of BNT162b2 had an excess per 10,000 persons of 45 documented infections and 11 hospitalizations compared with recipients of a third dose of mRNA-1273. We also found a higher risk of documented infection among recipients of a third dose of BNT162b2 compared with mRNA-1273 over 9 weeks of follow-up during a period of Omicron-variant predominance, although this estimate was less precise due to a smaller number of eligible persons.

Few head-to-head comparisons of a third dose of mRNA vaccines are available. A report from Spain estimated mRNA-1273 boosters to be 13% more effective than BNT162b2 boosters for the prevention of documented SARS-CoV-2 infection during a period of Omicron-variant predominance, but more severe outcomes could not be evaluated4. In two other reports, effectiveness against symptomatic COVID-19 was evaluated separately for each mRNA booster either compared with no vaccination (in two test-negative case-control studies in England5) or compared with the primary series (in two matched cohort studies in Qatar6). In each report, an indirect comparison of these results requires that the controls from each study had similar characteristics, and more severe outcomes could not be evaluated for both mRNA booster groups.

The differential effectiveness we report in our study might be explained by the higher mRNA content of mRNA-1273 (50 μg for booster doses, 100 μg for third primary doses) compared with of BNT162b2 (30 μg for booster and third primary doses). Our primary analysis considered any third dose of these vaccines, of which 83% were distinguished as booster doses, and results were similar in sensitivity analyses that were restricted to these.

The strengths of our study are that first, the VA healthcare databases contain rich data on demographics and medical history, which allowed us to carefully match recipients of each vaccine type according to key confounders. Second, the databases contain detailed information on laboratory test results and healthcare encounters, which allowed us to capture outcomes related to COVID-19 in both outpatient and inpatient settings. Third, the large size of the study population allowed us to evaluate less common COVID-19 outcomes (hospitalizations, ICU admissions, deaths). Fourth, the demographic composition of the US veteran population allowed us to provide evidence for a diverse cohort (24% Black, 8% Hispanic) and to conduct subgroup analyses among older persons (≥70 yr of age) and Black persons.

Our study also has several potential limitations. First, as in any observational analysis of comparative effectiveness, the vaccine groups could differ with respect to risk factors for the outcomes. However, after rigorously matching recipients of each vaccine type, the two groups had similar demographics, comorbidities, time since completion of the primary vaccination series, number of SARS-CoV-2 tests in the previous year and markers of healthcare utilization (for example, number of primary care visits and of influenza vaccinations in the previous 5 yr). Further, much less confounding is expected when comparing recipients of different third doses of vaccines than when comparing recipients of a third dose with, for example, unvaccinated persons. In addition, our two analyses involving negative-outcome controls suggested little confounding.

Second, the possibility of outcome misclassification cannot be ruled out if veterans sought testing or care outside the VA healthcare system or if testing was differential between groups. However, our use of the VA COVID-19 National Surveillance Tool allowed us to integrate laboratory data with clinical notes to capture infections documented inside and outside the system, and the SARS-CoV-2 testing frequencies over follow-up were similar in both groups (Table 2). Further, we selected regular VA users who had a known residential address and who had routinely collected information (body-mass index, smoking status) to improve outcome ascertainment. We expect any residual incomplete ascertainment to be non-differential between the vaccination groups under comparison, although absolute risks may be slightly biased downwards. Finally, our study population was mostly made up of men (96%) and older persons (95% were >50 yr old), which may limit the generalizability of our findings.

Given the high effectiveness of a third dose of both the BNT162b2 and mRNA-1273 vaccines, either vaccine is strongly recommended to any individual. This study provides evidence of clear and comparable benefits of these vaccines for the most severe outcomes (the difference in estimated 16-week risk of death between the two groups was two-thousandths of one percent). While the differences in estimated risk for less severe outcomes between the two groups were small on the absolute scale, they may be meaningful when considering the population scale at which these vaccines are deployed. However, decisions about vaccination campaigns are complex and must incorporate considerations that extend beyond the scope of the present study. Therefore, our findings need to be combined with those from previous studies to inform future decisions about choice of vaccine in coordinated public health responses.

In summary, although the absolute risks of breakthrough SARS-CoV-2 infection and more severe COVID-19 outcomes were low regardless of the third mRNA-based COVID-19 dose received, this study involving a nationwide cohort of US veterans provides evidence of a lower 16-week risk of COVID-19-related outcomes among recipients of a third dose of the mRNA-1273 vaccine compared with recipients of a third dose of the BNT162b2 vaccine, particularly for documented infection. This pattern was consistent across a period spanning Delta- and Omicron-variant predominance and a period restricted to Omicron-variant predominance. Further evaluation of the comparative effectiveness and safety of additional doses of these vaccines is needed.

Methods

Specification of the target trials

We designed an observational analysis to emulate a hypothetical pragmatic trial of a third dose of BNT162b2 compared with mRNA-1273 and risks of COVID-19 outcomes in the VA healthcare system. Supplementary Table 3 summarizes the key protocol components.

Eligibility criteria included veteran status; age ≥65 yr or 18–64 yr with high risk of severe COVID-19 between 20 October and 18 November 2021 (based on the presence of at least one coexisting condition listed in Table 1)8 or ≥18 yr between 19 November 2021 and 8 February 2022 (based on national guidelines for third dose deployment)9,10; receipt of the second dose of an mRNA vaccine primary series at least 6 months earlier (based on the same guidelines); known residential address outside of a long-term care facility; and known smoking status and known body-mass index within the previous year. Individuals had to have used the VA healthcare system during the previous year (defined as receiving care at a station eligible to administer the vaccines under study and having at least one primary care visit) but not within the previous 3 d (which may indicate the start of symptomatic disease).

The interventions of interest were a third dose of either the BNT162b2 or the mRNA-1273 vaccine. To ensure balance of important characteristics across groups, eligible veterans in the target trial would be randomly assigned to one of these two vaccines within strata defined by calendar date of the third dose (5 d bins), calendar month of the second mRNA vaccine dose, age (5 yr bins), sex (male, female), race (White, Black, other, unknown), urbanicity of residence (urban, not urban), geographic location (coded as 19 categories of Veterans Integrated Services Network) and number of SARS-CoV-2 tests performed in the past 12 months (0, 1, ≥2).

The five outcomes of interest were documented SARS-CoV-2 infection, documented symptomatic COVID-19 and COVID-19-related hospital admission, ICU admission and death. For each eligible individual, follow-up started on the day the third dose of vaccine was received (baseline) and ended on the day of the outcome of interest, death, 112 d (16 weeks) after baseline, or the end of the study period (15 February 2022), whichever happened first.

Our target trial evaluates comparative effectiveness of a third dose of the vaccines during a period spanning times when SARS-CoV-2 Delta- and Omicron-variants were circulating. The Delta variant had decreased to a share of 26% of circulating variants in the United States as of 25 December 2021, as it was rapidly displaced by the Omicron variant, which rose to a 100% share as of 12 February 202211. To evaluate the comparative effectiveness during a period of Omicron-variant predominance, we considered a second target trial that was identical to the first trial except that the recruitment period was 1 January to 1 March 2022, and the only outcome of interest was documented SARS-CoV-2 infection because the period was too short to accrue a sufficient number of rarer, more severe outcomes.

Emulation of the target trials

We emulated both target trials using the VA healthcare databases3. Vaccination was identified using records from the Corporate Data Warehouse and the VA COVID-19 Shared Data Resource. SARS-CoV-2 infections were identified using the VA COVID-19 National Surveillance Tool3,12, which integrates data on laboratory tests with natural language processing of clinical notes to capture diagnoses documented inside and outside the VA healthcare system. Symptomatic COVID-19 was defined as ≥1 of the following symptoms documented within 4 d of SARS-CoV-2 infection: fever, chills, cough, shortness of breath or difficulty breathing, sore throat, loss of taste or smell, headache, myalgia/muscle pain, diarrhoea and vomiting. COVID-19 hospitalization was defined as a hospitalization within 21 d after documented SARS-CoV-2 infection, COVID-19 ICU admission was defined as an ICU admission during a COVID-19 hospitalization, and COVID-19 death was defined as a death within 30 d after documented SARS-CoV-2 infection. Supplementary Table 4 provides detailed information on all study variables and their ascertainment.

We attempted to mimic the stratified randomization of the target trial by matching persons who received a third dose of BNT162b2 and of mRNA-1273 on the basis of the calendar date of the third dose, calendar month of the second dose, age, sex, race, urbanicity of residence, geographic location, and number of SARS-CoV-2 tests performed in the past 12 months, using the same matching algorithm described in our previous study3.

To explore the possibility of residual confounding (for example, by underlying health status or healthcare-seeking behaviour), we incorporated two negative outcome controls7. First, we evaluated the risk of symptomatic COVID-19 in the first 7 d after the third vaccine dose. Second, we evaluated the risk of death from causes other than COVID-19 during the follow-up period.

Statistical analyses

Covariate balance after matching was evaluated by plotting the mean differences between variable values (standardized for continuous variables) for the vaccination groups, with a difference of 0.1 or less considered to be acceptable13.

Cumulative incidence (risk) curves for the vaccine groups were estimated using the Kaplan-Meier estimator14. We then calculated 16-week risks of each outcome and compared them between the vaccine groups via differences and ratios. We conducted subgroup analyses by age (<70 or ≥70 yr), race (Black or White), time since completion of the COVID-19 vaccine primary series (6–7, 8, or ≥9 months) and vaccine type of the primary series (BNT162b2 or mRNA-1273). We conducted sensitivity analyses that excluded eligible individuals who (1) had previously documented SARS-CoV-2 infection, (2) did not receive a homologous third dose compared with the COVID-19 vaccine primary series and (3) had a third dose that could not be identified as a booster dose on the basis of procedural codes available in the ‘Inpatient’ and ‘Outpatient’ domains, to compare recipients of vaccines at known doses, as the dose of mRNA-1273 differs for booster doses (50 μg) vs third primary doses (100 μg).

We used a nonparametric bootstrapping procedure (including both matching and subsequent analyses) with 500 iterations to calculate percentile-based 95% confidence intervals for all estimates.

Analyses were performed using R software version 3.6.0 (R Foundation for Statistical Computing) and SAS Enterprise Guide version 8.2 (SAS Institute).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.