Note: The risk of bias by domain corresponds to the highest risk of bias among outcomes by domain.
The overall risk of bias corresponds to the overall highest risk of bias assessed among outcomes.
| Bias | Author's judgement | Support for judgement |
| Randomization |
Low |
Interim report (Baden et. al): "Participants were randomly assigned in a 1:1 ratio, through the use of a centralized interactive response technology system, to receive vaccine or placebo. Assignment was stratified, on the basis of age and Covid-19 complications risk criteria, into the following risk groups: persons 65 years of age or older, persons younger than 65 years of age who were at heightened risk (at risk) for severe Covid-19, and persons younger than 65 years of age without heightened risk (not at risk)."
Protocol: "The randomization will be in a blinded manner using a centralized Interactive Response Technology (IRT), in accordance with pre-generated randomization schedules" Comment: Allocation sequence random. Allocation sequence concealed. Minor imbalances in baseline characteristics appear to be compatible with chance. Risk assessed to be low. |
| Deviations from intervention |
Some concerns |
Quote: "The investigator, study staff, study participants, site monitors, and Sponsor personnel (or its designees) will be blinded to the IP administered until study end" (protocol) “Masking: Quadruple (Participant, Care Provider, Investigator, Outcomes Assessor” (registry)
Comment: Blinded study (participants and personnel/carers). SAFETY All randomized participants who received at least one dose were analyzed for safety outcomes. As we are assessing the effect of assignment to intervention, the analysis method performed on these safety outcomes, was considered appropriate. Risk assessed to be low for outcomes: Mortality. Local adverse events. Systemic adverse events. Adverse events. Serious adverse events. EFFICACY Per-protocol analysis was performed on the efficacy outcomes (as planned in the trial protocol). Reasons for exclusions were balanced between treatment groups (922 [6.1%] vs 1042 [6.9%]), with the majority of those excluded due to positive or unknown baseline SARS-CoV-2 status (434 vs 421). Other reasons: did not receive any injection (29 vs 40), Received an incorrect injection (6 vs 7), Discontinued without receiving second dose (334 vs 425), Received dose 2 outside planned time frame (102 vs 119), Other major protocol deviation (17 vs 30). As we are assessing the effect of assignment to intervention (intent-to-treat effect), we considered that the data were analyzed inappropriately. There was no substantial impact of failure to analyze participants according to their randomized assignment due to the small number. Risk assessed to be some concerns for the outcomes: Confirmed COVID. Confirmed symptomatic COVID. Confirmed severe COVID. IMMUNOGENICITY Per-protocol analysis was performed on the immunogenicity outcomes (as planned in the trial protocol). Quote: "Participants in the FAS with nonmissing baseline characteristics for the stra- ta who received both planned injections (injection 2 received within 21–42 days post–injection 1), no major protocol devia- tions, and serum samples available at both days 1 (baseline) and 57 were eligible for inclusion in the prespecified random subset" As we are assessing the effect of assignment to intervention (intent-to-treat effect), we considered that the data were analyzed inappropriately. There was probably no substantial impact of failure to analyze all participants according to their randomized assigment since baseline characteristics of the immunogenicity analysis population were generally balanced by treatment group. Risk assessed to be some concerns for the outcome: Neutralizing antibody GMT. |
| Missing outcome data |
Low |
Comment: 30,415 participants randomized; 30,317 to 30,346 participants analyzed for safety outcomes; 28,451 participants analyzed for efficacy outcomes. 1457 participants analyzed for immunogenicity outcomes.
SAFETY Data available for nearly all participants randomized. Risk assessed to be low for the outcomes: Mortality. Local adverse events. Systemic adverse events. Adverse events. Serious adverse events. EFFICACY Data not available for all or nearly participants randomized. No evidence that the result is not biased. Reasons: most patients were excluded because the planned analysis was a per-protocol analysis and the bias had been taken into account in domain 2. Risk assessed to be low for outcomes: Confirmed COVID. Confirmed symptomatic COVID. Confirmed severe COVID. IMMUNOGENICTY Data not available for all or nearly participants randomized. No evidence that the result is not biased. Reasons: most patients were excluded because the planned analysis was a per-protocol analysis and the bias had been taken into account in domain 2. Risk assessed to be low for outcome: Neutralizing antibody GMT. |
| Measurement of the outcome |
Low |
Quote: “Masking: Quadruple (Participant, Care Provider, Investigator, Outcomes Assessor” (registry) "COVID-19 Cases were adjudicated by a blinded committee"
Comment: Blinded study (outcome assessor). Risk assessed to be low for the outcomes: Confirmed COVID. Confirmed symptomatic COVID. Confirmed severe COVID. Mortality. Neutralizing antibody GMT. Local adverse events. Systemic adverse events. Adverse events. Serious adverse events. |
| Selection of the reported results |
Low |
Comment: The prospective protocol (dated June 15th 2020) and registry (dated July 11th, 2020) were available.
Outcomes pre-specified. Results were not selected from multiple outcome measurements or analyses of the data. Trial analyzed as pre-specified. Risk assessed to be low for the outcomes: Confirmed COVID. Confirmed symptomatic COVID. Confirmed severe COVID. Mortality. Neutralizing antibody GMT. Local adverse events. Systemic adverse events. Adverse events. Serious adverse events. |
| Overall risk of bias |
Some concerns |