Note: The risk of bias by domain corresponds to the highest risk of bias among outcomes by domain.
The overall risk of bias corresponds to the overall highest risk of bias assessed among outcomes.
| Bias | Author's judgement | Support for judgement |
| Randomization |
Low |
Quote: "Participants were randomly assigned to 1 of the 2 vaccine candidate groups or a control group receiving aluminum hydroxide (alum) adjuvant only (in a 1:1:1 ratio), according to unique serial numbers generated by an independent statistician. A stratified block randomization method was used, with study site as the stratification factor and block size in each stratum of 15. The concealed random grouping allocation and blind codes were kept in signed and sealed envelopes and were blinded to the investigators, participants, and statisticians."
Comment: Allocation sequence random. Allocation sequence concealed. Minor imbalances in baseline characteristics were likely due to chance. Risk assessed as low. |
| Deviations from intervention |
Some concerns |
Quote: "The concealed random grouping allocation and blind codes were kept in signed and sealed envelopes and were blinded to the investigators, participants, and statisticians" "The vaccines and controls were approved by the National Institutes for Food and Drug Control of China, and were supplied in coded, identical-appearing, single-dose vials." (report) "Masking: Quadruple (Participant, Care Provider, Investigator, Outcomes Assessor)" (registry)
Comment: Blinded study (participants and personnel/carers). SAFETY All randomized participants who received at least one dose were analyzed for safety outcomes. As we are assessing the effect of assignment to intervention, the analysis method performed on these safety outcomes, was considered appropriate. Risk assessed to be low for outcomes: Local adverse events. Systemic adverse events. Adverse events. Serious adverse events. EFFICACY All randomized participants who received at least one dose were analyzed for the outcome: All-cause mortality As we are assessing the effect of assignment to intervention, the analysis method performed on these safety outcomes, was considered appropriate. Risk assessed to be low for the outcome: All-cause mortality Per-protocol analysis was performed on the efficacy outcomes (as planned in the trial protocol). Reasons for exclusions were balanced between treatment groups: due to positive or unknown baseline SARS-CoV-2 RT-PCR status (118 vs 134 vs 98), did not receive any injection (11 vs 5 vs 13), did not receive second dose (393 vs 379 vs 387). As we are assessing the effect of assignment to intervention (intention-to-treat effect), we considered that the data were analyzed inappropriately. There was no substantial impact of failure to analyze participants according to their randomized assignment. Risk assessed to be some concerns for the outcomes: Confirmed COVID-19. Confirmed symptomatic COVID-19. Severe or critical COVID-19. |
| Missing outcome data |
Low |
Comment: 40,411 participants randomized; 40,382 participants analyzed for safety; 38,206 participants analyzed for efficacy (94.5%).
SAFETY Data available for nearly all participants randomized. Risk assessed to be low for the outcomes: All-cause mortality. Local adverse events. Systemic adverse events. Adverse events. Serious adverse events. EFFICACY Data available for nearly all participants randomized for the outcome All-cause mortality. Risk assessed to be low for the outcome: All-cause mortality. Data not available for all or nearly all participants randomized (5.5% missing data) for the outcomes: Confirmed COVID-19. Confirmed symptomatic COVID-19. Severe or critical COVID-19. No evidence that the result is not biased. Reasons for missing data: Most patients were excluded because the planned efficacy analysis was a per-protocol analysis and the bias has been taken into account in domain 2. Missingness could not depend on the true value of the outcome. Risk assessed to be low for the outcomes: Confirmed COVID-19. Confirmed symptomatic COVID-19. Severe or critical COVID-19. |
| Measurement of the outcome |
Low |
Comment: Method of measuring the outcome probably appropriate.
Measurement or ascertainment of outcome probably does not differ between groups. Blinded study (outcome assessor). Risk assessed to be low for the outcomes: Confirmed COVID-19. Confirmed symptomatic COVID-19. Severe or critical COVID-19. All-cause mortality. Local adverse events. Systemic adverse events. Adverse events. Serious adverse events. |
| Selection of the reported results |
Low |
Comment: The protocol, statistical analysis plan and two registries were available (ChiCTR posted 2020-07-18, recruitment start 2020-07-16, considered a prospective registration as explicitly reported as such and allowing for a delay between submission and final registration; NCT retrospective: 2020-08-12).
Outcomes were pre-specified. Results were not selected from multiple outcome measurements or analyses of the data. Trial analyzed as pre-specified. Risk assessed to be low for the outcomes: Confirmed COVID-19. Confirmed symptomatic COVID-19. Severe or critical COVID-19. All-cause mortality. Local adverse events. Systemic adverse events. Adverse events. Serious adverse events. |
| Overall risk of bias |
Some concerns |