Note: The risk of bias by domain corresponds to the highest risk of bias among outcomes by domain.
The overall risk of bias corresponds to the overall highest risk of bias assessed among outcomes.
| Bias | Author's judgement | Support for judgement |
| Confounding |
Serious |
The analysis adjusted for age, sex, ethnicity, race and comorbidities. It did not adjust for socioeconomic status, geographic location, health seeking behaviour, COVID-19 symptoms at the time of potential vaccination or hospitalization/need for health care. |
| Selection of participants into the study |
Serious |
Veterans were selected for inclusion in the analysis if they received a PCR test during the vaccinated or unvaccinated period. No information is provided about what determined whether or when individuals received a PCR test. It may depend importantly on both vaccination status (as acknowledged by the authors) and presence of infection, which would lead to selection bias. |
| Clasification of interventions |
Low |
No concerns in this domain - vaccination status is likely to be adequately measured. |
| Deviations from intervention |
Low |
No concerns in this domain - the study was observational. |
| Missing outcome data |
Moderate |
In addition to the concerns addressed in Domain 2, there is likely to be informative censoring when unvaccinated individuals became vaccinated. People who become vaccinated are likely to be different from those who do not. They might not have got vaccinated if they developed COVID-19 symptoms, although the number of these may be relatively small. |
| Measurement of the outcome |
Moderate |
The most recent PCR test during the study period was used to determine infection status, and this may be unlikely to pick up asymptomatic cases (as acknowledge by the authors). Furthermore, testing could be differential by vaccination status, leading to detection bias. |
| Selection of the reported results |
Moderate |
There is no evidence of a protocol, and we have concerns that results could have been selected for reporting because of the findings. |
| Overall risk of bias |
Serious |
|
| Overall comment | The preprint was used in data extraction and risk of bias assessment. Infection results are presented for the subgroup tested in July 2021.
Concerns over uncontrolled confounding and potential for selection bias due to restriction of the analysis sample to those who received a PCR test during the vaccinated or unvaccinated follow up time. |