Note: The risk of bias by domain corresponds to the highest risk of bias among outcomes by domain.
The overall risk of bias corresponds to the overall highest risk of bias assessed among outcomes.
| Bias | Author's judgement | Support for judgement |
| Confounding |
Serious |
The analysis adjusted for age, sex, race/ethnicity, BMI, cardiac disease, organ transplant, diabetes (by A1C), chronic obstructive pulmonary disease, renal disease, malignancy, hypertension, Charlson index, healthcare utilization in year prior to index date (outpatient, virtual, ED, inpatient), influenza and pneumococcal vaccinations in year prior, neighborhood deprivation index (NDI), and prior infection with SARS-CoV-2. Although this addresses almost all of our prespecified confounding domains, covid symptoms at the time of potential vaccination which is a potentially important confounder, was not adjusted for. There is also potential for uncontrolled confounding, given the observational nature of the data. |
| Selection of participants into the study |
Low |
No particular concerns in this domain - the study population consisted of all those >12 years old on the Kaiser Permanente Southern California healthcare system (with ≥1 year of prior membership). |
| Clasification of interventions |
Low |
No particular concerns in this domain - vaccination status is likely to be adequately measured (obtained from electronic health records) |
| Deviations from intervention |
Low |
No concerns in this domain - the study was observational. |
| Missing outcome data |
Moderate |
45% of PCR-positive samples sent for sequencing could not be sequenced (thus the presence of the delta variant could not be determined). No other concerns regarding missing data. |
| Measurement of the outcome |
Moderate |
No particular concerns regarding determination of severe disease (hospitalization). However, there is the potential for (differential) misclassification of infection status. |
| Selection of the reported results |
Moderate |
There is no evidence of a protocol or analysis plan and we have concerns that results could have been selected for reporting because of the findings. |
| Overall risk of bias |
Serious |
|
| Overall comment | In addition to the preprint, the study registry record was used in data extraction and risk of bias assessment. |