Note: The risk of bias by domain corresponds to the highest risk of bias among outcomes by domain.
The overall risk of bias corresponds to the overall highest risk of bias assessed among outcomes.
| Bias | Author's judgement | Support for judgement |
| Confounding |
Serious |
Analyses were adjusted for age, sex, race/ethnicity, frailty index (in quartiles), history of COVID-19 diagnosis, history of SARS-CoV-2 molecular test, number of outpatient and virtual visits, preventive care, Medicaid, neighborhood median household income, KPSC physician/employee status, medical center area. No adjustment for covid symptoms at time of potential vaccination - a potentially important confounder. |
| Selection of participants into the study |
Low |
No particular concerns in this domain - the study population consisted of all those >18 years old on the Kaiser Permanente Southern California healthcare system with ≥1 year of prior membership through 14 days after the index date (date of second vaccine for vaccinated individuals and their matched unvaccinated counterparts). |
| Clasification of interventions |
Low |
No particular concerns in this domain - vaccination status is likely to be adequately measured (obtained from electronic health records). |
| Deviations from intervention |
Low |
No concerns in this domain - the study was observational. |
| Missing outcome data |
Low |
No particular concerns in this domain - analyses of electronic medical records and data tables do not raise concerns with regard to missing data. |
| Measurement of the outcome |
Moderate |
No particular concerns regarding determination of severe disease or mortality. However, there is the potential for (differential) misclassification of infection status and symptomatic disease. |
| Selection of the reported results |
Moderate |
We were unable to locate a protocol or statistical analysis plan. The authors inform us that there was one (personal correspondence), but we are unable to check whether the analyses presented follow the prespecified plan. |
| Overall risk of bias |
Serious |
|
| Overall comment | Concerns regarding the lack of adjustment for covid symptoms at the time of potential vaccination - a potentially important confounder. |