Comparing Effect Estimates in Randomized Trials and Observational Studies From the Same Population: An Application to Percutaneous Coronary Intervention

Background To understand when results from observational studies and randomized trials are comparable, we performed an observational emulation of a target trial designed to ask similar questions as the VALIDATE (Bivalirudin Versus Heparin in ST‐Segment and Non–ST‐Segment Elevation Myocardial Infarction in Patients on Modern Antiplatelet Therapy) randomized trial. The VALIDATE trial compared the effect of bivalirudin and heparin during percutaneous coronary intervention on the risk of death, myocardial infarction, and bleeding across Sweden. Methods and Results We specified the protocol of a target trial similar to the VALIDATE trial, then emulated the target trial in the period before the VALIDATE trial took place using data from the SWEDEHEART (Swedish Web System for Enhancement and Development of Evidence‐Based Care in Heart Disease Evaluated According to Recommended Therapies) registry—the same registry in which the trial was undertaken. The target trial emulation and the VALIDATE trial both estimated little or no effect of bivalirudin versus heparin on the risk of death or myocardial infarction by 180 days (target trial emulation risk ratio for death, 1.21 [95% CI, 0.88 – 1.54]; VALIDATE trial hazard ratio for death, 1.05 [95% CI, 0.78 – 1.41]). The observational data, however, could not capture less severe cases of bleeding, resulting in an inability to define a bleeding outcome like the trial, and could not accurately estimate the comparative risk of death by 14 days, which may be the result of intractable confounding early in follow‐up or the inability to precisely emulate the trial’s eligibility criteria. Conclusions Using real‐world data to emulate a target trial can deliver accurate effect estimates. Yet, even with rich observational data, it is not always possible to estimate the short‐term effect of interventions or the effect on outcomes for which data are not routinely collected.


= ( )/ ( | )
Second, we fit pooled logistic regression models where individuals are weighted by their estimated SW A .
The predicted probabilities from the inverse probability weighted hazards model can then be multiplied over time, in the absence of censoring, to obtain an estimate of survival under each treatment strategy at each time, m. ̂= ∏ (α 0, + α 1 + α 2 + α 3 2 ) =1 The risk at time, k, can then be estimated by taking one minus ̂, and risk differences and risk ratios can be calculated. Finally, we use nonparametric bootstrapping with 500 samples to calculate all 95% confidence intervals.

Standardization models
First, we fit the pooled logistic regression model. We then use the predicted probabilities from these models, multiplied over time, in the absence of censoring, to obtain an estimate of survival of each individual, i, under each treatment strategy at each time, m, conditional on the individual's baseline confounders, Li.
Where n is the number of individuals in the study population. The risk at time, k, can then be estimated by taking one minus ̂, and risk differences and risk ratios can be calculated. Finally, we used nonparametric bootstrapping with 500 samples to calculate all 95% confidence intervals.

Sensitivity analysis 1: eligibility criteria 1
We expanded the definition of diagnoses that could indicate a life expectancy of less than a year (one of the exclusion criteria) to include all cancer diagnoses (ICD-10 code: C).

Sensitivity analysis 2: eligibility criteria 2
We excluded all patients over the age of 75 years at the time of PCI.

Sensitivity analysis 3: eligibility criteria 3
We only included those who were recorded as given P21Y2 before percutaneous coronary intervention, not under percutaneous intervention

Sensitivity analysis 4: eligibility criteria 4
We included everyone given any dose of heparin before coronary intervention

Sensitivity analysis 5: eligibility criteria 5
We included everyone with a missing GFR and imputed missing GFR and weight variables with the median GFR and weight in the analysis.

Sensitivity analysis 6: treatment strategy
We restricted treatment to those who were only given bivalirudin or heparin (original definition is exposure to bivalirudin if given both treatments as assumed that heparin is low dose if have been given both). 1,466 of the 1,951 (75%) of the patients originally assigned to bivalirudin were also given low dose heparin at the same time. These patients were excluded from this analysis.

Sensitivity analysis 7: outcome
We expanded the definition of bleeding to include less severe bleeding outcomes. The additional ICD-10 codes we identified in the inpatient and outpatient registers were:

Sensitivity analysis 8: confounders
We changed the definition to the confounders that were defined in the inpatient, outpatient, and prescribed drug registers (bleeding, warfarin, and NOAC), so they were identified from a 5-year look back period rather than 3-year.

Sensitivity analysis 9: missing data
We carried out a complete case analysis, excluding patients that had missing data for the Killip class and anemia variables (18%).

Sensitivity analysis 10: censoring at death
In the analyses for the myocardial infarction and bleeding outcomes, we censored individuals at date of death.

RIKSHIA a. < 2 days after PCI
If there was a record of a myocardial infarction in the RIKSHIA registry within 2 days after PCI, then there also needed to be record of troponin levels greater than 40ng/L (both high sensitivity and conventional troponin, and I and T troponin) or a reinfarction. This was to ensure the record was not a repeat record of the original myocardial infarction.
b. >2 days after PCI After 2 days, through until the end of follow up, all myocardial infarction records in the RIKSHIA registry were used (ICD 10 codes I21/I22).

Inpatient and outpatient registries
The patient registries were used to identify primary diagnoses of intracranial bleeding; major gastrointestinal bleeding; respiratory, renal/urinary tract, ocular, retroperitoneal or pericardial bleeding; and bleeding attributed to anemia.
These outcomes were identified using the following ICD-10 codes:

RIKSHIA
The RIKSHIA registry also records if a patient bled within a period of care. This was used to identify lethal bleeding, cerebral bleeding, or bleeding that required transfusion.