Prediction Score for Anticoagulation Control Quality Among Older Adults

Introduction

Vitamin K antagonist (VKA; eg, warfarin) therapy is an effective anticoagulation option for stroke prevention in patients with nonvalvular atrial fibrillation (AF) and for treatment and secondary prevention of venous thromboembolism (VTE; including deep vein thrombosis and pulmonary embolism).^{1, 2, 3} The safety and effectiveness of VKAs, however, depends on regular international normalized ratio (INR) monitoring and anticoagulation control quality, often measured by the time in therapeutic range (TTR), for which INR 2.0 to 3.0 is the standard therapeutic range for AF and VTE.^{4, 5, 6} Patients on VKA with poor anticoagulation quality (ie, low TTR) have been shown to have a higher risk of thromboembolic and bleeding complications and thus a worse risk–benefit ratio.^{4, 7, 8}

Although clinical trials have shown that direct‐acting oral anticoagulants (DOACs) are therapeutically advantageous over or at least noninferior to VKAs,^{9, 10, 11} clinical equipoise still exists when patients are likely to have good anticoagulation control based on pretreatment characteristics.^{7, 12} This choice is particularly difficult to make in older adults because DOACs have been associated with a higher risk of major gastrointestinal bleeding than VKAs in the older population.^{13, 14, 15} Moreover, chronic kidney disease is highly prevalent in older adults,¹⁶ which makes lack of routine monitoring tests for DOACs a challenge rather than an advantage because some DOACs are substantially renally excreted (eg, 80% for dabigatran). Consequently, it is critical to understand how patient characteristics are associated with anticoagulation quality so we can identify the ideal candidates for VKA therapy.

In the existing literature, there is only 1 published prediction score for anticoagulation quality: the SAMe‐TT₂R₂ score.¹⁷ It did not consider some clinically important predictors for TTR (eg, polypharmacy, hospitalizations, antibiotic use)^{18, 19, 20, 21, 22} and was found to have suboptimal performance in external validation populations (area under receiver operating characteristic curve [AUC] for relevant clinical end points <0.6).^{23, 24, 25} In addition, although the majority of oral anticoagulant users are older adults,^{3, 18} SAMe‐TT₂R₂ was developed with 52.7% of the population aged <70 years. Because comorbidity profiles vary substantially by age, the generalizability and applicability of SAMe‐TT₂R₂ in the older population is unclear.

We aimed to develop and validate a new prediction model for TTR, particularly in patients aged ≥65 years taking VKA for nonvalvular AF or VTE. Because prior studies found that the TTR predictors identified in AF patients were similar to those in VTE patients,¹⁸ for clinical simplicity we developed 1 score for both indications but validated the performance in patients with nonvalvular AF and VTE separately.

Methods

Data Source

We linked electronic health record (EHR) data from 2 large US academic provider networks with Medicare claims data. The first network consists of 1 tertiary hospital, 2 community hospitals, 17 primary care centers, and 1 anticoagulation clinic that manages VKA‐related care for all patients within the network. The second network includes 1 tertiary hospital, 1 community hospital, 16 primary care centers, and an anticoagulation clinic. Patients in network 1 were used as the training set for the prediction model derivation, and those in network 2 were used as the validation set. The EHR database contains information on patient demographics, diagnosis and procedure codes, medications, lifestyle factors, laboratory data, and various clinical notes. Both inpatient and outpatient EHR data were used in this study. The Medicare claims data contain information on demographics, inpatient and outpatient diagnosis and procedure codes, and dispensed medications.²⁶ This study was approved by Partners HealthCare Institutional Review Board (IRB).

Study Population

In the linked Medicare claims–EHR data, we identified all patients aged ≥65 years with nonvalvular AF or VTE initiating a VKA from January 1, 2007, to December 31, 2014, with no use of any oral anticoagulants (VKAs or DOACs) in the prior 90 days (new user design²⁷). The VKA initiation date was the index (cohort entry) date. The study cohort was required to have at least 180 days of continuous enrollment in Medicare inpatient, outpatient, and prescription benefits with at least 1 EHR encounter with date of service after January 1, 2007, and before the index date. To ensure our ability to assess the primary outcome reliably, patients were required to have at least 5 INR values recorded in the system. To assess whether this requirement would select an unrepresentative cohort, we compared the distributions of the combined comorbidity score²⁸ in those with versus without at least 5 INRs. We computed standardized differences between proportions of each combined comorbidity score category in those with versus without 5 INRs. A standardized difference of <0.1 was used to indicate an acceptable discrepancy.²⁹

Outcome Definition of the Anticoagulation Control

We calculated TTR using Rosendaal's method,³⁰ which assigns an INR value to each day by linear interpolation of successive observed INR values with gaps <56 days. After interpolation, we computed the proportions of time that fell within the therapeutic range (INR 2.0–3.0). We ascertained TTR starting the 29th day after the index date until the earliest of the following: 12 months after the index date, lack of INRs with a gap ≤56 days, death, discontinuation of VKA, or study end (December 31, 2014). We did not assess TTR for the first month because variability of INR values in the first month generally reflects expected fluctuations in INRs during the titration phase of VKA therapy. VKA discontinuation was defined based on an algorithm validated in a prior study in which high agreement with actual VKA use was demonstrated by chart review (κ=0.84).³¹

Candidate Predictors and Building of the TTR Prediction Model

We conducted a systematic review to identify original articles reporting predictors of anticoagulation control quality (assessed by TTR or INR variability) in users of VKAs, after multivariate adjustment. Figure 1 summarizes the search terms and the selection process. The significant predictors reported by the selected articles, along with variables deemed clinically important to predict anticoagulation quality, were used as the candidate predictors to build our prediction model. Based on these variables, we built a model predicting continuous TTR by Lasso regression with 5‐fold cross‐validation, using the data in the training set.³² We referred to the predicted TTR derived from this model as the geriatric TTR score. To build a simplified model for clinical use, we excluded biophysiologic variables requiring additional testing and used Lasso regression with a Bayesian information criterion, which tends to generate a more parsimonious model than do other criteria.³³ The points of the scoring system were the nearest integer proportional to the unstandardized coefficient in this simplified model. All predictors were assessed in the 180 days before (and including) the initiation of a VKA, with the exception of receiving structured anticoagulation management service, which was assessed until 28 days after VKA initiation (immediately before the start of the follow‐up).

• Download PowerPoint

Performance of the Geriatric TTR Score Versus SAMe‐TT₂R₂

We calculated a coefficient‐based and simplified version of the SAMe‐TT₂R₂ score for each patient (see Table S1 for details).¹⁷ Model performance was compared (1) between the geriatric TTR score and the coefficient‐based SAMe‐TT₂R₂ score and (2) between the simplified point system of the geriatric TTR score and the simple SAMe‐TT₂R₂ score. The validation set was subdivided into AF and VTE populations. We computed the AUC when predicting TTR >70%, a cutoff to indicate good anticoagulation quality in the literature.^{34, 35} We then computed the AUC when predicting incidence of a composite clinical outcome of stroke, systemic arterial embolism, VTE, and major bleeding (see detailed definitions in Table S2) that occurred between the 29th and 365th days following the index date (ie, the same ascertainment period as TTR). We also evaluated thromboembolic and bleeding events separately. In addition, we computed Hosmer–Lemeshow goodness‐of‐fit statistics to assess calibration of the models. The hypothesis testing for AUC comparison was done with methods proposed by DeLong et al.³⁶

Missing Data

The information on smoking and body mass index was recorded in the study EHRs as both structured data and text in the clinical notes. To reduce missing data, we used natural language processing³⁷ to extract information on these 2 variables from the clinical notes; this approach reduced the proportion of patients missing smoking data from 54.4% to 7.8% and of those missing body mass index from 38.5% to 32.2% (see Data S1 for details). For those still missing smoking and body mass index information after natural language processing and with other variables with missing data, we used the missing indicator method in the analysis.

Statistical Analyses

First, we tested the sensitivity of our results to the length of the baseline assessment period (365 instead of 180 days) and the definition of the new initiator of VKA (no use of VKA in the 180 days instead of 90 days before the index date). We calculated the Spearman correlation coefficient between the new scores based on revised strategies and the original score to quantify discrepancies. Second, to evaluate whether our results were sensitive to outliers or skewed the distribution of TTR, we repeated our analyses after (1) Box–Cox transformation of TTR³⁸ and (2) exclusion of those with extreme outcomes (TTR=0) from the analysis. The statistical analyses were conducted with SAS 9.4 (SAS Institute Inc).

Results

Systematic Review

From a total of 3692 studies, we selected 16 articles (Figure 1 summarizes the search and selection process). Among them, 11 articles investigated patients taking VKA for nonvalvular AF,^{17, 18, 20, 22, 39, 40, 41, 42, 43, 44, 45} 4 for mixed indications^{19, 21, 46, 47} and 2 for VTE.^{18, 48} Based on these selected articles, we identified 8 positive and 42 negative predictors (Table 1). To enrich the candidate predictor pool, we added 23 variables based on clinical knowledge (see Table S3 for the full list of candidate predictors and their definitions).

Patient Characteristics of the Study Population

Among 14250 VKA new initiators with at least 180 days of Medicare enrollment and 1 EHR encounter in the study system, we selected the study cohort with at least 5 INRs recorded in our database, including 1663 patients in the training set, 694 in the AF validation set, and 487 in the VTE validation set. The distribution of combined comorbidity score was similar in patients with versus without 5 INRs with a mean standardized difference of 0.02 between proportions of all combined comorbidity score categories in those included versus excluded (Figure S1). The mean TTR was 0.47 to 0.56 in our training and validation sets (Table 2 and Figure S2). We observed modest differences in some predictors in AF versus VTE validation populations (eg, higher prevalence of cancer and more use of antibiotics in the VTE; Table 2).

Prediction Models for TTR

From 50 predictors identified in the systematic review and 23 additional variables, we built the new geriatric TTR score with 42 predictors through lasso regression (R²=0.19; Table 3). The simplified model included a total of 7 variables (R²=0.14). We summarized these variables using the acronym PROSPER (Pneumonia, Renal dysfunction, Oozing blood [prior bleeding], Staying in hospital ≥7 days, use of Pain medications, lack of Enhanced [dedicated and structured] anticoagulation care, Rx for antibiotics; see Table 4 and Table S3 for detailed definitions). There was no significant difference between the AUCs of PROSPER versus the full geriatric TTR model predicting TTR >70% in the validation set (AUC 0.678 versus 0.680, P=0.86 for difference). The 2 most influential predictors of TTR were lack of participation in a dedicated anticoagulation management service (assigned 4 points) and renal dysfunction (assigned 2 points). The rest of the variables were assigned 1 point each.

Comparison of Performance: SAMe‐TT₂R₂ Versus Geriatric TTR Score

In the training set, the AUC for the geriatric TTR score predicting TTR >70% (AUC=0.71) was substantially larger than that for coefficient‐based SAMe‐TT₂R₂ (AUC=0.57, P<0.001 for difference); the AUC for the geriatric TTR score predicting the primary clinical outcome (AUC=0.65) was significantly larger than that for SAMe‐TT₂R₂ (AUC=0.53, P<0.001 for difference). The results were similar in the validation set (Figure 2). This pattern was consistent when the validation set was subdivided into AF and VTE validation sets (Table 5). We also found similar findings when the composite clinical outcome was subdivided into thromboembolic versus bleeding outcomes (Table S4). The Hosmer–Lemeshow goodness‐of‐fit test for predicting TTR >70% confirmed good calibration for the both the full geriatric model and coefficient‐based SAMe‐TT₂R₂ in the training and validation sets (Table S5).

• Download PowerPoint

Comparison of Performance: SAMe‐TT₂R₂ Simple Scoring System Versus PROSPER

In the training set, the AUC for PROSPER predicting TTR >70% (AUC=0.67) was substantially larger than that for SAMe‐TT₂R₂ (AUC=0.55, P<0.001 for difference); the AUC for PROSPER predicting the primary clinical outcome (AUC=0.62) was significantly larger than that for SAMe‐TT₂R₂ (AUC=0.52, P<0.001 for difference). A similar pattern was observed in the AF and VTE validation sets when predicting both types of outcomes (Table 5). Patients stratified by PROSPER had a clear decreasing trend of mean TTR, ranging from 0.71 to 0.30, in both the training and validation sets (Table 6).

Sensitivity Analyses

After changing the length of baseline assessment period from 180 to 365 days, the revised prediction score was highly correlated with the original one (Spearman coefficient=0.89). After defining new initiation of VKA as no use in the 180 days rather than 90 days before the index date, the revised prediction score was highly correlated with the original one (Spearman coefficient=0.99). The performance of these revised models was similar to that of the original model (Table S6). After Box–Cox transformation, the distribution of TTR became more symmetric (Fisher‐Pearson skewness coefficient⁴⁹ reduced by 46%), resulting in a prediction score highly correlated with the predicted value generated by the original model (Spearman coefficient=0.99). Similar patterns were found when excluding those with TTR 0 (Table S6).

Discussion

We developed and validated a new prediction score in the older adult population. Our geriatric TTR score included 42 predictors, and the simplified clinical scoring system, PROSPER, had 7 variables. The geriatric TTR score and PROSPER outperformed the corresponding coefficient‐based and simple version of SAMe‐TT₂R₂, available for the past 4 years, when predicting TTR ≥70% and thromboembolic and bleeding outcomes for those aged ≥65 years. The performance of PROSPER was not significantly worse than that of the full model in the validation set.

Physicians can use geriatric TTR scores to identify patients with good predicted TTR (>70%) as good candidates for VKA therapy for nonvalvular AF or VTE; otherwise, a DOAC may be preferred unless contraindicated. It is feasible to develop an automated program in an EHR system for computing the predicted TTR based on the full model as a clinical decision support tool; otherwise, PROSPER can be readily calculated without an aid. Our findings suggest that a PROSPER score >2 is predictive of having poor TTR; therefore, initiating a VKA may not be ideal. This cut point is associated with reasonable specificity (75%) for TTR >70% and sensitivity (85%; Table 7) for TTR <50% (another cut point suggested in the literature to indicate poor anticoagulation quality³⁴). Alternatively, the categorization of PROSPER as 0 to 2, 3 to 6, and ≥7 approximately subdivided the population into tertiles that correlated well with TTR. These 3 categories may be used to indicate low, moderate, and high risk of having poor TTR (Table 8). Our work highlights the importance of a structured approach to warfarin management; lack of a dedicated anticoagulation management service was found to be the strongest predictor of poor TTR. This finding is in line with several prior studies in which structured anticoagulation care was shown to improve TTR and to reduce risk of complications.^{50, 51, 52, 53} In the current era when DOACs are available, unstructured warfarin management is a particularly unattractive treatment option. If DOAC treatment is not possible and a patient has a PROSPER score >2, providers should encourage patient participation in a dedicated anticoagulation management service or some equivalently well‐organized warfarin treatment setting (eg, a practice with a nurse dedicated to managing warfarin). Renal dysfunction, defined as the presence of acute kidney injury, chronic kidney disease, or end‐stage kidney disease in the prior 180 days, was also found to be an important predictor of poor TTR. The anticoagulation decision is particularly difficult in AF patients with renal dysfunction, for whom there is uncertainty as to the net benefit of warfarin or DOACs with poor renal function. One approach can be to favor use of a DOAC that is less renally excreted (eg, apixaban) with necessary dose adjustment.

Our score can also be helpful in a research context. First, researchers can evaluate the potential treatment‐effect heterogeneity by levels of predicted anticoagulation quality based on the full model with a computer program. This assessment can provide direct evidence for choosing the ideal candidates for VKA versus DOACs based on the pretreatment characteristics predictive of anticoagulation quality. Next, our score can be used as a proxy adjustment tool for confounding by anticoagulation control quality. This adjustment is otherwise difficult because calculating TTR requires intensive INR recording in the study databases, which are often incomplete or nonexistent. Because some researchers do not have information on biophysiologic variables, we also presented an alternative model excluding these variables that requires additional testing (Table S7). This alternative model had performance similar to the geriatric TTR score (data not shown).

We have demonstrated that the performance of the new geriatric TTR score was clearly superior to that of SAMe‐TT₂R₂ in the older adult population. The authors of SAMe‐TT₂R₂ demonstrated good discrimination performance only when predicting those with TTR <5th percentile but not for those with TTR <25th percentile (AUC=0.58).¹⁷ However, the latter is closer to the clinical relevant cut points (eg, TTR >70% usually composes about 30% to 40% of the population^{17, 18, 34}). Because the majority of VKA users are older adults^{3, 18} who are also more vulnerable to developing bleeding complications,⁵⁴ we developed an alternative score dedicated to patients aged ≥65 years. We built a prediction model for both nonvalvular AF and VTE indications because prior studies found that AF and VTE patients share many risk factors for TTR¹⁸ and because including interaction terms with the VKA indications did not materially improve the model in our analysis. We validated the performance of our model in AF and VTE populations separately and found consistent results.

There are some important limitations. We used linked claims EHR data for higher data quality: The claims data provided comprehensive data across care settings, and EHRs provided necessary clinical information to ascertain TTR and important predictors. However, requiring overlap of the 2 databases reduces our sample size substantially and limits our ability to investigate each clinical outcome individually.⁵⁵ Besides, for biophysiologic variables (eg, body mass index, albumin levels), we had 29% to 34% people with missing data in the relevant period. We handled it by the missing indicator method because not having certain tests done could, by itself, be informative of the general health state, and this approach allows use of these scores even if some variables are not available. As a sensitivity analysis, the scores not including these variables with missing data were highly correlated with the original one and had similar model performance (data not shown). In addition, some of the possible determinants of TTR are not available in our data sets, such as diet information and genetic profiles associated with warfarin pharmacokinetics, which may limit our model performance. Consequently, our prediction scores should be used as an aid, not as the only ground for decision‐making. Next, we chose to build one prediction model for both nonvalvular AF and VTE indications because prior studies found that AF and VTE patients have many common risk factors for poor anticoagulation qualilty.¹⁸ We validated the performance of this score in AF and VTE populations separately and found consistent results. Nonetheless, we acknowledge the alternative approach to build separate models for different indications, which may increase specificity at the cost of simplicity and applicability. Last, the predictors identified in our study should not be interpreted as having causal effect on anticoagulation quality because they could be merely the markers or proxies of the real causal factors.

In conclusion, we developed and validated a prediction score for anticoagulation control quality quantified by TTR in the older adult population. It outperformed the published score, SAMe‐TT₂R₂, in patients aged ≥65 years when predicting TTR as well as thromboembolic and bleeding events. The full model of the geriatric TTR score can be used as an embedded algorithm within an EHR or for a research study. The simplified scoring system, PROSPER, had comparable performance and can be used in daily practice to help choose the best candidates to receive VKA therapy.

Sources of Funding

Lin received a stipend from the Pharmacoepidemiology program in the Department of Epidemiology, Harvard T.H. Chan School of Public Health and Department of Medicine, Brigham and Women's Hospital, Harvard Medical School. Singer was supported by the Eliot B. and Edith C. Shoolman Fund of the Massachusetts General Hospital (Boston, MA). Drs. Lin and Schneeweiss were supported by PCORI grant 282364.5077585.0007 (ARCH/SCILHS).

Disclosures

Oertel has occasionally participated on Advisory Boards for Roche Diagnostics and Alere. Dr. Schneeweiss is consultant to WHISCON, LLC and to Aetion, Inc., a software manufacturer of which he also owns equity. He is principal investigator of investigator‐initiated grants to the Brigham and Women’s Hospital from Bayer, Genentech, and Boehringer Ingelheim unrelated to the topic of this study. The remaining authors have no disclosures to report.

Supplementary Information

Footnotes