Clinical Prediction Model for Time in Therapeutic Range While on Warfarin in Newly Diagnosed Atrial Fibrillation

Atrial fibrillation (AF) is a growing public health and clinical problem, as an aging population, prolonged survival of patients with cardiac conditions predisposing to AF, enhanced detection of more sporadic forms of AF, and other factors have collectively served to expand the number of AF diagnoses.¹ AF portends a 5‐fold increased risk of thromboembolic events; thus, any new AF diagnosis requires assessment of thromboembolic risk and in many cases, proper oral anticoagulation (OAC) to mitigate excess risk.^{2, 3, 4, 5, 6} Though warfarin and other vitamin K antagonists (VKA) have been the primary OACs for several decades, the recent introduction of 4 non–vitamin K antagonists, direct OACs (DOACs), has expanded the therapeutic options for thromboprophylaxis in AF. Meta‐analyses have reported small improvements in efficacy and safety with DOACs compared with warfarin; however, these modest improvements must be considered with the higher cost of DOACs, which have created uncertainty about their overall cost‐effectiveness.^{7, 8, 9, 10} Other concerns limiting more widespread use of DOACs include an inability to monitor their anticoagulation effects, short half‐lives that increase thrombosis risk when doses are missed, and lack of experience with reversal agents that may be needed in urgent settings.¹¹ Alternatively, warfarin is an inexpensive, long‐used, well‐researched, and effective therapeutic, associated with a 64% reduction in ischemic stroke in nonvalvular AF compared with placebo.^{12, 13} However, warfarin possesses multiple untoward attributes complicating its use, including a narrow therapeutic range, numerous drug and dietary interactions, and a metabolism strongly dependent on multiple genetic polymorphisms that make warfarin's ultimate pharmacologic effects difficult to predict.^{9, 12, 13, 14}

The safety and efficacy of warfarin is highly dependent on the quality of anticoagulation control, often measured by the percent of time in therapeutic range (TTR), with TTR demonstrating strong inverse associations with both ischemic stroke and bleeding events.^{15, 16, 17, 18, 19, 20} In light of available evidence, warfarin might be the preferred initial choice of OAC provided a high TTR can be expected, while DOACs might alternatively be preferred with a low expected TTR. Though multiple studies have evaluated predictors of TTR, only a single study has combined the effects of multiple TTR predictors into a prediction tool designed to assist decision making between warfarin and DOACs (the SAMe‐TT₂R₂ [sex, age, medical history, treatment, tobacco, race] score).^{9, 21} The SAMe‐TT₂R₂ score was derived from a relatively small randomized clinical trial population, considered a limited number of candidate predictors, and demonstrated less than ideal predictive performance at external validation.^{16, 21, 22, 23} Accordingly, the current study sought to create an improved clinical prediction model for estimating TTR among newly diagnosed AF patients for whom warfarin was initiated for anticoagulation control. The proposed prediction model was compared with the SAMe‐TT₂R₂ score with regard to predictive performance while simultaneously providing an additional external validation of the SAMe‐TT₂R₂ score. Finally, numerical results are provided in a format allowing clinical application of the model.

Methods

This retrospective study incorporates the patient population and electronic medical record (EMR) data warehouse of the Geisinger Health System (Geisinger). The study was approved by the Geisinger Institutional Review Board who granted a waiver of patient consent. The study was restricted to patients receiving primary care and other healthcare services through Geisinger for at least a 2‐year period in order to better rule out pre‐existing AF before entering the Geisinger EMR system, identify warfarin use before a new AF diagnosis, and provide a detailed account of baseline characteristics at the time of newly diagnosed AF (see below).

Study Population

The intent of the study inclusion criteria was to identify a group of warfarin‐naïve patients with newly diagnosed AF subsequently prescribed warfarin but who would have been equally eligible for a DOAC under contemporary guidelines.³ Newly diagnosed AF occurring between 2003 and 2014 was defined by observing the appropriate International Classification of Diseases—9th Revision codes (427.3, 427.31, 427.32) at either 1 inpatient or 2 separate outpatient encounters following a minimum 2‐year period where no EMR documentation of these codes was found.^{24, 25, 26, 27, 28} Study patients also had no EMR documentation of warfarin use in the 2+ years before the AF diagnosis and were subsequently prescribed warfarin within 90 days of diagnosis. In accord with recent guidelines, patients with a documented history of rheumatic mitral stenosis, valve replacement, or mitral valve repair were excluded because of valvular AF.³ Study patients were required to have at least 4 international normalized ratio (INR) measurements within the first year following warfarin initiation in order to provide a reasonably valid estimate of TTR. The 1‐year postwarfarin initiation time period for TTR determination was chosen as a reasonable interval for judging anticoagulation quality with newly initiated warfarin. The TTR percentage was calculated as the percentage of days where the estimated INR was between 2.0 and 3.0 inclusive using linear interpolation to estimate unmeasured INRs between consecutive measurements.²⁹ INRs were interpolated only when time intervals between successive measurements were 60 days or less, consistent with prior studies.^{21, 30, 31, 32, 33, 34, 35, 36}

Candidate TTR Predictors

A set of 85 candidate predictors of TTR was identified through data elements gathered during usual clinical care and stored within patient EMRs. The general strategy for candidate predictor selection was to assemble a large and diverse set of potential predictors from various domains that might assist in predicting TTR on warfarin. These variable domains included demographics, vital signs, medical history including diagnoses and procedures, medications, and laboratory tests (Table 1). All study variables were evaluated with reference to a baseline date, the date of the first warfarin prescription following the AF diagnosis. All historical diagnoses and procedures clinically documented at any encounter at the baseline date or earlier were considered present at baseline. Diagnoses and procedures are defined by International Classification of Diseases—9th Revision and/or Current Procedural Terminology codes. Vital signs and laboratory values assigned to a patient at baseline were determined in a hierarchical manner with (1) an outpatient value measured on the baseline date given highest priority; followed by (2) the outpatient value measured prior, but closest to, the baseline date; and then (3) the outpatient value measured following, but closest to, the baseline date up to 90 days following baseline. Inpatient values were considered in the same temporal fashion when no outpatient value was available. All laboratory tests available on >75% of study patients were considered as candidate predictors. Missing data for vital signs and laboratory tests are not missing at random (missing data imply better health); thus, usual imputation strategies are of questionable validity. Accordingly, a conservative imputation approach was taken whereby missing values were imputed via random selection from the empirical distribution. The random selection was repeated multiple times to evaluate the sensitivity of effect estimates to this approach. Medications at baseline were those ordered or affirmed on medication reconciliation lists up to 1 year before or 90 days following baseline with the exception of aspirin and other platelet aggregation inhibitors such as clopidogrel. As use of these medications must be re‐evaluated and possibly discontinued when starting warfarin, only new orders documented after the warfarin initiation date defined users of these medication subclasses at baseline.

Analytic Strategy

The primary analysis focuses on TTR within the first year of warfarin initiation as a continuous variable. Though multiple thresholds for poor and optimal TTR have been suggested, there remains no consensus and accordingly various TTR thresholds were evaluated as secondary end points.^{9, 12, 19, 23, 31} All continuous variables were categorized into 6 groups split at the 10th, 25th, 50th, 75th, and 90th percentiles of the empirical distributions in order to accommodate possible nonlinear and nonmonotonic associations between continuous covariates and TTR, identify possible threshold effects, and circumvent the potential adverse impact of extreme outliers on the magnitude of regression coefficients. The 6 ordinal groups are henceforth referred to as very low (<10th percentile), low (10–25), low‐normal (25–50), high‐normal (50–75), high (75–90), and very high (>90).

The patient cohort was temporally split in an ≈2:1 ratio at May 1, 2011, into development and validation sets; an initial prediction model was built using the development set and tested on the validation set.^{37, 38} Linear regression modeling was used and a forward stepwise variable selection algorithm applied in order to identify the strongest independent predictors of continuous TTR.^{38, 39} All models generated during the variable selection process were reviewed closely for collinearity and adjustments made as needed. As all variables were placed on categorical scales (binary or 6‐group variables as described above), the criterion for variable inclusion/exclusion in the development set was an absolute 3% difference in TTR across a variable's levels as opposed to the usual P‐value threshold criterion. For the 6‐group variables, if adjusted regression model coefficients differed by ≥3% across any pair of levels, the variable was kept in the development set model. Once the final development set model was confirmed, a model with the same parameterization was applied to the validation set. Development model variables were considered validated if the 3% difference criterion was also observed in the independent validation set. In essence, the development set is used to hypothesize potential data patterns and any observed pattern is considered validated if also observed in a second, independent data set. This modeling tactic serves as a safeguard against model overfitting by requiring effects observed in the development set to persist in an independent group of patients. The final reported model then considers the entire patient cohort and is restricted to the validated variables. For all models, the R² statistic is reported, quantifying the amount of variation in TTR explained by the collective set of included variables.

Secondary Analyses

Several secondary analyses were performed. The large sample size permitted fitting a full model containing all 85 candidate covariates in a 220‐degree of freedom model, which allows quantifying via R², the maximum predictive capacity of the entire collection of candidate predictors. The full model is less informative for evaluating effects of individual predictors because of extensive collinearity; thus, regression coefficients are not reported. Next, several TTR cut points indicative of possible poor or optimal TTR were identified and logistic regression models fit to evaluate the impact of the final model predictors with respect to these cut points. C‐statistics are reported for all logistic regression models. All predictive performance metrics were repeated after including the SAMe‐TT₂R₂ score as the sole covariate in the various models. All statistical analyses were conducted with the SAS statistical software, version 9.4.

This study was an investigator‐initiated study funded by Daiichi Sankyo, who made minor suggestions to the original study proposal but otherwise had no role in study design, execution, or article preparation.

Results

Out of 20 183 new AF diagnoses between 2003 and 2014, 10 652 (53%) were taking warfarin within 90 days of diagnosis, while 1267 (6%) were taking a DOAC. Among warfarin users, 7877 (74%) met the definition for nonvalvular AF, had no documentation of warfarin use before diagnosis, and had at least 4 INR measurements within 1 year following warfarin initiation. The median (interquartile range) TTR within the first year following warfarin initiation was 51% (32, 67), and beyond the first year was 61% (44, 74). The distribution of first‐year TTRs is shown in Figure. The linear regression model containing all 85 candidate predictors (the full model) had R²=19.5%.

• Download PowerPoint

Development Set

The development set contained 5173 patients with newly diagnosed AF before May 1, 2011, and had a median first‐year TTR of 52% (33, 67). After applying the forward stepwise variable selection algorithm, 32 of the original 85 candidate predictors met development set model inclusion criteria (Table 2). The 10 strongest predictors of TTR in descending rank order according to F statistic magnitude were as follows: (1) any antiarrhythmic; (2) aspirin; (3) valve disease; (4) lung disease; (5) myocardial infarction; (6) anemia; (7) gastrointestinal bleeding; (8) albumin; (9) venous thromboembolism; and (10) stroke—hemorrhagic. The R² for this model was 18.3%.

Validation Set

The validation set contained 2704 patients with newly diagnosed AF on or after May 1, 2011, and had a median first‐year TTR of 51% (32, 67). When applying the development set model to the validation set, the R² statistic was 19.4%, but 17 of the development set variables did not meet the specified validation criteria (Table 2). The 17 predictors failing to validate were removed, and the remaining 15 predictors were fit in a final model using the entire cohort with development and validation sets recombined. The predictive strength of the TTR predictors in the final model in descending rank order according to F statistic magnitude were as follows: (1) any antiarrhythmic; (2) aspirin; (3) anemia; (4) lung disease; (5) albumin; (6) red blood cell count; (7) venous thromboembolism; (8) thrombocytopenia; (9) alcohol problem; (10) red blood cell distribution width; (11) systolic blood pressure; (12) stroke—hemorrhagic; (13) age; (14) body mass index; and (15) neutrophil percentage (Table 2). All variables included in the final model had P values <0.05, with the highest P value being 0.003 (P values for categorized continuous variables were for differences across all levels). The R² statistic for the final model was 15.4%. A user‐friendly format for applying the final model is provided in Table 3. In Table 3, the final model is parameterized so all regression coefficients are negative (≤ −3.0%). Thus, estimated per‐individual TTRs are calculated by subtracting coefficients for the applicable model elements from the overall model intercept (69.5%). As reported, estimated TTR will be <60% if 4 or more poor TTR factors are present, and <50% if 7 or more poor TTR factors are present. When different random imputations for missing data were applied, regression coefficients typically differed by <0.3.

Comparison to SAMe‐TT₂R₂ Score

The various models reported above showed modest discrimination for various TTR thresholds (Table 4). C‐statistics were near or above 0.70, and were generally higher as TTR percent thresholds were lowered. When including the SAMe‐TT₂R₂ score as the sole variable in the various regression models, R² statistics ranged from 2.2% to 3.5% (versus 15.4–19.5% in the newly developed models), and c‐statistics were consistently below 0.60 (Table 4).

Discussion

The current study developed and validated a clinical prediction model for estimating TTR within the first year following warfarin initiation among previously warfarin‐naïve patients with a new diagnosis of AF. The study was able to consider a more extensive array of candidate predictors than previous studies, including several historical diagnoses and laboratory tests collected during usual clinical care, many of which were found to be associated with TTR. The proposed prediction model demonstrated stronger predictive performance in a validation cohort than a competing model designed with the same intent, and may prove valuable in differentiating those likely to achieve an adequate TTR on warfarin from those who may be more properly anticoagulated with a DOAC. Despite the improved predictive performance, the proposed prediction model only explained a modest amount of variation in TTR.

The efficacy and safety of warfarin is highly dependent on the quality of anticoagulation achieved as measured by the TTR, so among newly diagnosed, warfarin‐naïve AF patients, it may be clinically valuable to identify those individuals likely to have poor TTRs on warfarin so alternative therapies such as DOACs can be applied. Multiple studies have examined predictors of poor TTR, and our study shows many consistencies with, yet extends, previous findings.^{9, 21, 32, 35, 40, 41, 42, 43, 44} In particular, our study affirms younger age, lower body mass, lung disease, application of a rhythm control treatment strategy, kidney dysfunction, and alcohol problems as predictors of poor TTR.^{9, 21, 32, 35, 40, 41, 42, 43, 44} Our study advances prior work by considering an extended list of historical diagnoses and several inexpensively attained, commonly measured laboratory tests that serve as markers of potentially salient physiologic features such as kidney function, liver function, inflammation, nutritional status, metabolic derangements, coagulation propensity, red blood cell production and function, volume status, and frailty. Notably, 4 of the final 15 validated predictors were laboratory tests—albumin, neutrophil percentage, red blood cell count, and red blood cell distribution width and, as more detailed descriptors of the ailments they characterize (eg, kidney function), laboratory tests often outpredicted their dichotomous surrogates (eg, history of kidney disease). The limited predictive power provided by the collection of model elements reinforces that anticoagulation control with warfarin is inherently difficult to predict, as even our full model containing all 85 candidate predictors explained just 19.5% of the total variation in first‐year TTR, while our final model explained 15.4% of the variation. Likewise, another TTR prediction model (the SAMe‐TT₂R₂ score) explained just 10% of TTR variation in its development set while explaining about 3% of TTR variation in the external validation described here.²¹ Notably, several predictors found associated with poor TTR in the SAMe‐TT₂R₂ score and other prior studies were not confirmed in our study, including female sex, nonwhite race, and recent smoking status.^{21, 32, 35, 43, 44, 45}

The proposed TTR prediction model allows estimating per‐individual TTR percentage within the first year following newly initiated warfarin among patients with a new AF diagnosis as a means to assist decision making regarding proper OAC selection (warfarin versus DOAC). The study inclusion criteria were designed to identify a set of AF patients for whom the warfarin‐or‐DOAC question was most applicable (no prior warfarin use, nonvalvular AF). Unfortunately, there is no universally accepted definition of nonvalvular AF that could be applied, and indeed, the 4 major DOAC trials applied slightly different definitions in their respective trials.^{3, 46, 47} A previously developed prediction model, the SAMe‐TT₂R₂ score, was developed with the same intent, though it did not focus specifically on new AF diagnoses.^{21, 48} The external validation assessment of the SAMe‐TT₂R₂ score performed here did not suggest a strong predictive performance in our patient cohort, with the score explaining about 3% of TTR variation, and c‐statistics for discrimination uniformly below 0.60 for multiple TTR cut points. The poor validation may be related to the different patient populations (new AF versus not) and/or study settings (observational study versus clinical trial).

Quantitative application of the prediction model proposed here produces an estimated TTR (from 0% to 100%) for any individual patient. Incorporation of the resulting numerical value into clinical decision making requires careful clinical judgment, as no uniform thresholds for poor or optimal TTR have reached consensus. Stated optimal TTRs have typically been in the >60% to 70% range, whereas poor TTRs are more difficult to pinpoint but may serve as the more salient decision point in the clinical environment.^{5, 12, 49} TTRs as low as 40% to 50% have been suggested as possible “poor” thresholds, as some studies have shown at least some benefit with warfarin compared with no OAC above this threshold, but not below, with improving rates of efficacy and safety as TTR rises.^{9, 19, 31} Prediction model estimates below these values likely warrant more serious consideration of a DOAC. Of note, our study population consists of warfarin‐naïve patients at the time of AF diagnosis, and TTRs are known to increase over time among new warfarin users as experience grows and dosing is optimized.^{32, 35, 43, 50} Indeed, the median TTR during the first year following warfarin initiation (51%) was noticeably lower than the median TTR restricted to INR measurements taken after the first year (61%). The 1‐year threshold was chosen as a reasonable time interval to judge the quality of anticoagulation control with newly started warfarin. Notably, our overall median TTR (55%) observed in a real‐world clinical environment is very consistent with previous observational studies but noticeably lower than average TTRs observed in the major DOAC trials.^{9, 30, 31, 35} Nonetheless, in situations where expected TTRs are systematically higher or lower than observed in the current study, a model recalibration (via an intercept adjustment) may be needed in order to provide the most valid estimate of TTR.

Some limitations of the current study should be noted. The current study, though designed to capture a real‐world cohort of newly diagnosed AF patients, was performed within a single healthcare system serving a predominantly white population, so the proposed prediction model should be externally validated at other institutions. Though the proposed decision tool was designed to be applied in a situation of clinical equipoise between warfarin and DOACs, we acknowledge that certain patient characteristics may favor one form of OAC over another despite lack of acknowledgment in published guidelines. Furthermore, any predictor of poor TTR related to warfarin compliance (eg, history of an alcohol problem) could also be related to DOAC compliance. Thus, it must be stressed that prediction model output should only be 1 part of the decision making process in OAC selection. EMR‐derived variables are prone to misclassification and measurement error, but our study likely reflects the extent of data quality and completeness prevalent in real clinical practice where the proposed model is most applicable. Missing vital sign and laboratory data are not missing at random—missing implies healthier—thus, usual imputation techniques were not applied.^{37, 38} The imputation method was intentionally chosen to be a conservative approach, more likely to attenuate effect estimates. Other data analysis tactics could have been used; however, we emphasize that the final prediction model only endorses data patterns independently observed in both the development and validation sets, which lend credibility to their veracity. Finally, our study could not consider those patients who were OAC‐eligible but not prescribed warfarin, or those warfarin users with fewer than 4 INR measurements for whom an appropriate TTR could not be calculated.

In conclusion, our proposed prediction model consists of several validated predictors of first‐year TTR on warfarin that can be measured easily and inexpensively in the clinical setting, enabling individual‐level estimation of the expected TTR among patients with newly diagnosed AF. Together with other relevant clinical factors, application of the proposed model may assist decision making regarding the proper mode of OAC in this growing patient population. However, TTR prediction remains challenging, and future studies should attempt to find additional predictors that explain a greater proportion of TTR variance.

Sources of Funding

Funding for this study was provided by Daiichi Sankyo.

Disclosures

Williams receives research funding from Merck, Roche, Biosense Webster, Gilead, Boehringer Ingelheim, Edwards Lifesciences, and Janssen. Evans, Honushefsky, and Berger have no disclosures to report.

Footnotes