Derivation and Validation of a Preoperative Prediction Rule for Delirium After Cardiac Surgery

Delirium, an acute alteration in attention and cognition, was first reported as a complication of cardiac surgery >40 years ago.¹ Since that time, advances in surgical and anesthesia practice have improved the efficiency and outcomes of cardiac surgery. However, delirium remains a frequent but underrecognized complication after cardiac surgery² that is associated with short-term complications such as mortality, morbidity, and increased length of stay.^3–6 Moreover, although generally thought of as a short-term disorder of cognition, delirium can have long-term sequelae, including persistent cognitive deficits, loss of independence, functional decline, increased costs, and increased mortality for up to 2 years.^3,4 Importantly, delirium and its complications can be prevented in the surgical population.^6,7

Delirium prediction rules use cogent clinical items from the history, physical, and initial diagnostic tests to prospectively identify patients at high risk for delirium.⁸ Prediction rules also allow the patient and family to be better informed of risk and targeting of surveillance and prevention efforts for delirium. Despite research identifying independent risk factors for delirium,^2,9 there is no validated preoperative prediction rule for delirium risk after cardiac surgery.

The objective of the present study was to develop and validate a clinical prediction rule based on preoperative factors for the development of delirium after cardiac surgery. We used state-of-the-art methods, including rigorous prospective assessment of preoperative risk factors, comprehensive delirium detection methods, advanced statistical methods for identification of independent variables, and validation in an independent sample. We also examined the association of intraoperative variables with delirium, adjusting for preoperative risk. We believe that our results will aid clinicians caring for cardiac surgery patients and inform research studies to improve cognitive outcomes after cardiac surgery.

Methods

Participants

From September 1, 2002, until October 31, 2004, we recruited 122 patients >60 years of age who were planning to undergo cardiac surgery at 2 academic medical centers and 1 Veterans Administration hospital for our derivation set. Subsequently, a validation set of 109 patients was recruited from November 1, 2004, until June 30, 2006, at 1 academic medical center and 1 Veterans Administration hospital using identical criteria. Eligible cardiac procedures included coronary artery bypass graft (CABG), mitral or aortic valve replacement or repair (valve), and combined CABG-valve. All patients provided written informed consent, and the study was approved by the Institutional Review Board at each institution. Exclusion criteria included living >60 miles from the study center, medical instability limiting preoperative assessment, emergency surgery, delirium before surgery, concurrent aortic or carotid surgical procedures, and non-English speaking.

Preoperative Assessment

Before surgery, all patients underwent extensive assessment with interviews and medical record review of demographics, behavioral factors, comorbidity, mental health, physical function, cognitive function, laboratory profile, and planned surgery. Participants provided information on age, sex, race, marital status, educational level, and alcohol and tobacco exposure. The number of pack-years of tobacco use was calculated. Alcohol use was categorized according to drinks per week: nonusers, users (≤7 drinks per week), and heavy users (>7 drinks per week). Body mass index was calculated from the preoperative anesthesia assessment. Comorbidity information was collected from medical records and supplemented by patient interview. We recorded the presence of hypertension, hyperlipidemia, diabetes, cancer, and a prior stroke, transient ischemic attack (TIA), or hemiparesis. Patients were interviewed using the Geriatric Depression Scale (GDS), an assessment of 15 symptoms of depression (range, 0 to 15; 15=worst).¹⁰ The patient’s ability to function and care for himself or herself was assessed with the activities of daily living, a self-care scale (range, 0 to 18; 0=worst),¹¹ and instrumental activities of daily living, an independent task scale (range, 0 to 21; 0=worst).¹² American Society of Anesthesiologists class, a measure of preoperative illness,¹³ was recorded from the anesthesia record. The Mini Mental State Examination¹⁴ (MMSE) was administered to all patients before surgery (range, 0 to 30; 0=worst). The preoperative laboratory values most proximal to the operation that were recorded included sodium, potassium, glucose, white blood cell count, hematocrit, ratio of blood urea nitrogen to creatinine, and albumin. The type of surgery was included in our preoperative risk model because the type of surgery (CABG with or without valve) is usually known preoperatively. Between catheterization and surgery, patients who remained in the hospital were considered urgent procedures, and those who were discharged were considered elective procedures.

Delirium Assessment

A brief delirium assessment (<15 minutes) was performed preoperatively and daily during the postoperative period beginning on day 2. Because of the intensive postoperative care required after the CABG procedure, patients were not assessed on the day of surgery or postoperative day 1. Delirium was assessed using the validated diagnostic algorithm of the confusion assessment method.¹⁵ Before the confusion assessment method was completed, a standardized mental status interview was conducted, which included the MMSE,¹⁴ digit span, the Delirium Symptom Interview,¹⁶ and the Memorial Delirium Assessment Scale.¹⁷ Digit span, which asks patients to repeat a series of digits forward and backward, is a test of working memory and attention. The Delirium Symptom Interview is a validated interview for eliciting 8 key symptoms of delirium. The Memorial Delirium Assessment Scale is a validated severity scale for delirium (range, 0 to 30; 30=worst). This combined assessment for delirium has been shown to be highly reliable (κ=0.95) when administered by trained, nonclinician interviewers.¹⁸ If the patient was intubated, we assessed delirium using the confusion assessment method for the intensive care unit, a validated assessment for intubated patients that uses the confusion assessment method diagnostic algorithm.¹⁹ The daily assessment was augmented with medical record review for evidence of intervening delirium features.

Operative Procedure

All patients underwent cardiac surgery (CABG, valve, or combined CABG-valve) under general anesthesia. The anesthesia protocol and operative procedure were performed in accordance with local hospital policies and protocols. The use of cardiopulmonary bypass, aortic cross-clamp, high-dose heparin, apoprotinin, and hypothermia was at the discretion of the attending surgeon. Postoperative care, including pain control, was administered in accordance with local hospital policy and practice. Intraoperative variables, including the use of cardiopulmonary bypass, were recorded from the operative, anesthesia, and perfusion records. Intraoperative variables were compared in those with and without delirium and adjusted for the preoperative prediction rule.

Statistics

Comparison of Derivation and Validation Cohorts

Preoperative characteristics of patients in the derivation cohort were compared with those in the validation cohort using a t test for continuous variables and a χ² test for ordinal or dichotomous variables.

Derivation of Prediction Rule

We examined the distribution of all continuous variables. Variables with extremely skewed distributions were categorized according to clinically relevant cut points if available or at naturally occurring inflection points. There were few (n=6) patients with heavy alcohol use, and alcohol use was dichotomized into users and nonusers. Instrumental activities of daily living and activities of daily living compared those with maximum numerical score to those with less than maximal score. The prevalence of individual laboratory abnormalities was low, and we created a categorical variable using the cut points from the Acute Physiology, Age, and Chronic Health Evaluation-III.²⁰ Albumin was categorized as an abnormal (≤3.5 or ≥4.5 g/dL) or normal (3.6 to 4.4 g/dL) on the basis of clinical cut points at our institution and a u-shaped bivariable relationship between albumin and delirium risk. Ratio of blood urea nitrogen to creatinine ≥18 has been identified as an independent risk factor for delirium in medical patients,²¹ and this dichotomization was preserved. Normally distributed, continuous variables (age, body mass index, MMSE) were maintained as continuous in the imputation, bootstrapping, and multivariable models but were categorized for the final predictive model.

Preoperative characteristics of patients who developed delirium were compared with those who did not develop delirium using a t test for continuous variables and a χ² test for ordinal or dichotomous variables. Variables with a bivariable relationship associated with a test statistic with a value of P≤0.10 were selected for inclusion in the multivariable analyses using imputation and bootstrapping.

Missing data among the predictor variables were handled using a multiple imputation procedure with 20 resampling replications, which generated an augmented database with (122×20) 2440 observations with complete data.²² To protect from overfitting the data and to limit chance associations in the derivation sample from influencing the development of the prediction rule, we used a bootstrap resampling procedure to determine the independent factors associated with delirium. We generated 100 bootstrap samples from the derivation cohort (each of 122, drawn randomly with replacement from the augmented sample). With each bootstrap sample, we modeled the prediction of delirium given selected predictors using a backward stepwise logistic regression model, retaining parameters significant at the 0.05 level. Factors associated with delirium were defined as those variables returning regression coefficients significant at a level of α=0.05 in at least 50% of the bootstrap samples. To finalize the prediction rule, the remaining continuous independent risk factors were categorized using clinically meaningful cut points.

Validation Sample and Model Performance

The clinical prediction rule was applied to the validation sample using multiple imputation procedures for missing data. We present the incidence of delirium with increasing clinical prediction rule points and risk ratio relative to the lowest-risk group. Model performance in both cohorts was measured with the C statistic. A sensitivity analysis examined the C statistic excluding patients who did not have cardiopulmonary bypass (“off pump”).

The authors had full access to and take full responsibility for the integrity of the data. All authors have read and agree to the manuscript as written.

Results

Delirium developed in 52% (63 of 122) of the validation cohort and in 44% (48 of 109) of the derivation cohort. The peak incidence of delirium occurred on postoperative day 2 and persisted for a median of 1 day (range, 1 to 27 days). Patients with delirium were more likely to have a longer hospital stay (10.2±6.3 versus 7.5±3.8 days; P<0.001) and to be discharged to a nursing home or rehabilitation hospital (73% versus 27%; P<0.001). Table 1 describes the baseline characteristics of the derivation (n=122) and validation (n=109) cohorts. Typical of the cardiac surgery patients at our medical centers and nationwide, patients were of advanced age (73.7±6.7 years), mostly male (79%), and predominantly white (96%).²³ The derivation and validation cohorts were similar in their preoperative characteristics. However, the derivation cohort was slightly older, endorsed more depressive symptoms, had abnormal albumin more frequently, was more likely to undergo urgent surgery, and was more likely to have a CABG surgery. Five patients in the derivation and 3 patients in the validation cohort underwent off-pump surgery.

The bivariable analysis comparing the risk factors in those with and without delirium in the derivation cohort is displayed in Table 2. Patients who developed delirium were significantly more likely to be older and tended to be female. Alcohol consumption was associated with a reduced delirium risk. Body mass index was significantly lower in patients with delirium. Patients with prior stroke/TIA were significantly more likely to develop delirium. Patients who developed delirium described significantly more depressive symptoms and had lower MMSE scores preoperatively. Suboptimal functional performance was not associated with delirium. In the laboratory profile, abnormal albumin was associated with increased incidence of delirium. Neither the urgency nor type of surgery was associated with delirium.

Table 3 presents the results of the multivariable model. Four variables were associated with delirium in at least 50% of the bootstrap samples at a level of α=0.05 after backward selection: MMSE, history of stroke/TIA, GDS, and abnormal albumin. These variables were selected for the development of the prediction rule. Based on the results of the multivariable modeling, a final prediction rule was developed using categorical versions of retained variables. The GDS was dichotomized at >4, which has been shown to have good sensitivity (83% to 100%) and specificity (65% to 84%) for clinical depression.^24,25 MMSE was categorized into established clinically important ranges for definitive impairment (≤23), mild impairment (24 to 27), and not impaired (28 to 30).^14,26

Clinical prediction rule points were assigned after standardization to the lowest regression coefficient. MMSE ≤23 was assigned 2 points. One point was assigned to MMSE 24 to 27, history of stroke/TIA, GDS >4, and abnormal albumin. Table 4 describes the performance of the clinical prediction rule in the derivation and validation cohorts. In both cohorts, there was increasing risk of postoperative delirium with increasing risk score, and the model predicted well in both the derivation (C statistic=0.74) and validation (C statistic=0.75) cohorts, with no degradation of model performance in the validation cohort. With off-pump patients excluded, the model C statistic was 0.73 in the derivation cohort and 0.78 in the validation cohort. Additionally, delirium severity, as measured by the Memorial Delirium Assessment Scale, increased with increasing risk score. For point levels of 0, 1, 2, and ≥3, the mean Memorial Delirium Assessment Scale score in the derivation cohort was 5, 7, 9, and 11 (P<0.01) and in the validation cohort was 4, 7, 8, and 9 (P<0.001), respectively.

Intraoperative variables are presented in Table 5 for those with delirium compared with those without delirium. After adjustment for the clinical prediction rule, duration of anesthesia remained a significant predictor of delirium (risk ratio, 1.2; 95% CI, 1.01 to 1.4).

Discussion

In the present prospective study of cardiac surgery patients, delirium was extremely common. We identified 4 preoperative factors that were independently associated with postoperative delirium: impaired cognition, depressive symptoms, prior stroke or TIA, and abnormal albumin. With these factors, we developed a clinical prediction rule and validated this rule in a separate cohort. The rule performed well in both the derivation (C statistic=0.74) and validation (C statistic=0.75) cohorts. Applying the risk stratification system showed that compared with no points, the presence of 1 point more than doubles the delirium risk, the presence of 2 points more than triples the delirium risk, and the presence of ≥3 points more than quadruples delirium risk.

Our clinical prediction rule has face validity in that 3 of our risk factors, impaired cognition,²⁷ prior stroke,^28,29 and depressive symptoms,³⁰ have been identified in previous studies, but it adds substantial incremental value in that we have integrated these risk factors into a prediction rule that clinicians can use to stratify overall risk. Importantly, the identification of the additional risk factor of abnormal albumin extends the previous work. Albumin level is associated with operative mortality³¹ and has been hypothesized to be an overall biomarker of frailty and nutritional and functional abilities.^32,33 In a noncardiac surgery delirium prediction rule,³⁴ low albumin was associated with delirium, but the high missing data rate precluded its inclusion in the modeling. Additionally, albumin plays an important role in intravascular volume status and drug binding. Thus, when recorded at the time of admission, low albumin may be a laboratory variable associated with lower functional level, as well as affecting hemodynamic shifts and pharmacokinetics of cognitively active drugs.

The cardiac surgery prediction rule for delirium may provide insights into our understanding of the pathophysiology of delirium. Several factors that we identified are potentially associated with central nervous system atherosclerotic disease (prior stroke/TIA, cognitive impairment, depression).³⁵ The 2 graded categories of cognitive function provide insight into the delirium risk of patients with milder degrees of cognitive impairment who do not meet the traditional dementia threshold (MMSE <24); this concept is consistent with the subdementia threshold frequently associated with vascular cognitive impairment. Furthermore, there is increasing literature on the association of vascular risk and depression.³⁶ Thus, atherosclerosis may be a common risk factor that can predispose patients to delirium.⁹

Our prediction rule conforms to a widely adopted approach for evaluating delirium risk that considers predisposing factors (present before surgery) and precipitating factors (occur during and after surgery).^21,34 Our overall goal was to develop a preoperative clinical prediction rule based on predisposing factors. However, we also analyzed the additive contribution of intraoperative precipitating factors. The finding that duration of anesthesia was associated with delirium after adjustment for the preoperative prediction rule could represent worse underlying disease, more complex surgery, and/or additional exposure to anesthetics. We will consider intraoperative and postoperative precipitating factors for delirium in future work to determine additional delirium prevention strategies. As in previous studies, prevention of delirium in high-risk patients would focus on environmental modifications, early mobilization, psychoactive medication reduction, and prevention of complications.^6,7

The rate of delirium in the present study is at the higher end of published reports. There are several reasons for this. First, the present study used state-of-the-art delirium detection methods, including a standardized assessment that was delivered daily. This standardized delirium battery includes assessments for attention impairment, which may not be identified in a routine clinical interview. The incidence of delirium after cardiac surgery varies widely (2% to 73%).² Studies using a standardized battery^37,38 have found a higher incidence of delirium than studies that assess delirium via chart review or nursing report.³⁹ Second, older age can be a risk factor for postoperative delirium.^28–30,34 This study enrolled patients >60 years of age, resulting in a mean age of 73 years, which is older than other studies of delirium after cardiac surgery and reflects the trend toward operating on older patients.²³ Finally, we included patients regardless of preoperative cognitive function. Thus, we likely have patients with vascular cognitive impairment, mild cognitive impairment, and/or dementia who are much more likely to develop delirium.

Several strengths of this study warrant mention. First, the study derived and validated the prediction rule in independent cohorts at >1 medical center. The study included patients undergoing elective or urgent cardiac surgical procedures. Emergency patients would likely have a higher delirium rate but were not included because of an inability to obtain a preoperative baseline interview. Additionally, we aggressively identified and verified preoperative characteristics that were included in the model to ensure accurate risk factor identification. MMSE and GDS took ≈15 minutes to administer before surgery and were performed by trained research assistants. Additionally, the model performed similarly after the exclusion of off-pump patients. Finally, our analytic approach combined data augmentation via multiple imputation and bootstrap resampling procedures for deriving the independent predictors included in the clinical prediction rule. This advanced statistical methodology may provide more stable variables for prediction-validation rules by minimizing the impact of missing data and limiting model overfitting and is superior to list-wise deletion or regression to the mean.^40,41

Several limitations need to be described. First, our population consisted of patients who were mostly white and well educated (>50% with education beyond high school) and recruited at academic medical centers in a single geographical region. MMSE performance may be improved with increased education, but in this study, education was not associated with delirium. This may limit generalizability to less educated populations, but internal validity should not be challenged. There was variability in the derivation and validation cohorts, specifically in characteristics that were included in the model such as age, GDS, and albumin. However, the differences in these variables in the validation cohort bias toward the null with respect to delirium risk (lower age, less depressive symptoms, higher incidence of normal albumin, etc), yet the overall model performance showed no degradation between the derivation and validation cohorts. We were unable to measure all preoperative characteristics such as carotid stenosis that may predispose to delirium.⁹ Finally, the lowest risk in our prediction rule is 18% to 19%, which limits our ability to identify patients who might be excluded from interventions. The prediction rule performed better at predicting higher levels of delirium risk.

This study identified 4 cogent risk factors for delirium: MMSE, prior stroke or TIA, depression, and abnormal albumin. Delirium risk more than quadruples from the lowest to highest risk levels. Clinically, patients who are stratified into moderate and high risk for delirium would benefit from frequent delirium screening and implementation of delirium prevention strategies. Importantly, this cardiac surgery delirium prediction rule provides a method to preoperatively stratify at-risk older patients for such interventions to ultimately reduce the morbidity, mortality, and cost of postoperative delirium.

Footnotes