Skip to main content
Research Article
Originally Published 15 November 2017
Open Access

Derivation and Validation of a Geriatric‐Sensitive Perioperative Cardiac Risk Index

Journal of the American Heart Association

Abstract

Background

Surgical patients aged 65 and over face a higher risk of cardiac complications from noncardiac surgery. The Revised Cardiac Risk Index (RCRI) and the Gupta Myocardial Infarction or Cardiac Arrest (MICA) calculator are widely used to predict this risk, but they are not specifically designed to predict MICA in geriatric patients. Our hypothesis is that a new geriatric‐sensitive index, derived from geriatric data, will capture this population's unique response to risk factors.

Methods and Results

The model was developed using the NSQIP (National Surgical Quality Improvement Program) 2013 geriatric cohort (N=584,931) (210,914 age ≥65) and validated on the NSQIP 2012 geriatric cohort (N= 485,426) (172,905 age ≥65). Least Angle Shrinkage and Selection Operator regression was used for initial variable selection. The Geriatric‐Sensitive Cardiac Risk Index (GSCRI) was then evaluated in the 2012 data set. The area under the curve (AUC) was compared among the GSCRI, RCRI, and Gupta MICA in the 2012 data set. The GSCRI had an AUC of 0.76 in the validation cohort among geriatric patients. When the Gupta MICA was tested on geriatric patients in the validation cohort, a significant deterioration (≈17%) was noted, as well as a significant underestimation of the risk. The GSCRI AUC of 0.76 in the geriatric subset was significantly greater (P<0.001) than those in the RCRI (AUC=0.63) or Gupta MICA (AUC=0.70) models, outperforming the RCRI and Gupta MICA models in geriatric patients by 13% and 6%, respectively, with a ΔAUC and P‐value of 0.13 (P<0.001), and 0.06 (P<0.001).

Conclusions

The GSCRI is a significantly better predictor of cardiac risk in geriatric patients undergoing noncardiac surgery.

Clinical Perspective

What Is New?

This study provides a new perioperative risk‐prediction tool designed specifically for geriatric patients.
This study demonstrates that currently used risk models of the Lee Revised Cardiac Risk Index and Gupta Myocardial Infarction and Cardiac Arrest have moderate performance in older patients and tend to underestimate the actual cardiac risk in this age group.
This study demonstrates the importance of developing and integrating geriatric‐specific risk models.

What Are the Clinical Implications?

The Geriatric‐Sensitive Perioperative Cardiac Risk Index may be used for perioperative cardiac risk stratification and evaluations for patients 65 years of age or older.
With a significant change in the demographics and an increasing older patient population undergoing surgery, this Geriatric‐Sensitive Perioperative Cardiac Risk Index could be an effective tool for cardiac risk evaluation.
There are 40 million people aged 65 and over living in the United States today.1 Although they account for just 15% of the US population, they receive one third of all inpatient surgeries.1, 2 By 2030, 72 million Americans will be 65 and over, accounting for 20% of the US population and an increasing number of surgeries.1, 3, 4 All inpatient surgery carries a risk of cardiac complications for all adult patients, regardless of age.5, 6, 7 Cardiac arrest after noncardiac surgery is associated with a hospital mortality rate of 65%.8, 9 Myocardial infarction (MI) after noncardiac surgery is associated with a hospital mortality rate of 15% to 25%.10, 11, 12 Nonfatal MI is associated with increased mortality during the first 6 months after surgery.8, 11 Older adults are more prone to MI and cardiac arrest during or after surgery.11 Researchers have developed clinical tools for estimating cardiac risk. The Revised Cardiac Risk Index (RCRI) and Gupta MI and Cardiac Arrest (MICA) calculator are widely used indices to estimate perioperative risk; however, neither tool is specifically designed to assess the risk in geriatric patients. The objectives of this study are to develop and to validate a geriatric‐sensitive cardiac risk index. Our hypothesis is that a new geriatric‐sensitive index, derived specifically from geriatric data, will capture this population's different response to risk factors.13, 14 With a growing geriatric population and a projected increase in noncardiac, usually elective, surgeries4, 15 and the association with substantial cardiac morbidity and mortality,16, 17 it becomes imperative to have accurate estimations of the cardiac risk for geriatric patients.
The aims of this study were (1) to investigate the performance of the RCRI and Gupta MICA perioperative cardiac risk models in a geriatric population, (2) to evaluate the incidence of MICA after noncardiac nonemergency surgery across the age spectrum, (3) to develop a geriatric‐sensitive perioperative cardiac risk index (GSCRI) optimized for use with geriatric patients and sensitive to the clinical and physiologic uniqueness of this population (given that prior hypothesis‐driven objectives were met), and (4) to conduct comparative performance analysis of the GSCRI, RCRI, and Gupta MICA models.

Methods

Participants

We utilized the NSQIP (National Surgical Quality Improvement Program) cohort, a multicenter database of surgical outcomes collected prospectively by trained professionals in a systematic fashion.18 At each center, a certified Surgical Clinical Reviewer collects the data using a variety of methods, including medical chart abstraction. NSQIP developed various mechanisms to ensure data quality, including establishing high interrater reliability and auditing of selected participating sites. In addition, reviewers undergo rigorous training and annual certification to ensure they have the knowledge and resources available to collect high‐quality data.18 NSQIP collected data for over 300 variables, including risk factors for postsurgical cardiac events and 30‐day postsurgery morbidity and mortality outcomes.
In this study NSQIP years 2012 (N=543 885) and 2013 (N=651 940) were used. Participants who had emergency surgery (2012=54 729; 2013=63 980) or cardiac surgery (2012=3730; 2013=3029) were excluded, leaving a sample size of 485 426 (172 905 age ≥65) in the 2012 data and 584 931 (210 914 age ≥65) in the data from 2013. The UCLA Institutional Review Board determined that the analysis of the deidentified data set was exempt from review.

Outcome

The end point of interest is intraoperative/postoperative MICA within 30 days of surgery. Cardiac arrest is defined in the NSQIP
The absence of cardiac rhythm or presence of chaotic cardiac rhythm, intraoperatively or within 30 days following surgery, that results in a cardiac arrest requiring the initiation of CPR, which includes chest compressions. Patients are included who are in a pulseless ventricular tachycardia or fibrillation in which defibrillation is performed and pulseless electrical activity arrests requiring chest compressions. Patients with automatic implantable cardioverter defibrillators that fire although the patient has no loss of consciousness should be excluded.18

Myocardial infarction is defined in the NSQIP as.

An acute MI that occurred intraoperatively or within 30 days following surgery as manifested by one of the following: (1) documentation of electrocardiogram changes indicative of acute MI (one or more of the following: ST elevation >1 mm in 2 or more contiguous leads, new left bundle branch block, new q‐wave in 2 or more contiguous leads); (2) new elevation in troponin greater than 3 times the upper level of the reference range in the setting of suspected myocardial ischemia; (3) physician diagnosis of MI.18

The Revised Cardiac Risk Index.

The RCRI is a previously published index of postsurgical cardiac risk that uses 6 risk factors of major cardiac complications. These risk factors are high‐risk surgery, history of ischemic heart disease, history of heart failure, history of cerebrovascular disease, diabetes mellitus requiring insulin treatment, and serum creatinine >2.0 mg/dL. The risk factors are binary (present/absent). High‐risk surgery is defined as vascular surgery and open intraperitoneal or intrathoracic procedures. Participants with no risk factors were assigned a predicted probability of 0.4%; those with 1 risk factor are assigned 1.0%, 2 risk factors 2.4%, and 3 or more risk factors 5.4%.8 There were a total of 485 426 (172 905 age ≥65) participants in the NSQIP 2012 who were able to have their RCRI scores computed.

The Gupta MICA Model.

The Gupta MICA is a risk score for perioperative cardiac risk developed in the NSQIP 2007 and validated in the NSQIP 2008 data sets.19 The risk score is comprised of 5 items: the participant's functional status, American Society of Anesthesiologists (ASA) classification, creatinine levels, age, and type of surgery that will be performed. There were a total of 479 453 (170 737 age ≥65) participants in the NSQIP 2012 that had their Gupta MICA scores computed.

Statistical Analysis

Analyses were conducted in Stata version 13.1 (College Station, TX) and R version 3.1.0 (R Foundation, Vienna, Austria). Details of the specific analyses and statistical methods are presented in the descriptions of the model building process below. The 2 NSQIP data sets used in this study were assessed separately with the NSQIP 2013 used as a derivation data set and the NSQIP 2012 used as a validation data set.

Candidate Variables for a GSCRI.

Candidate variables for use in the GSCRI were chosen from the NSQIP based on the previous literature and other risk indices currently in use. The variables examined were sex, high‐risk surgery, history of congestive heart failure, history of stroke, currently taking insulin, diabetes mellitus status, dialysis, being on medication to control hypertension, smoking status, history of chronic obstructive pulmonary disease, ASA classification, functional status, creatinine level, type of surgery to be performed, dyspnea, high blood urea nitrogen levels, and high‐risk surgery including vascular surgery and open intraperitoneal or intrathoracic procedures. Bivariate logistic regression models showing the odds ratio and 95% confidence intervals for each of the risk factors when included as the sole predictor of MICA are displayed in Table 1. A multivariable model of these risk factors is additionally provided in Table S1.
Table 1. Odds Ratios for Predictors of MICA in Geriatric Patients for Noncardiac Surgeries (NSQIP 2012)
VariableGroupAge ≥65 y
OR (95% CI)P Value
AgePer 1 y1.05 (1.04, 1.06)<0.001
Sex (Ref=Female)Male1.55 (1.41, 1.71)<0.001
High‐risk surgery (Ref=No)Yes1.72 (1.56, 1.90)<0.001
Hx CHF (Ref=No)Yes5.49 (4.45, 6.78)<0.001
Stroke (Ref=No)Yes2.91 (2.35, 3.62)<0.001
Taking insulin (Ref=No)Yes2.85 (2.52, 3.23)<0.001
Diabetes mellitus (Ref=No)Yes, not insulin dependent1.43 (1.26, 1.63)0.001
Yes, insulin dependent3.04 (2.68, 3.45)0.001
Dialysis (Ref=No)Yes5.16 (4.29, 6.21)<0.001
Medications for hypertension (Ref=No)Yes2.34 (2.05, 2.66)<0.001
Smoking status (Ref=Former/Never)Current1.48 (1.29, 1.69)<0.001
Hx COPD (Ref=No)Yes2.24 (1.96, 2.55)<0.001
ASA class (Ref=I)II4.20 (1.04, 16.93)0.044
III14.93 (3.73, 59.8)<0.001
IV48.1 (11.9, 192.8)<0.001
V81.13 (14.6, 450)<0.001
Functional status (Ref=Independent)Partially dependent3.02 (2.59, 3.51)<0.001
Totally dependent3.90 (2.88, 5.27)<0.001
Creatinine category (Ref <1.5)1.5 to 2.52.67 (2.33, 3.06)<0.001
>2.54.78 (4.05, 5.64)<0.001
Missing0.34 (0.25, 0.45)<0.001
Surgical category (Ref=Hernia)Anorectal3.90 (2.41, 6.32)<0.001
Aortic7.10 (5.05, 9.97)<0.001
Bariatric2.02 (0.96, 4.28)0.065
Brain3.89 (2.26, 6.68)<0.001
Breast0.30 (0.15, 0.59)0.001
ENT1.64 (0.74, 3.62)0.223
Foregut/hepatopancreatobiliary4.19 (3.07, 5.74)<0.001
GBAAS/intestinal4.77 (3.52, 6.45)<0.001
Neck0.50 (0.23, 1.11)0.087
Obstetric/gynecologic1.13 (0.69, 1.85)0.630
Orthopedic2.99 (2.22, 4.02)<0.001
Other abdomen2.94 (1.79, 4.82)<0.001
Peripheral vascular4.93 (3.61, 6.73)<0.001
Skin3.48 (2.20, 5.51)<0.001
Spine2.25 (1.55, 3.27)<0.001
Thoracic4.25 (2.85, 6.34)<0.001
Vein10.9 (7.96, 14.99)<0.001
Urology2.39 (1.71, 3.34)<0.001
Dyspnea (Ref=No)Yes2.10 (1.86, 2.38)<0.001
BUN (Ref <30)>302.98 (2.65, 3.35)<0.001
Missing0.41 (0.33, 0.50)<0.001
Laparoscopic surgery (Ref=No)Yes0.69 (0.56, 0.84)<0.001
ASA Class indicates American Society of Anesthesiology Physical Status Classification System Class; BUN, blood urea nitrogen levels; CI, confidence interval; ENT, ear, nose, or throat; GBAAS; Gallbladder, appendix, adrenals or spleen; Hx CHF, history of congestive heart failure; Hx COPD, history of chronic obstructive pulmonary disease; MICA, myocardial infarction or cardiac arrest; NSQIP, National Surgical Quality Improvement Program; OR, odds ratio.

Variable Selection.

In order to develop an index that is more specific to geriatric patients, the aforementioned candidate predictors were used in a Least Angle Shrinkage and Selection Operator regression analysis in the NSQIP 2013 derivation data set on the geriatric subset20 implemented in the R package glmnet. Tenfold cross‐validation was used to select the appropriate shrinkage parameter, which was determined to be 0.00001801. Because of the size of the NSQIP data set and known theoretical underpinnings of the candidate variable list, no variable had coefficients shrunk to 0 completely. As a result, in order to develop a parsimonious model, predictors with shrunken coefficients greater than 0.7 were selected for use in the final model in addition to variables with known clinical importance. This value of 0.7 was chosen based on the scree plot of the rank‐ordered shrunken coefficients as the inflection point, after which the difference in coefficient magnitudes between successive variables was negligible. The developed GSCRI will thus represent a hybridization of data‐driven approaches and clinical insight. Table 2 summarizes the variables used in each risk model. Description of variables from NSQIP 2012 can be found in Table S2.
Table 2. Summary of the Variables Used in Each Risk Model
Risk FactorsExisting IndicesStudy Index
RCRIGuptaGSCRI
High‐risk surgery  
Type of surgery 
Heart failure 
Ischemic heart disease  
Cerebrovascular disease  
Stroke history  
Diabetes mellitus requiring insulin  
Diabetes mellitus status  
Creatinine level ≥2.0 mg/dL  
Creatinine level ≥1.5 mg/dL 
ASA classification 
Functional status 
Age  
ASA classification indicates American Society of Anesthesiology Physical Status Classification System Class; GSCRI, Geriatric‐Sensitive Cardiac Risk Index; Gupta, Gupta Myocardial Infarction and Cardiac Arrest; RCRI, Lee Revised Cardiac Risk Index,

Model Building.

The variables identified for inclusion from the Least Angle Shrinkage and Selection Operator model were history of stroke, ASA classification, and type of surgery. Additional clinically relevant variables such as functional status, creatinine level (>1.5), diabetes mellitus status, and a history of congestive heart failure were also selected for the final model. These additional variables have been demonstrated to be important factors in risk prediction for MICA.5, 19 These variables, having been selected to be sensitive to geriatric patients, were then used in a logistic regression model in the NSQIP 2013 derivation data to predict MI and cardiac arrest in the geriatric subset. Table 3 shows the final GSCRI model.
Table 3. Coefficients From Variables Selected by Least Angle Shrinkage and Selection Operator and Hypothesis in Noncardiac Surgery (NSQIP 2013)
VariableGroupNoncardiac Surgery
ln (OR) (95% CI)P Value
Stroke (Ref=No)Yes2.08 (1.81, 2.35)<0.001
ASA class (Ref=I)II0.28 (−0.61, 1.17)0.539
III1.34 (0.46, 2.23)0.002
IV2.04 (1.15, 2.93)<0.001
V3.63 (2.52, 4.74)<0.001
Surgical category (Ref=Hernia)Anorectal1.02 (0.59, 1.46)<0.001
Aortic1.32 (1.02, 1.63)<0.001
Bariatric0.31 (−0.34, 0.95)0.348
Brain0.24 (−0.35, 0.83)0.426
Breast−1.14 (−1.74, −0.55)<0.001
ENT0.32 (−0.42, 1.06)0.393
Foregut/hepatopancreatobiliary1.03 (0.75, 1.31)<0.001
GBAAS/intestinal1.13 (0.87, 1.40)<0.001
Neck−0.04 (−0.58, 0.49)0.869
Obstetric/gynecologic0.12 (−0.32, 0.56)0.587
Orthopedic0.47 (0.20, 0.73)<0.001
Other abdomen0.16 (−0.32, 0.63)0.513
Peripheral vascular0.82 (0.55, 1.10)<0.001
Skin0.41 (−0.03, 0.84)0.065
Spine0.42 (0.08, 0.75)0.014
Thoracic1.06 (0.71, 1.41)<0.001
Vein1.35 (1.06, 1.64)<0.001
Urology0.55 (0.26, 0.85)<0.001
Functional status (Ref=independent)Partially dependent0.23 (0.06, 0.41)0.007
Totally dependent0.72 (0.43, 1.01)<0.001
Creatinine category (Ref <1.5)>1.50.57 (0.45, 0.70)<0.001
Missing−0.41 (−0.68, −0.14)0.002
Hx CHF (Ref=No)Yes0.60 (0.39, 0.80)<0.001
Diabetes mellitus (Ref=No)Yes, not insulin dependent0.09 (−0.04, 0.22)0.164
Yes, insulin dependent0.47 (0.34, 0.60)<0.001
Constant −6.79 (−7.70, −5.89)<0.001
ASA Class indicates American Society of Anesthesiology Physical Status Classification System Class; CI, confidence interval; GBAAS, Gallbladder, appendix, adrenals or spleen; Hx CHF, history of congestive heart failure; NSQIP, National Surgical Quality Improvement Program; OR, odds ratio.

Model Evaluation.

For comparison with the previously published risk scores, the coefficients from the 2013 derivation data were then used to predict the risk of MI in the NSQIP 2012 validation data set. There were a total of 485,426 participants in the NSQIP 2012 who were able to have this GSCRI computed. Previous published coefficients of the RCRI and Gupta MICA risk indices were used to predict the risk of postsurgical MI in the NSQIP 2012 data set. The predictive value of the models was used to calculate the area under the receiver operating characteristics curve for the overall sample and within the geriatric (age ≥65) age group and compared between models using the Delong method in the pROC package in R version 3.1.0. Plots of observed versus predicted risk (calibration plots) were used to visually assess the fit of these models in the geriatric group.

Evaluation of Upper Limit for Model Performance

Although we believe in the clinical utility of a parsimonious model for predicting MICA, modern computational algorithms can be used to maximize the predictive ability of the predictors by modeling complex nonlinear and high‐order interactions. Often called machine or statistical learning, these techniques provide greater predictive accuracy, though often at the expense of interpretability. In our study we used 1 such technique, stochastic gradient boosting, in order to provide us with an upper‐limit benchmark for our GSCRI model performance. Using a gradient‐boosted machines (GBM) model21, 22; with the gbm package in R,23 we evaluated the set of predictors from the RCRI, Gupta MICA, and GSCRI in the geriatric subset of the NSQIP 2013 derivation data set. Tenfold cross validation was used to select the interaction depth and number of iterations that minimized the cross‐validated error at a shrinkage rate of 0.01. The optimal model, once selected using cross validation, was then applied to the 2012 validation data set to predict MICA risk. Area under the curve (AUC) was computed from this model and used as our likely upper limit of predictive ability from the set of predictors under evaluation.

Results

In the NSQIP 2012 validation data set the majority of the sample was female (58%) and middle‐aged (age mean=57, SD=16), with few instances of perioperative MI (N=2357, ≈0.5%).
The odds of MICA were 4.8 times greater in those 65 or over (≈1% versus ≈0.2%). Figure 1 shows how the risk of MICA increases nonlinearly with age. Among geriatric patients, the risks of postoperative cardiac arrest and MI were 0.37% and 0.67%, respectively. The risk of death in this group was 1.64%, although these models do not evaluate this risk. Sample characteristics were not substantively different in the NSQIP 2013 derivation data. Clinical characteristics of the derivation and validation cohorts are displayed in Table S3.
image
Figure 1. Association of actual MICA incidence with age along with the linear and lowess fits. MICA indicates myocardial infarction and cardiac arrest.

Development of the GSCRI

The coefficients from the GSCRI in the NSQIP 2013 data set are displayed in Table 3. All variables were statistically significant predictors of MICA (P<0.05). Figure 2 shows the relative importance of each variable to the GSCRI in the derivation set (NSQIP 2013) as measured by partial χ2 minus the predictor degrees of freedom. ASA Class and surgical category were the most influential variables in the model, while congestive heart failure, diabetes mellitus, and functional status contribute almost equally and were least influential in forming the final index.
image
Figure 2. Importance of each variable to the GSCRI in the training set as measured by partial χ2 minus the predictor degrees of freedom. ASA Class indicates American Society of Anesthesiology Physical Status Classification System Class; GSCRI, Geriatric‐Sensitive Cardiac Risk Index; Hx CHF, history of congestive heart failure.

Comparison of Risk Scores

AUCs for each of the models can be found in Table 4. The GSCRI had a significantly higher AUC than either the RCRI or Gupta MICA in both the geriatric group (AUC=0.76) and overall sample (AUC=0.83). The Gupta MICA also outperformed the RCRI in the geriatric group (0.70 versus 0.63) and overall sample (0.72 versus 0.68). We additionally found the Gupta MICA model to be poorly calibrated (Figure 3), with an underestimation of risk in the geriatric sample. Although both the RCRI and GSCRI also underestimated the risk, the median difference from the observed risk was only −0.28 and −0.04 percentage points different, respectively, whereas the Gupta MICA was off by −0.73% in the geriatric patients. In Figure 4 we see that the Gupta MICA model severely underestimates the risk in the low‐risk categories and overestimates the risk in high‐risk categories. Given this, we find the Gupta MICA model to be poorly calibrated in the NSQIP 2012 validation data. Although the RCRI also underestimates risk for low‐risk patients, it is well calibrated for the highest‐risk group (2.4% to 5.4% in RCRI). The GSCRI is very well calibrated and tends to only slightly overestimate the risk in the highest predicted risk group. For the performance of risk prediction in geriatric patients for noncardiac surgery groups by age, see Table S4. Additionally, to compare the performance of this new risk score GSCRI with the RCRI as well as the Gupta MICA risk scores, net reclassification index (5% threshold) and integrated discrimination improvement metrics were compared, and the GSCRI provides an improvement over both the RCRI and Gupta MICA (see Table S5).
Table 4. Differential Performance of Risk Prediction in Geriatric Patients for Noncardiac Surgeries (NSQIP 2012)
 Age ≥65 y AUC (95% CI)Overall AUC (95% CI)
 Δ, P ValueΔ, P Value
RCRI0.63 (0.62, 0.65)0.68 (0.67, 0.69)
Gupta MICA0.70 (0.69, 0.71)0.72 (0.71. 0.73)
GSCRI0.76 (0.75, 0.77)0.83 (0.82, 0.83)
RCRI vs Gupta MICA0.07, P<0.0010.04, P<0.001
GSCRI vs RCRI0.13, P<0.0010.15, P<0.001
GSCRI vs Gupta MICA0.06, P<0.0010.11, P<0.001
AUC indicates area under the curve; CI, confidence interval; Δ, difference between indices; GSCRI, Geriatric‐Sensitive Cardiac Risk Index; Gupta MICA, Gupta Myocardial Infarction and Cardiac Arrest; NSQIP, National Surgical Quality Improvement Program; RCRI, Lee Revised Cardiac Risk Index.
image
Figure 3. Models' calibration among geriatric patients. Calibration plots compare the 3 studied models among geriatric patients. Gupta MICA model shows underestimation of the MICA risk in geriatric patients. GSCRI indicates Geriatric‐Sensitive Cardiac Risk Index; Gupta MICA, Gupta myocardial infarction and cardiac arrest; NSQIP, National Surgical Quality Improvement Program; RCRI, Lee Revised Cardiac Risk Index.
image
Figure 4. Alternate calibration plot. The average observed risk is presented for each of the predicted risk categories (defined by the RCRI). GSCRI indicates Geriatric‐Sensitive Cardiac Risk Index; Gupta MICA, Gupta myocardial infarction and cardiac arrest; RCRI, Lee Revised Cardiac Risk Index.

Upper Limit of Model Performance

The GBM model indicated an optimal interaction depth of 3 and 3086 iterations and was developed in the NSQIP 2013 derivation data set. When this model was applied to the 2012 validation data set, the AUC was found to be 0.79, indicating a likely upper limit of performance for these variables on geriatric patients in the NSQIP data sets. The GSCRI, with an AUC of 0.76, approaches this upper limit while maintaining parsimony.

Discussion

This article demonstrates the concept of developing a predictive model in the geriatric population, in contrast to other articles where the model development is across a wider age spectrum. Currently, geriatric patients have low participation in clinical trials and are often excluded due to age‐related comorbidities. When included, the data of older participants are often pooled together with participants of younger ages who have much lower risk, which possibly leads to inaccurate parameter estimation. Developing predictive models on these pooled data that ignore age categories can lead to models that are dominated by variables and coefficients not optimized for performance in geriatric patients and hence provide decreased predictive accuracy and lower sensitivity to certain geriatric characteristics. This holds true especially in the development of predictive models because even minor inaccuracies in the derivation of the parameter estimates could dramatically affect the discrimination and calibration of a model.
In this study our hypothesis of the need for specific geriatric analysis and model derivation proved to be valid, and our efforts culminated in producing the Geriatric‐Sensitive Perioperative Cardiac Risk Index, GSCRI. Our GSCRI has an AUC of 0.76 and outperformed the RCRI and Gupta MICA models by 13% (P<0.001) and 6% (P<0.001) in geriatric patients of the validation cohort, respectively (see Table 4). Although the GSCRI was developed for optimal performance in a geriatric population, we wished to test the GSCRI against the RCRI and Gupta MICA in the overall population as well. We found that the GSCRI has an AUC of 0.83, which outperformed the RCRI and Gupta MICA by 15% (P<0.001) and 11% (P<0.001), respectively (Table 4). When the Gupta MICA was tested with the published coefficients on geriatric patients, a significant deterioration (≈17%) from the previously published performance in the NSQIP 2007 was noted, and a significant underestimation of the risk was also noted, likely resulting from assuming a linearity of age and deriving estimates that are not specific for the geriatric population when conducting the analysis for the Gupta MICA calculator.
The GSCRI model contained 7 variables, and the first 3 variables (stroke, ASA Class, surgical category) were selected using a Least Angle Shrinkage and Selection Operator regression analysis in the NSQIP 2013 data on the geriatric subset. The method selects the most statistically important variables that contribute to the occurrence of the outcome. The other variables (diabetes mellitus, functional status, elevated creatinine >1.5 mg/dL, congestive heart failure) were added to include clinically significant variables that are common across various indices of perioperative cardiac risk.5, 19 Although additional relevant variables could have been added, the increased model complexity would not meaningfully improve the model's predictive ability based on our examination of the upper limit of model performance from the candidate variables. Creating a parsimonious model was essential to ensure the ease of use that physicians working in clinical settings require.
Our data‐driven variable selection method (Least Angle Shrinkage and Selection Operator) selected 3 variables, but we felt that the addition of common clinically important risk factors to the model would improve estimation of the MICA risk in geriatric patients. The inclusion of additional risk factors that are known to increase the surgical cardiac risk, such as congestive heart failure,5 elevated creatinine,5, 19 diabetes mellitus,5, 19 and functional status,19 may be important in generalizing the GSCRI to novel samples outside of NSQIP. In modeling these variables, we chose a creatinine level of 1.5 mg/dL as opposed to 2 mg/dL because geriatric patients often have a decreased glomerular filtration rates with lower serum creatinine levels in comparison with younger patients.24 The performance of the GBM achieved a performance of AUC=0.79, indicating that a complex model containing higher‐order interactions would achieve a performance close to our model (ΔAUC=3%); therefore, we believe our model was able to achieve good performance without the loss of interpretability common to statistical learning algorithms such as GBM.
These models reflect contemporary risk associated with each surgical category; hence, updating these models every few years is imperative to take into account the improved surgical outcomes and decreased complication rates that result from enhanced medical care and improved surgical techniques. This possibly explains why the GSCRI outperformed the other 2 models in nongeriatric patients, as it has the advantage of being tested on a data set only 1 year apart from the derivation data set, whereas the Gupta MICA was developed on a 2007 data set.19 Additionally, the RCRI was not derived to predict the cardiac risk within 30 days of surgery but is aimed solely at predicting the risk during a hospital stay.5 In the modern world, using equations developed so long ago and on unique populations (ie, RCRI) is of questionable value, particularly in an era when curated data sources such as the NSQIP and other large data sets are readily available.
With the growth of the geriatric population and increased awareness of the uniqueness of this growing segment,11, 25 The GSCRI represents a step forward for cardiac risk prediction for geriatric patients. Our study demonstrated the necessity of developing risk models optimized for geriatric patients in order to produce accurate predictions. The GSCRI outperforms the Gupta MICA and RCRI in the AUC by 7% and 13%, respectively.
We believe we might have reached a predictive limit in our ability to predict perioperative risk in geriatric patients in our sample. GBM, an exploratory statistical learning technique, was used to examine the maximal predictive ability for the set of predictors available in the NSQIP 2012 data set. One strength of this technique is the ability to utilize nonlinear and high‐order interactions that provide the maximal predictive accuracy for the outcome given the data set; however, it comes at the expense of interpretability. Even with the complex GBM modeling we could not reach a C‐statistic that was >0.8 in the geriatric patients. This low value is not unexpected, especially given the wide variation in the health status of geriatric patients. We may need to consider other variables for predicting the risk in geriatric patients in order to achieve more accurate predictions; however, we are currently limited by the variables that are available in the NSQIP databases.
The findings of the study were driven by our initial hypothesis, and therefore, the GSCRI represents a hybrid of hypothesis and data‐driven approaches. We hope our novel index may help set a new standard in surgical risk estimation for geriatric patients. To facilitate that purpose, we intend to develop an online calculator to increase the utility of the GSCRI for physicians. Physicians will be able to readily estimate the cardiac surgical risk for a geriatric patient by answering 7 online questions, and the index will produce cardiac risk probability by integrating these answers into the equation of the GSCRI from Table 3. Finally, we would like to stress that the GSCRI should be accompanied by clinical evaluation and comprehensive geriatric assessment to add further insights to the actual risk. Because no risk model can substitute for the clinical judgment of physicians, the GSCRI is meant to be a supplemental tool to aid in the process of perioperative cardiac risk management.
Our study has certain limitations that need to be considered. There is a paucity of geriatric‐specific data available in the data set. Additional variables could provide further prognostication and predictive value that is relevant for this population. Therefore, this model is the first of a series of models that will need to be updated by integrating geriatric‐relevant data in a timely manner. Hence, our future endeavors will focus on integrating and testing the usability of biologic variables such as inflammatory factors and other significant factors including nutritional status, functional status, depression, cognition, and frailty indices.2, 11, 26, 27, 28 In addition, the external validation was conducted using a different data‐set year from the same organization: NSQIP. Additional studies further validating this model using data from different external data sets are warranted. An additional limitation was the finding that functional status had a small impact on the outcome, perhaps because fewer than 1% of the patients were “totally dependent,” but also, perhaps, we need a much better classification system for functional status than the simple 3 categorical variables, a true limitation of the available data.
In conclusion, current risk‐prediction models have poor performance in geriatric patients; therefore, we developed a new index, the GSCRI, to help improve the accuracy of prediction in this unique population. The GSCRI as a new tool is a work in progress, and we expect to continuously update and improve this new index as more data become available.

Supplemental Material

File (jah32691-sup-0001-supinfo.pdf)

References

1.
Vincent GK, Velkoff VA. The Next Four Decades: The Older Population in the United States: 2010 to 2050. Washington, DC: US Department of Commerce, Economics and Statistics Administration, US Census Bureau; 2010.
2.
Groban L, Kim S, Brooks A. Preoperative assessment of the older surgical patient: honing in on geriatric syndromes. Clin Interv Aging. 2015;10:13–27.
3.
Werner CA. The older population: 2010. Report Number: C2010BR‐09. Available at https://www.census.gov/library/publications/2011/dec/c2010br-09.html. Accessed October 28, 2001.
4.
McGory ML, Kao KK, Shekelle PG, Rubenstein LZ, Leonardi MJ, Parikh JA, Fink A, Ko CY. Developing quality indicators for elderly surgical patients. Ann Surg. 2009;250:338–347.
5.
Lee TH, Marcantonio ER, Mangione CM, Thomas EJ, Polanczyk CA, Cook EF, Sugarbaker DJ, Donaldson MC, Poss R, Ho KKL, and others. Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation. 1999;100:1043–1049.
6.
L'Italien GJ, Paul SD, Hendel RC, Leppo JA, Cohen MC, Fleisher LA, Brown KA, Zarich SW, Cambria RP, Cutler BS, Eagle KA. Development and validation of a Bayesian model for perioperative cardiac risk assessment in a cohort of 1,081 vascular surgical candidates. J Am Coll Cardiol. 1996;27:779–786.
7.
Mangano DT, Layug EL, Wallace A, Tateo I. Effect of atenolol on mortality and cardiovascular morbidity after noncardiac surgery. N Engl J Med. 1996;335:1713–1721.
8.
Devereaux PJ. Perioperative cardiac events in patients undergoing noncardiac surgery: a review of the magnitude of the problem, the pathophysiology of the events and methods to estimate and communicate risk. Can Med Assoc J. 2005;173:627–634.
9.
Sprung J, Warner ME, Contreras MG, Schroeder DR, Beighley CM, Wilson GA, Warner DO. Predictors of survival following cardiac arrest in patients undergoing noncardiac surgery: a study of 518,294 patients at a tertiary referral center. Anesthesiology. 2003;99:259–269.
10.
Roger VL, Go AS, Lloyd‐Jones DM, Adams RJ, Berry JD, Brown TM, Carnethon MR, Dai S, de Simone G, Ford ES, Fox CS, Fullerton HJ, Gillespie C, Greenlund KJ, Hailpern SM, Heit JA, Ho PM, Howard VJ, Kissela BM, Kittner SJ, Lackland DT, Lichtman JH, Lisabeth LD, Makuc DM, Marcus GM, Marelli A, Matchar DB, McDermott MM, Meigs JB, Moy CS, Mozaffarian D, Mussolino ME, Nichol G, Paynter NP, Rosamond WD, Sorlie PD, Stafford RS, Turan TN, Turner MB, Wong ND, Wylie‐Rosett J; on behalf of the American Heart Association Statistics Committee, Stroke Statistics Subcommittee . Heart disease and stroke statistics—2011 update: a report from the American Heart Association. Circulation. 2011;123:e18–e209.
11.
Chow WB, Rosenthal RA, Merkow RP, Ko CY, Esnaola NF. Optimal preoperative assessment of the geriatric surgical patient: a best practices guideline from the American College of Surgeons National Surgical Quality Improvement Program and the American Geriatrics Society. J Am Coll Surg. 2012;215:453–466.
12.
Robinson TN, Rosenthal RA. The ACS NSQIP Geriatric Surgery Pilot Project: improving care for older surgical patients. Bull Am Coll Surg. 2014;99:21–23.
13.
Alrezk R, Al Rezk M, Jackson N, Elashoff D. Risk factors for myocardial infarction/cardiac arrest (MICA) have differential weight in geriatrics vs. non‐geriatrics patients. Eur Heart J. 2016;37:1335–1335. Abstract.
14.
Alrezk R, Al Rezk M. Congestive heart failure has differential perioperative risk for myocardial infarction/cardiac arrest (MICA) in geriatrics vs. non‐geriatrics patients. Eur J Heart Fail. 2016;18:53. Abstract.
15.
Patel AY, Eagle KA, Vaishnava P. Cardiac risk of noncardiac surgery. J Am Coll Cardiol. 2015;66:2140–2148.
16.
Devereaux PJ, Sessler DI. Cardiac complications in patients undergoing major noncardiac surgery. N Engl J Med. 2015;373:2258–2269.
17.
Mangano DT, Browner WS, Hollenberg M, London MJ, Tubau JF, Tateo IM. Association of perioperative myocardial ischemia with cardiac morbidity and mortality in men undergoing noncardiac surgery. N Engl J Med. 1990;323:1781–1788.
18.
ACS NSQIP PUF DATA USER GUIDE 2013. ACS NSQIP. 2014.
19.
Gupta PK, Gupta H, Sundaram A, Kaushik M, Fang X, Miller WJ, Esterbrooks DJ, Hunter CB, Pipinos II, Johanning JM, Lynch TG, Forse RA, Mohiuddin SM, Mooss AN. Development and validation of a risk calculator for prediction of cardiac risk after surgery. Circulation. 2011;124:381–387.
20.
Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc Series B Stat Methodol. 1996;58:267–288.
21.
Friedman JH. Greedy function approximation: a gradient boosting machine. Ann Stat. 2001;29:1189–1232.
22.
Friedman JH. Stochastic gradient boosting. Comput Stat Data Anal. 2002;38:367–378.
23.
Ridgeway G. gbm: Generalized Boosted Regression Models. R package version 2.0‐8. 2013.
24.
Munikrishnappa D. Limitations of various formulae and other ways of assessing GFR in the elderly: is there a role for cystatin C? Nephrology. 2009; Available at: https://www.asn-online.org/education/distancelearning/curricula/geriatrics/Chapter6.pdf. Accessed October 28, 2001.
25.
Mohanty S, Rosenthal RA, Russell MM, Neuman MD, Ko CY, Esnaola NF. Optimal perioperative management of the geriatric patient: a best practices guideline from the American College of Surgeons NSQIP and the American Geriatrics Society. J Am Coll Surg. 2016;222:930–947.
26.
Clegg A, Young J, Iliffe S, Rikkert MO, Rockwood K. Frailty in elderly people. Lancet. 2013;381:752–762.
27.
Makary MA, Segev DL, Pronovost PJ, Syin D, Bandeen‐Roche K, Patel P, Takenaga R, Devgan L, Holzmueller CG, Tian J, Fried LP. Frailty as a predictor of surgical outcomes in older patients. J Am Coll Surg. 2010;210:901–908.
28.
Collard RM, Boter H, Schoevers RA, Oude Voshaar RC. Prevalence of frailty in community‐dwelling older persons: a systematic review. J Am Geriatr Soc. 2012;60:1487–1492.

eLetters(0)

eLetters should relate to an article recently published in the journal and are not a forum for providing unpublished data. Comments are reviewed for appropriate use of tone and language. Comments are not peer-reviewed. Acceptable comments are posted to the journal website only. Comments are not published in an issue and are not indexed in PubMed. Comments should be no longer than 500 words and will only be posted online. References are limited to 10. Authors of the article cited in the comment will be invited to reply, as appropriate.

Comments and feedback on AHA/ASA Scientific Statements and Guidelines should be directed to the AHA/ASA Manuscript Oversight Committee via its Correspondence page.

Information & Authors

Information

Published In

Go to Journal of the American Heart Association
Go to Journal of the American Heart Association
Journal of the American Heart Association
PubMed: 29146612

History

Received: 14 May 2017
Accepted: 19 September 2017
Published in print: 1 November 2017
Published online: 15 November 2017

Permissions

Request permissions for this article.

Keywords

  1. geriatrics
  2. myocardial infarction
  3. perioperative risk model
  4. risk prediction
  5. risk stratification
  6. surgery

Subjects

Authors

Affiliations

Rami Alrezk, MD, MS, FACP* [email protected]
VA Greater Los Angeles Healthcare System, Los Angeles, CA
Division of Geriatrics, David Geffen School of Medicine at UCLA, Los Angeles, CA
GRECC VA Greater Los Angeles Healthcare System, Los Angeles, CA
Department of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA
Nicholas Jackson, MPH, MA
Department of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA
Mohanad Al Rezk, MD
VA Greater Los Angeles Healthcare System, Los Angeles, CA
Robert Elashoff, PhD
Department of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA
Nancy Weintraub, MD
VA Greater Los Angeles Healthcare System, Los Angeles, CA
Division of Geriatrics, David Geffen School of Medicine at UCLA, Los Angeles, CA
GRECC VA Greater Los Angeles Healthcare System, Los Angeles, CA
Department of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA
David Elashoff, PhD
Department of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA
Gregg C. Fonarow, MD
Division of Cardiology, David Geffen School of Medicine at UCLA, Los Angeles, CA
Department of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA

Notes

*
Correspondence to: Rami Alrezk, MD, MS, FACP, National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), National Institutes of Health, 10 Center Drive, B10CRC, Rm 6‐3940, Bethesda, MD 20892. E‐mail: [email protected]

Disclosures

None. The contents do not represent the views of the US Department of Veterans Affairs, National Institutes of Health (NIH), or the US Government. The American College of Surgeons National Surgical Quality Improvement Program and the hospitals participating in the ACS NSQIP are the source of the data used herein; they have not verified and are not responsible for the statistical validity of the data analysis or the conclusions derived by the authors.

Funding Information

National Institutes of Health
National Center for Advancing Translational Sciences
University of California, Los Angeles
Clinical and Translational Science Institute: UL1TR001881
Department of Veterans Affairs
This research was supported by the National Institutes of Health, National Center for Advancing Translational Sciences (NCATS), University of California, Los Angeles (UCLA), Clinical and Translational Science Institute (CTSI) Grant Number UL1TR001881. This material is based on work supported by the office of Academic Affiliations, Department of Veterans Affairs.

Metrics & Citations

Metrics

Citations

Download Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Select your manager software from the list below and click Download.

  1. Perioperative Cardiovascular Outcomes and Risk Assessment in Older Adults for Noncardiac Surgery, Current Anesthesiology Reports, 15, 1, (2025).https://doi.org/10.1007/s40140-024-00659-4
    Crossref
  2. Continuity of Care: A Multidimensional Vision for the Frail Patient, The Frail Surgical Patient, (363-370), (2025).https://doi.org/10.1007/978-3-031-77707-3_25
    Crossref
  3. Comprehensive Peri-Operative Risk Assessment and Management of Geriatric Patients, Diagnostics, 14, 19, (2153), (2024).https://doi.org/10.3390/diagnostics14192153
    Crossref
  4. PREDICTION OF CARDIOVASCULAR RISK IN ABDOMINAL SURGERY: RESULTS OF AN OBSERVATIONAL MULTICENTER STUDY, Complex Issues of Cardiovascular Diseases, 13, 2, (176-195), (2024).https://doi.org/10.17802/2306-1278-2024-13-2-176-195
    Crossref
  5. A simple machine learning model for the prediction of acute kidney injury following noncardiac surgery in geriatric patients: a prospective cohort study, BMC Geriatrics, 24, 1, (2024).https://doi.org/10.1186/s12877-024-05148-1
    Crossref
  6. 2024 AHA/ACC/ACS/ASNC/HRS/SCA/SCCT/SCMR/SVM Guideline for Perioperative Cardiovascular Management for Noncardiac Surgery: A Report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines, Circulation, 150, 19, (e351-e442), (2024)./doi/10.1161/CIR.0000000000001285
    Abstract
  7. Iterative random forest-based identification of a novel population with high risk of complications post non-cardiac surgery, Scientific Reports, 14, 1, (2024).https://doi.org/10.1038/s41598-024-78482-4
    Crossref
  8. Postoperative short-term mortality between insulin-treated and non-insulin-treated patients with diabetes after non-cardiac surgery: a systematic review and meta-analysis, Frontiers in Medicine, 10, (2023).https://doi.org/10.3389/fmed.2023.1142490
    Crossref
  9. Differentiated approach to preoperative preparation of patients with various cardiovascular risk in locally advanced cancer of abdominal organs: experimental study, Marine Medicine, 9, 4, (72-84), (2023).https://doi.org/10.22328/2413-5747-2023-9-4-72-84
    Crossref
  10. Modern approaches to projection and differentiated assessment of cardiovascular risks in surgery: review, Marine Medicine, 8, 4, (7-17), (2023).https://doi.org/10.22328/2413-5747-2022-8-4-7-17
    Crossref
  11. See more
Loading...

View Options

View options

PDF and All Supplements

Download PDF and All Supplements

PDF/EPUB

View PDF/EPUB
Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Personal login Institutional Login
Purchase Options

Purchase this article to access the full text.

Purchase access to this article for 24 hours

Restore your content access

Enter your email address to restore your content access:

Note: This functionality works only for purchases done as a guest. If you already have an account, log in to access the content to which you are entitled.

Figures

Tables

Media

Share

Share

Share article link

Share

Comment Response