Skip main navigation

External Validation of Risk Prediction Models to Improve Selection of Patients for Carotid Endarterectomy

Originally publishedhttps://doi.org/10.1161/STROKEAHA.120.032527Stroke. 2022;53:87–99

Abstract

Background and Purpose:

The net benefit of carotid endarterectomy (CEA) is determined partly by the risk of procedural stroke or death. Current guidelines recommend CEA if 30-day risks are <6% for symptomatic stenosis and <3% for asymptomatic stenosis. We aimed to identify prediction models for procedural stroke or death after CEA and to externally validate these models in a large registry of patients from the United States.

Methods:

We conducted a systematic search in MEDLINE and EMBASE for prediction models of procedural outcomes after CEA. We validated these models with data from patients who underwent CEA in the American College of Surgeons National Surgical Quality Improvement Program (2011–2017). We assessed discrimination using C statistics and calibration graphically. We determined the number of patients with predicted risks that exceeded recommended thresholds of procedural risks to perform CEA.

Results:

After screening 788 reports, 15 studies describing 17 prediction models were included. Nine were developed in populations including both asymptomatic and symptomatic patients, 2 in symptomatic and 5 in asymptomatic populations. In the external validation cohort of 26 293 patients who underwent CEA, 702 (2.7%) developed a stroke or died within 30-days. C statistics varied between 0.52 and 0.64 using all patients, between 0.51 and 0.59 using symptomatic patients, and between 0.49 to 0.58 using asymptomatic patients. The Ontario Carotid Endarterectomy Registry model that included symptomatic status, diabetes, heart failure, and contralateral occlusion as predictors, had C statistic of 0.64 and the best concordance between predicted and observed risks. This model identified 4.5% of symptomatic and 2.1% of asymptomatic patients with procedural risks that exceeded recommended thresholds.

Conclusions:

Of the 17 externally validated prediction models, the Ontario Carotid Endarterectomy Registry risk model had most reliable predictions of procedural stroke or death after CEA and can inform patients about procedural hazards and help focus CEA toward patients who would benefit most from it.

Carotid endarterectomy (CEA) aims to prevent long-term stroke and should be performed in patients who may derive greatest benefit in terms of stroke risk reduction. Symptomatic patients who have had a recent stroke or transient ischemic attack (TIA) related to an ipsilateral high-grade stenosis are recommended to undergo CEA in addition to medical therapy.1–5 The absolute benefits of CEA have become smaller due to improvement of medical preventive therapy in patients without recent symptoms related to the carotid stenosis.6–9 The net benefit depends not only on the long-term reductions in stroke risk but is also determined by the procedural hazards of CEA.

Current guidelines recommend to consider CEA in patients who have a risk of 30 day stroke or death of <6% in symptomatic patients and <3% in asymptomatic patients who also have a life expectancy of 5 years.1,10 The risk thresholds for 30 day stroke or death might be reduced to 4% for symptomatic and 2% for asymptomatic patients as a result of improved medical therapy.

Risk prediction models might help to inform patients about procedural hazards and possibly patient selection for CEA by providing individualized risk estimations by taking several patient and disease characteristics into account.11 Before implementation of prognostic risk prediction models in clinical practice, external validation in a contemporary cohort of patients who underwent CEA is needed.

We aimed to identify available risk prediction models of procedural stroke or death after CEA and to externally validate these models in a large contemporary registry of patients who underwent CEA in the United States.

Methods

Data Sharing

The data that support the findings of this study are available to researchers participating in NSQIP.

Systematic Review

We conducted a systematic review according to a protocol that we registered (PROSPERO CRD42019141835), and report the results of our systematic review consistent with the Preferred Reporting Items for Systematic reviews and Meta-Analysis.12

Search Strategy

We performed electronic searches in MEDLINE (via PubMed interface) and EMBASE (via EMBASE interface) from December 2016 to January 1, 2020 to update a previous systematic review of risk prediction models for outcomes after carotid revascularization (Table I in the Data Supplement).11

Eligibility Criteria

We included studies that (1) addressed development (with or without internal or external validation) of prognostic prediction models to select patients for CEA using procedural (in-hospital or 30-days) risk of stroke or death as predicted outcome; (2) in patients with carotid stenosis; (3) regardless of symptomatic status; (4) using predictors that are available before the intervention and can be used for patient selection; (5) based on data from observational studies or randomized clinical trials; (6) without restrictions on baseline characteristics such as age, sex, or ethnicity; and (7) were published in peer-reviewed journals without any language restrictions.

Screening Process and Data Extraction

Two authors (Drs Poorthuis and Dansey) independently screened all titles and abstracts of the retrieved references and subsequently independently reviewed full-text copies for final inclusion in this study. We performed backward citation searching using the bibliographies of included studies.

Two authors (Dr Poorthuis and Herings) independently extracted the following data from the included prediction models based on the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Model Studies checklist13: source of data, study setting, geographic area (country and continent), study years, modeling method (eg, logistic model), proportion of participants with missing data, handling of missing data, appropriateness of modeling assumptions, methods for predictor selection, shrinkage of predictor weights, number of outcome events, number of participants, degree of stenosis, number and type of predictors used in the final model, number of outcome events per variable, presentation of model, model performance (discrimination and calibration). In studies that reported internal validation of prediction models, we extracted the following additional data: method of internal validation (eg, cross-validation, bootstrap); whether the model was adjusted or updated after internal validation. In studies reporting external validation of a prediction model, we extracted the following additional data: type of external validation (eg, geographic and temporal distinct population); whether authors of the external validation also developed the original model; performance of the model before or after model recalibration.

Critical Appraisal

One author (Herings) assessed the included prediction models for the risk of bias and applicability using the Prediction model Risk of Bias ASsessment Tool and the assessment was supervised by one author (Dr Damen).14 The assessment of risk of bias consisted of 4 domains (participants; predictors; outcome; and analysis) and the applicability consisted of 3 domains (participants; predictors; and outcome). Risk of bias and applicability was judged as low, high, or unclear for each domain.15 Each model was evaluated separately if multiple models were developed in one study.

External Validation

For external validation, we adhered to the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis statement. The Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis checklist is provided as Table II in the Data Supplement.16

External Validation Cohort

We used a prospectively maintained cohort of patients who were registered between January 2011 and December 2017 in the Targeted Vascular module of the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) registry for external validation of the prediction models identified in our systematic review.17 In 2017, 708 participating hospitals collected data of patients who underwent a carotid intervention, including 30-days procedural outcomes. Classification of procedures was based on Current Procedural Terminology of the American Medical Association. Trained surgical clinical reviewers in each hospital collected data from medical charts and operative case logs using strict variable definitions to maintain uniformity across hospitals. Patients were contacted by letter or phone calls at 30-days after the carotid intervention to obtain data on procedural complications after discharge, if necessary. In addition, the ACS-NSQIP performed random reliability auditing to minimize information bias. Definition of variables in ACS-NSQIP and their validity have been investigated in previous reports.18

The Targeted Vascular Module of the ACS-NSQIP recorded additional disease- and procedure-related data and outcomes that were deemed crucial by vascular surgeons in a subset of 29 participating hospital of all hospitals performing vascular interventions.

Predicted Outcome

We externally validated the prediction models for stroke or death within 30-days after CEA. Procedural strokes were defined as any new acute focal neurological deficit lasting >24 hours.

Statistical Analysis

Characteristics of the external validation cohort were summarized using standard methods.

Missing data were imputed using chained equation with the MICE package in R. The imputation model included the predicted outcome of procedural death or stroke and predictors of the included prediction models. The algorithm started with the lowest proportion of missing data. The imputation continued until convergence of each predictor with a maximum of 20 iterations for each imputed data set. Fifteen imputed data sets were computed. The number of imputed data sets was determined by taking the highest percentage of missing values and round that to the closest multiple of 5.19,20 The imputation was evaluated graphically with convergence plots.

The regression formula, including the intercept and beta-coefficients (predictor weights) that allows calculation of the predicted probabilities, was used to calculate the 30-day risk of stroke or death for each patient. We contacted authors to provide the regression formula if it was not provided in the original report. If the authors could not provide the regression formula, we calculated a sum score (total points) for each participant by summing the scores of the score chart assigned to each predictor in the original reports. We matched the predictors with the variables in the external validation cohort. Proxies were used if a direct match was not available. An overview of the proxies is provided in Table III in the Data Supplement. Predictors were excluded from the external validation if no proxy was available. The risk equations to calculate the predicted probabilities used in our external validation are provided in Table IV in the Data Supplement.

We assessed the predictive performance in terms of discrimination and calibration of the included prediction models. Discrimination is the ability of the prediction model to distinguish between participants with and without stroke or death within 30-days after CEA and was assessed using C statistics. C statistics were calculated per imputed data set and results were combined using Rubin’s rules.21,22 Calibration is the agreement between predicted risk and observed risk and was assessed using calibration plots showing predicted risks calculated with the prediction models against the observed risks in the external validation cohort. The predicted probabilities were split in deciles to enable comparison between calibration plots, and the mean predicted and observed risk with corresponding 95% CIs was calculated for each decile. We used a lower number of groups (with a minimum number of 500 patients for each group to obtain precise estimates) for models that did not allow splitting in deciles because the variation of probabilities was too limited. Calibration plots were created for each imputed data set, and we found that the calibration plots did not differ materially across the imputed data sets. The calibration plots using the fifteenth imputed data set were, therefore, presented. We recalibrated the prediction models to the mean incidence of 30-days stroke or death in our external validation cohort to adapt the models to current clinical practice, and because some models were developed with either stroke alone or death alone as predicted outcome or were restricted to outcomes that occurred in-hospital. For this, we re-estimated the intercept (referred to as recalibration-in-the-large or updating the intercept) by fitting a logistic model with a fixed slope and the intercept as the only free parameter.23

Three assessments of discrimination and calibration were performed: (1) including all patients who underwent CEA regardless of symptomatic status; (2) in patients who underwent CEA for symptomatic carotid stenosis; and (3) for CEA in patients with asymptomatic carotid stenosis.

We calculated the number of symptomatic patients with risk of procedural stroke or death exceeding 6% and 4% and the number of asymptomatic patients with risk of procedural stroke or death exceeding 3% and 2%.

We performed sensitivity analysis in complete cases.

R version 3.5.1 was used for statistical analyses and constructing figures.

Results

After screening of 788 unique reports and assessing the full-texts of 59 for eligibility, we included 15 studies reporting 17 prediction models (Figure 1 and Table V in the Data Supplement).24–38 Two (12%) models were developed in populations of symptomatic patients,33,34 5 (29%) models in populations of asymptomatic patients,35–37 and 9 (53%) in populations of both symptomatic and asymptomatic patients (Table 1). Symptomatic status was included as predictor in these nine prediction models24–32 of which one used type of qualifying event as predictor.28 Other predictors used frequently in the 17 included models were age in 8 (47%),25–29,32,36,38 sex in 7 (41%),28,32,34,35,37 heart failure in 11 (65%),24–30,37,38 coronary heart disease in 7 (41%),24,28,35–37 and degree of contralateral stenosis in 7 (47%).25,27,29,30,33,35,37 An overview of the included predictors is provided in Figure 2. The number of predictors varied from 3 to 11. The number of patients used for development varied from 218 to 39 411. Four (24%) models considered outcome events before discharge25–27,38 and 13 (76%) outcome events during 30-days after CEA.24,28–37 Nine models (59%) were internally validated (Table VI in the Data Supplement).28,29,34–38

Table 1. Selected Characteristics of Included Prediction Models

First author, year of publicationSymptomatic/asymptomatic patientsPredicted outcome(s)No. of events/No. of patients (%)Timeframe of outcomeNo. of predictors
Sridharan et al, 201824BothStroke, death, MI56/1496 (3.7)30 days7
Eslami et al, 201625BothStroke, MI, death, or discharge to rehabilitation facility389/8661 (4.5)In-hospital8
Chaudhry et al, 201626BothStroke, cardiac complications, or death1494/49 411 (3.0)In-hospital7
Wimmer et al, 201427BothStroke or death213/12 889 (1.7)In-hospital7
Bekelis et al, 201328BothStroke, MI, or death994/35 698 (2.8)30 days11
Goodney et al, 200829BothStroke or death60/3092 (1.9)30 days6
Tu et al, 200330BothStroke or death362/6038 (5.9)30 days5
Kuhan et al, 200131BothMajor stroke or death29/741 (3.9)30 days3
Kucey et al, 199832BothStroke or death81/1280 (6.3)30 days7
Stavrinou et al, 201633SymptomaticTIA, PRIND, amaurosis fugax, or MI12/218 (5.5)30 days6
Rothwell et al, 199934SymptomaticMajor stroke or death84/1203 (6.9)30 days3
DeMartino et al, 201735AsymptomaticStroke287/31 939 (0.9)30 days11
Gupta et al, 201336AsymptomaticStroke, MI, or death324/17 692 (1.8)30 days6
Calvillo-King et al, 2010a37AsymptomaticStroke or death200/6553 (3.1)30 days8
Calvillo-King et al, 2010b37AsymptomaticStroke or death200/6553 (3.1)30 days7
Calvillo-King et al, 2010c37AsymptomaticStroke165/6553 (2.5)30 days7
Matsen et al, 200538NRDeath125/23 237 (0.5)In-hospital6

MI indicates myocardial infarction; PRIND, prolonged reversible ischemic neurological deficit; and TIA, transient ischemic attack.

Figure 1.

Figure 1. Flowchart of literature search.

Figure 2.

Figure 2. Overview of included predictors. ASA indicates American Society of Anesthesiologists; COPD, chronic obstructive pulmonary disease; ICA, internal carotid artery; mRS, modified Rankin Scale; and TIA, transient ischemic attack. *Symptomatic status was included as predictor in all studies that were developed in populations of both asymptomatic and symptomatic patients.

Risk of Bias

The overall risk of bias was deemed low in 4 models,25,27,35,36 unclear in one model,28 and high in 12 models.24,26,29–34,37,38 Concerns with applicability of the models to our population of interest was deemed low in 5 models,26,27,29,30,35 and high in 12 models.24,25,28,31–34,36–38 Reasons for high concerns included using a different predicted outcome for development compared with our external validation or using single center data for development. An overview of the risk of bias and applicability of each model is provided in Table VII in the Data Supplement.

External Validation

The validation cohort consisted of 26 293 patients who underwent CEA, of whom 702 (2.7%) developed a stroke or died within 30-days. In total, 423 (3.8%) of the 11 035 symptomatic patients developed a stroke or died within 30-days, and 267 (1.8%) of the 14 772 asymptomatic patients. In the group of patients with symptomatic carotid stenosis, the qualifying event was stroke for 5096 (46%) patients, hemispheric TIA for 4083 (37%) patients, and amaurosis fugax for 1856 (17%) patients. Characteristics of the external validation cohort in the models are provided in Table 2.

Table 2. Selected Characteristics of External Validation Cohort

All patients (n=26 293)Patients with procedural stroke or death (n=702)Patients without procedural stroke or death (n=25 591)Percentage of participants with missing data
Patient characteristics
 Age, y71±9.272±9.571±9.20
 Male sex16 136 (61%)434 (62%)15 702 (61%)0
 Diabetes8082 (31%)261 (37%)7821 (31%)0
 Current smoker7011 (27%)210 (30%)6801 (27%)0
 COPD2650 (10%)99 (14%)2551 (10%)0
 Heart failure380 (2%)34 (5%)346 (1%)0
 Non-White race1720 (7%)56 (9%)1664 (7%)9.0
 Preoperative hematocrit, %39±4.939±5.740±4.93.8
 Preoperative creatinine, mg/dL1.1±0.721.2±0.901.1±0.713.5
 Preprocedural antiplatelet medication23 426 (89%)617 (89%)22 809 (89%)0.4
 Antihypertensive drugs use21 924 (83%)606 (86%)21 318 (83%)0
 ASA classification0.1
  ASA I–III20 997 (80%)461 (65%)20 536 (80%)
  ASA IV–V5262 (20%)240 (35%)5022 (20%)
 Functional status0.1
  Independent25 541 (97%)662 (94%)24 878 (97%)
  Partially dependent672 (3%)34 (5%)638 (2%)
  Totally dependent44 (0%)5 (1%)39 (1%)
Disease characteristics
 Symptomatic status1.8
  Stroke5096 (20%)249 (36%)4847 (19%)
  Hemispheric TIA4083 (16%)142 (21%)3941 (16%)
  Amaurosis fugax1856 (7%)32 (5%)1824 (7%)
  Asymptomatic14 772 (57%)267 (38%)14 505 (58%)
 Ipsilateral ICA stenosis*2.1
  <50%316 (1%)8 (1%)308 (1%)
  50%–79%7875 (31%)218 (32%)7657 (31%)
  80%–99%17 245 (67%)449 (65%)16 796 (67%)
  100%296 (1%)11 (2%)285 (1%)
 Contralateral ICA stenosis*11.8
  <50%13 281 (58%)290 (46%)12 991 (58%)
  50%–79%7169 (31%)218 (36%)6951 (31%)
  80%–99%1725 (7%)65 (11%)1660 (7%)
  100%1004 (4%)40 (7%)964 (4%)
Elective surgery22 194 (82%)458 (64%)21 035 (82%)0.1

Continuous variables are presented as mean±SD, categorical variables are presented as N (%). ASA indicates American Society of Anesthesiologists; COPD, chronic obstructive pulmonary disease; CT, computed tomography; ICA, internal carotid artery; and TIA, transient ischemic attack.

* Measured with doppler ultrasound or CT angiography.

All Patients

The C statistics of 9 prediction models varied between 0.60 and 0.6425–27, 29, 30, 32, 33, 37 and of 8 between 0.52 and 0.5924, 28, 31, 34–38 in the validation population (Figure 3). The 3 models with the highest C statistics were Ontario Carotid Endarterectomy Registry (OCER) model with 0.64 (95% CI, 0.62–0.66), Vascular Study Group of New England model with 0.64 (95% CI, 0.62–0.66), and The National Cardiovascular Data Registry model with 0.62 (95% CI, 0.60–0.64).25,27,30 Discrimination values of the original models are provided in Table VIII in the Data Supplement.

Figure 3.

Figure 3. Discriminative performance of risk prediction models.

The calibration plots of the National Cardiovascular Data Registry and Vascular Study Group of New England models showed overestimations of risks in the highest risk groups (Figure I in the Data Supplement). The OCER model showed the best concordance across all risk groups (Figure 4).30 The OCER model was developed in a population that consisted of symptomatic and asymptomatic patients and was validated in our validation cohort using symptomatic status (stroke or hemispheric TIA versus amaurosis fugax or retinal infarct versus asymptomatic), diabetes, heart failure, and contralateral occlusion as predictors. Most patients (42.9%) were in the lowest risk group with a predicted and observed risk of 1.9% and 1.4%, respectively. The highest risk group of 638 (2.3%) patients showed a predicted and observed risk of 6.0% and 6.7%, respectively (Figure 4). Calibration plots of other validated models are provided in Figures I and II and Table IX in the Data Supplement.

Figure 4.

Figure 4. Calibration plots of the Ontario Carotid Endarterectomy Registry (OCER) model.

Symptomatic Patients

External validation in 11 035 patients who underwent CEA for symptomatic carotid stenosis showed C statistics that varied between 0.51 and 0.59 (Figure 3).24–38 Two models were developed in populations of symptomatic patients and had C statistics of 0.59 (95% CI, 0.56–0.61; Münster model) and 0.54 (95% CI, 0.51–0.56; ECST [European Carotid Surgery Trial] model), but calibration was inaccurate.33,34

The Vascular Study Group of New England model showed the highest discrimination values of 0.59 (95% CI, 0.56–0.62)25 (Figure 3). The calibration plot Vascular Study Group of New England model showed good concordance between predicted and observed risks in lower risk groups but overestimated risks in the high risk group.25

The OCER model showed C statistic of 0.58 (95% CI, 0.56–0.61) and the calibration plot of the OCER model showed good concordance across all risk groups.30 Most patients (71.7%) were in the lowest risk group with a predicted and observed risk of 3.4% and 3.3%, respectively.

In total, 508 (4.6%) patients had a predicted risk above 6% and 3167 patients (28.2%) above 4% (Figure 4). Of these 3167 patients, 1211 (38.2%) had ipsilateral stenosis of 50% to 79% and 1831 (57.8%) had ipsilateral stenosis of 80% to 99%. Calibration plots are provided in Figures I and II and Table IX in the Data Supplement.

Asymptomatic Patients

External validation in 14 772 patients who underwent CEA for asymptomatic carotid stenosis showed C statistics that varied between 0.49 and 0.58.24–38 Models that were developed in populations of asymptomatic patients had C statistics between 0.49 and 0.56.35–37 The OCER model had the highest discrimination value of 0.58 (95% CI, 0.56–0.59)30 (Figure 3). The calibration plot showed good concordance across all risk groups. Most patients (64.0%) were in the lowest risk group with a predicted and observed risk of 1.6% and 1.4%, respectively. The highest risk group of 859 (6.2%) patients had a predicted and observed risk of 3.0% and 3.7%, respectively. Calibration plots are provided in Figures I and II and Table IX in the Data Supplement.

In total, 306 (2.1%) patients had a predicted risk of above 3% and 5423 (36.0%) above 2% (Figure 4).

Sensitivity Analysis

Complete case analysis showed similar results (Table X in the Data Supplement).

Discussion

Our study compared the predictive performance of 17 risk models of procedural stroke or death after CEA in a representative contemporary setting. We found that the OCER model that included symptomatic status, diabetes, heart failure, and contralateral occlusion as predictors showed C statistic of 0.64 and good concordance between predicted and observed risks of procedural stroke or death after CEA. This model could, therefore, reliably inform patients and clinicians about expected procedural risks of CEA in symptomatic and asymptomatic patients. The model identified 508 (4.6%) symptomatic and 306 (2.1%) asymptomatic patients with procedural risks exceeding recommended thresholds of 6% and 3% for symptomatic and asymptomatic patients, respectively.

The OCER risk prediction model which showed in whom CEA can be performed with acceptable risk. Current procedural risk thresholds to consider CEA are 6% in symptomatic and 3% in asymptomatic patients and the OCER model only identified a small proportion of patients with procedural risks that exceeded these thresholds, but these procedural risks might be reduced in the future since the risk of stroke in medically treated patients has decreased.39 If the thresholds of procedural stroke or death are to be reduced to 4% in symptomatic and 2% in asymptomatic patients, the proportion of patients with predicted risks based on the OCER model exceeding these thresholds will increase to 28% of symptomatic and 36% of asymptomatic patients.

A previous external validation of the Carotid Stenosis Trialists’ Collaboration that compared prediction models of short-term outcome after CEA found poor discriminative performance.40 Their external validation cohort consisted of 4754 patients from the EVA-3S (Endarterectomy Versus Angioplasty in Patients With Severe Symptomatic Carotid Stenosis), SPACE (Stent-Protected Angioplasty Versus Carotid Endarterectomy), ICSS (International Carotid Stenting Study), and CREST (Carotid Revascularization Versus Stenting Trial) trials resulting in a more homogenous population as a result of patient selection.41–44 This might explain the poorer discriminative performance compared with our study.

Risks of procedural stroke or death also depend on the qualifying symptom and timing of CEA.45,46 Patients with ischemic strokes have higher risks compared with ocular symptoms and possibly hemispheric TIAs. Procedural risks are higher when CEA is performed within 48 hours after ischemic stroke.47 The risk of stroke recurrence is also high initially and decreases over time, reducing the benefit of CEA. The optimal timing of CEA should, therefore, balance the risk of recurrent events and procedural risks.

The beneficial effect is clear in symptomatic patients with 70% to 99% stenosis without near-occlusion and somewhat less clear in patients with 50% to 69% stenosis.48 These benefits have become less clear over time since medical therapy for stroke prevention has improved and the absolute gains that individual patients might receive from CEA is smaller. The Carotid Stenosis Risk score has been developed to predict the risk of ipsilateral stroke in patients with recently symptomatic carotid stenosis on medical therapy.49 The recently terminated ECST-2 re-evaluates the net benefit in symptomatic patients with moderate risk of stroke recurrence, ie. <20% 5-year risk (ISRCTN97744893) calculated with the Carotid Stenosis Risk score. Some predictors of the Carotid Stenosis Risk score overlap with predictors of models for procedural stroke or death, indicating that these predictors identify patient at high risk of stroke rather than selecting patients for the appropriate management strategy.

The absolute risk of a first ipsilateral stroke in patients with asymptomatic carotid stenosis in patients using medical preventive therapy is presumably low, but estimates are based on historic cohorts and lack preciseness due to small sample sizes.39,50,51 In addition, not all patients are using adequate medical preventive therapy.52 Stratification tools aiming to identify patients with a higher risk of ipsilateral stroke despite medical therapy have been developed but not validated in contemporary cohorts.53,54 The use of imaging to identify characteristics of plaques vulnerability might improve prediction but have not been included in established risk prediction models.55 Validated models of ipsilateral stroke risk in patients with medically treated carotid stenosis showing good predictive performance could be used in conjunction with the OCER model of procedural risks to determine who might benefit from carotid interventions.56

Strengths and Limitations

The present study has several strengths. We conducted a comprehensive literature search to identify existing prediction models according to a prespecified protocol. A large registry representative of contemporary clinical practice was used for validation. Missing data were limited for most variables and our findings were unaffected by missing data. We also included 30-day outcome events that occurred after discharge to estimate procedural hazards reliably.57,58 We performed additional analyses by symptomatic status to determine absolute risks of procedural stroke or death in those patients.

The present study also has several limitations. First, though data were collected prospectively, these were not collected primarily for the present analyses. We used proxies when a direct match between the predictors in the models and variables in the external validation cohort was not available, but proxies were not available for some predictors. This might have influenced predictive performance of the validated prediction models. We have, therefore, provided the linear predictor functions that we used for validation (Table IV in the Data Supplement). Second, some predictors were not available in the external validation cohort or could not be used due too many missing values (Table II in the Data Supplement). Third, some risk prediction models did not allow splitting predicted risks in deciles hampering a direct comparison of calibration plots. However, visual assessment of the calibration plots clearly showed the concordance between predicted and observed risks. Fourth, data on hospital and operator volume, possibly 2 of the most important determinants of procedural hazards,59,60 were not available in the external validation cohort and could, therefore, not be validated in one model that included annual surgeon volume.32 Fifth, it is unclear whether our findings are generalizable to patients with restenosis, tandem stenosis, or who received previous cervical radiation therapy.61,62 Sixth, the number of patients who were deemed ineligible for carotid intervention was not collected. Seventh, we were not able to validate some risk prediction models that were developed (partly) in the same data set.63–65

Implications for Practice and Future Research

Risk prediction models help inform patients about procedural hazards and might also contribute to the calculation of the net benefit of CEA in contemporary practice. The OCER model might be further refined by adding additional predictors, such as age, medical history, type of symptom, timing of CEA, and imaging characteristics, as well as by addressing identified shortcomings (in the statistical methods).

Future research will also determine which patients have the greatest reduction in absolute risk by undergoing CEA in contemporary practice weighting short-term procedural hazards against long-term stroke rates in unoperated patients and proportional reduction in nonperioperative stroke following successful CEA. Validation of established risk prediction models and assessment of predictive value of additional imaging characteristics to determine stroke risk in medically managed asymptomatic and (low-risk) symptomatic carotid stenosis is urgently needed.66 This together with the second CREST-2 (NCT02089217), the ECST-2 (ISRCTN97744893), and the ACTRIS (Asymptomatic Severe Atherosclerotic Carotid Artery Stenosis at Higher Than Average Risk of Ipsilateral Stroke; NCT02841098) will provide reliable evidence to guide clinical decision making.

Conclusions

This external validation study assessed the predictive performance of 17 models for procedural stroke or death after CEA. We found that the OCER model that included symptomatic status, diabetes, heart failure, and contralateral occlusion as predictors showed C statistic of 0.64 and good concordance between predicted and observed risks in the calibration plot. This model can be applied to symptomatic and asymptomatic patients and can reliably inform patients and clinicians about expected risks of procedural stroke or death of CEA. The OCER model might help focus CEA toward patients who benefit most from it.

Article Information

Acknowledgments

We are grateful to Randall R. DeMartino, MD, MS, (Division of Vascular and Endovascular Surgery, Mayo Clinic, Rochester) and Dan Neal MS, (Division of Vascular Surgery, University of Florida, Gainesville) for providing the original risk equation. We are grateful to Olena Seminog for help with the translation of 2 Russian articles and to Paul Sherliker for help with creating figures.

Drs Poorthuis, Schermerhorn, and de Borst designed the study. Dr Poorthuis designed the search strategy, performed literature searches and removed duplicates. Drs Poorthuis and Dansey screened titles and abstracts and assessed full-text articles and reference lists of included studies. Herings performed the statistical analyses supervised by Drs Poorthuis, Damen, and Greving. The article was drafted by Dr Poorthuis. All authors interpreted the data, contributed to revision and editing of the manuscript and approved the final version of the article for submission for publication.

Supplemental Materials

Online Tables I–X

Online Figures I and II

Risk calculator

References 24–38,40,53,63–65,67–126

Nonstandard Abbreviations and Acronyms

ACS-NSQIP

American College of Surgeons National Surgical Quality Improvement Program

ACTRIS

Asymptomatic Severe Atherosclerotic Carotid Artery Stenosis at Higher Than Average Risk of Ipsilateral Stroke

CEA

carotid endarterectomy

ECST

European Carotid Surgery Trial

OCER

Ontario Carotid Endarterectomy Registry

TIA

transient ischemic attack

Disclosures Dr Schermerhorn reports personal fees from Cook and personal fees from Philips outside the submitted work. The other authors report no conflicts.

Footnotes

The Data Supplement is available with this article at https://www.ahajournals.org/doi/suppl/10.1161/STROKEAHA.120.032527.

For Sources of Funding and Disclosures, see page 97.

Correspondence to: Gert J. de Borst, MD, PhD, Department of Vascular Surgery, Room G.04.129, University Medical Center, PO Box 85500, 3508 GA Utrecht, the Netherlands. Email

References

  • 1. Naylor AR, Ricco JB, de Borst GJ, Debus S, de Haro J, Halliday Aet al. Management of atherosclerotic carotid and vertebral artery disease: 2017 clinical practice guidelines of the european society for vascular surgery (ESVS).Eur J Vasc Endovasc Surg. 2018; 55:3–81.CrossrefMedlineGoogle Scholar
  • 2. Randomised trial of endarterectomy for recently symptomatic carotid stenosis: final results of the MRC European Carotid Surgery Trial (ECST).Lancet. 1998; 351:1379–1387.CrossrefMedlineGoogle Scholar
  • 3. Barnett HJ, Taylor DW, Eliasziw M, Fox AJ, Ferguson GG, Haynes RB, Rankin RN, Clagett GP, Hachinski VC, Sackett DLet al. Benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenosis. North American Symptomatic Carotid Endarterectomy Trial Collaborators.N Engl J Med. 1998; 339:1415–1425. doi: 10.1056/NEJM199811123392002CrossrefMedlineGoogle Scholar
  • 4. Mayberg MR, Wilson SE, Yatsu F, Weiss DG, Messina L, Hershey LA, Colling C, Eskridge J, Deykin D, Winn HR. Carotid endarterectomy and prevention of cerebral ischemia in symptomatic carotid stenosis. Veterans Affairs Cooperative Studies Program 309 Trialist Group.JAMA. 1991; 266:3289–3294.CrossrefMedlineGoogle Scholar
  • 5. Rothwell PM, Eliasziw M, Gutnikov SA, Fox AJ, Taylor DW, Mayberg MR, Warlow CP, Barnett HJ; Carotid Endarterectomy Trialists’ Collaboration. Analysis of pooled data from the randomised controlled trials of endarterectomy for symptomatic carotid stenosis.Lancet. 2003; 361:107–116. doi: 10.1016/s0140-6736(03)12228-3CrossrefMedlineGoogle Scholar
  • 6. Executive Committee for the Asymptomatic Carotid Atherosclerosis Study.Endarterectomy for asymptomatic carotid artery stenosis.JAMA. 1995; 273:1421–1428.CrossrefMedlineGoogle Scholar
  • 7. Halliday A, Harrison M, Hayter E, Kong X, Mansfield A, Marro J, Pan H, Peto R, Potter J, Rahimi Ket al; Asymptomatic Carotid Surgery Trial (ACST) Collaborative Group. 10-year stroke prevention after successful carotid endarterectomy for asymptomatic stenosis (ACST-1): a multicentre randomised trial.Lancet. 2010; 376:1074–1084. doi: 10.1016/S0140-6736(10)61197-XCrossrefMedlineGoogle Scholar
  • 8. Halliday A, Mansfield A, Marro J, Peto C, Peto R, Potter J, Thomas D; MRC Asymptomatic Carotid Surgery Trial (ACST) Collaborative Group. Prevention of disabling and fatal strokes by successful carotid endarterectomy in patients without recent neurological symptoms: randomised controlled trial.Lancet. 2004; 363:1491–1502. doi: 10.1016/S0140-6736(04)16146-1CrossrefMedlineGoogle Scholar
  • 9. Hobson RW, Weiss DG, Fields WS, Goldstone J, Moore WS, Towne JB, Wright CB. Efficacy of carotid endarterectomy for asymptomatic carotid stenosis. The Veterans Affairs Cooperative Study Group.N Engl J Med. 1993; 328:221–227. doi: 10.1056/NEJM199301283280401CrossrefMedlineGoogle Scholar
  • 10. Ricotta JJ, Aburahma A, Ascher E, Eskandari M, Faries P, Lal BK; Society for Vascular Surgery. Updated Society for Vascular Surgery guidelines for management of extracranial carotid disease.J Vasc Surg. 2011; 54:e1–e31. doi: 10.1016/j.jvs.2011.07.031CrossrefMedlineGoogle Scholar
  • 11. Volkers EJ, Algra A, Kappelle LJ, Greving JP. Prediction models for clinical outcome after a carotid revascularisation procedure: a systematic review.Eur Stroke J. 2018; 3:57–65. doi: 10.1177/2396987317739122CrossrefGoogle Scholar
  • 12. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration.PLoS Med. 2009; 6:e1000100. doi: 10.1371/journal.pmed.1000100CrossrefMedlineGoogle Scholar
  • 13. Moons KG, de Groot JA, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG, Reitsma JB, Collins GS. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist.PLoS Med. 2014; 11:e1001744. doi: 10.1371/journal.pmed.1001744CrossrefMedlineGoogle Scholar
  • 14. Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, Reitsma JB, Kleijnen J, Mallett S; PROBAST Group†. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies.Ann Intern Med. 2019; 170:51–58. doi: 10.7326/M18-1376CrossrefMedlineGoogle Scholar
  • 15. Moons KGM, Wolff RF, Riley RD, Whiting PF, Westwood M, Collins GS, Reitsma JB, Kleijnen J, Mallett S. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration.Ann Intern Med. 2019; 170:W1–W33. doi: 10.7326/M18-1377CrossrefMedlineGoogle Scholar
  • 16. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement.Ann Intern Med. 2015; 162:55–63. doi: 10.7326/M14-0697CrossrefMedlineGoogle Scholar
  • 17. Ingraham AM, Richards KE, Hall BL, Ko CY. Quality improvement in surgery: the American College of Surgeons National Surgical Quality Improvement Program approach.Adv Surg. 2010; 44:251–267. doi: 10.1016/j.yasu.2010.05.003CrossrefGoogle Scholar
  • 18. Bensley RP, Yoshida S, Lo RC, Fokkema M, Hamdan AD, Wyers MC, Chaikof EL, Schermerhorn ML. Accuracy of administrative data versus clinical data to evaluate carotid endarterectomy and carotid stenting.J Vasc Surg. 2013; 58:412–419. doi: 10.1016/j.jvs.2013.01.010CrossrefMedlineGoogle Scholar
  • 19. White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice.Stat Med. 2011; 30:377–399. doi: 10.1002/sim.4067CrossrefMedlineGoogle Scholar
  • 20. Bodner TE. What improves with increased missing data imputations?Struct Equ Modeling. 2008; 15:651–675.CrossrefGoogle Scholar
  • 21. Marshall A, Altman DG, Holder RL, Royston P. Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines.BMC Med Res Methodol. 2009; 9:57. doi: 10.1186/1471-2288-9-57CrossrefMedlineGoogle Scholar
  • 22. Rubin DB. Inference and missing data.Biometrika. 1976; 63:581–592.CrossrefGoogle Scholar
  • 23. Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. Springer-Verlag; 2009.CrossrefGoogle Scholar
  • 24. Sridharan ND, Chaer RA, Wu BB, Eslami MH, Makaroun MS, Avgerinos ED. An accumulated deficits model predicts perioperative and long-term adverse events after carotid endarterectomy.Ann Vasc Surg. 2018; 46:97–103. doi: 10.1016/j.avsg.2017.06.150Google Scholar
  • 25. Eslami MH, Rybin D, Doros G, Farber A. An externally validated robust risk predictive model of adverse outcomes after carotid endarterectomy.J Vasc Surg. 2016; 63:345–354. doi: 10.1016/j.jvs.2015.09.003CrossrefMedlineGoogle Scholar
  • 26. Chaudhry SA, Afzal MR, Kassab A, Hussain SI, Qureshi AI. A new risk index for predicting outcomes among patients undergoing carotid endarterectomy in large administrative data sets.J Stroke Cerebrovasc Dis. 2016; 25:1978–1983. doi: 10.1016/j.jstrokecerebrovasdis.2016.01.023Google Scholar
  • 27. Wimmer NJ, Spertus JA, Kennedy KF, Anderson HV, Curtis JP, Weintraub WS, Singh M, Rumsfeld JS, Masoudi FA, Yeh RW. Clinical prediction model suitable for assessing hospital quality for patients undergoing carotid endarterectomy.J Am Heart Assoc. 2014; 3:e000728. doi: 10.1161/JAHA.113.000728LinkGoogle Scholar
  • 28. Bekelis K, Bakhoum SF, Desai A, Mackenzie TA, Goodney P, Labropoulos N. A risk factor-based predictive model of outcomes in carotid endarterectomy: the National Surgical Quality Improvement Program 2005-2010.Stroke. 2013; 44:1085–1090. doi: 10.1161/STROKEAHA.111.674358LinkGoogle Scholar
  • 29. Goodney PP, Likosky DS, Cronenwett JL; Vascular Study Group of Northern New England. Factors associated with stroke or death after carotid endarterectomy in Northern New England.J Vasc Surg. 2008; 48:1139–1145. doi: 10.1016/j.jvs.2008.05.013CrossrefMedlineGoogle Scholar
  • 30. Tu JV, Wang H, Bowyer B, Green L, Fang J, Kucey D; Participants in the Ontario Carotid Endarterectomy Registry. Risk factors for death or stroke after carotid endarterectomy: observations from the Ontario Carotid Endarterectomy Registry.Stroke. 2003; 34:2568–2573. doi: 10.1161/01.STR.0000092491.45227.0FLinkGoogle Scholar
  • 31. Kuhan G, Gardiner ED, Abidia AF, Chetter IC, Renwick PM, Johnson BF, Wilkinson AR, McCollum PT. Risk modelling study for carotid endarterectomy.Br J Surg. 2001; 88:1590–1594. doi: 10.1046/j.0007-1323.2001.01938.xGoogle Scholar
  • 32. Kucey DS, Bowyer B, Iron K, Austin P, Anderson G, Tu JV. Determinants of outcome after carotid endarterectomy.J Vasc Surg. 1998; 28:1051–1058. doi: 10.1016/s0741-5214(98)70031-xGoogle Scholar
  • 33. Stavrinou P, Bergmann J, Palkowiz S, Goldbrunner R, Rieger B. Identifying risk factors and proposing a risk-profile scoring scale for perioperative ischemic complications in carotid endarterectomies.J Neurosurg Sci. 2016; 60:11–17.Google Scholar
  • 34. Rothwell PM, Warlow CP. Prediction of benefit from carotid endarterectomy in individual patients: a risk-modelling study. European Carotid Surgery Trialists’ Collaborative Group.Lancet. 1999; 353:2105–2110. doi: 10.1016/s0140-6736(98)11415-0CrossrefMedlineGoogle Scholar
  • 35. DeMartino RR, Brooke BS, Neal D, Beck AW, Conrad MF, Arya S, Desai S, Aziz F, Ryan P, Cronenwett JLet al; Vascular Quality Initiative. Development of a validated model to predict 30-day stroke and 1-year survival after carotid endarterectomy for asymptomatic stenosis using the Vascular Quality Initiative.J Vasc Surg. 2017; 66:433–444. doi: 10.1016/j.jvs.2017.03.427Google Scholar
  • 36. Gupta PK, Ramanan B, Mactaggart JN, Sundaram A, Fang X, Gupta H, Johanning JM, Pipinos II. Risk index for predicting perioperative stroke, myocardial infarction, or death risk in asymptomatic patients undergoing carotid endarterectomy.J Vasc Surg. 2013; 57:318–326. doi: 10.1016/j.jvs.2012.08.116CrossrefMedlineGoogle Scholar
  • 37. Calvillo-King L, Xuan L, Zhang S, Tuhrim S, Halm EA. Predicting risk of perioperative death and stroke after carotid endarterectomy in asymptomatic patients: derivation and validation of a clinical risk score.Stroke. 2010; 41:2786–2794. doi: 10.1161/STROKEAHA.110.599019LinkGoogle Scholar
  • 38. Matsen SL, Perler BA, Chang DC. A preliminary clinical scale to predict the risk of in-hospital death after carotid endarterectomy.J Vasc Surg. 2005; 42:861–8. doi: 10.1016/j.jvs.2005.08.005Google Scholar
  • 39. Pini R, Faggioli G, Vacirca A, Cacioppa LM, Gallitto E, Gargiulo M, Stella A. The fate of asymptomatic severe carotid stenosis in the era of best medical therapy.Brain Inj. 2017; 31:1711–1717. doi: 10.1080/02699052.2017.1346285Google Scholar
  • 40. Volkers EJ, Algra A, Kappelle LJ, Jansen O, Howard G, Hendrikse J, Halliday A, Gregson J, Fraedrich G, Eckstein HHet al; Carotid Stenosis Trialists’ Collaboration. Prediction models for clinical outcome after a carotid revascularization procedure.Stroke. 2018; 49:1880–1885. doi: 10.1161/STROKEAHA.117.020486LinkGoogle Scholar
  • 41. Bonati LH, Dobson J, Featherstone RL, Ederle J, van der Worp HB, de Borst GJ, Mali WP, Beard JD, Cleveland T, Engelter STet al; International Carotid Stenting Study investigators. Long-term outcomes after stenting versus endarterectomy for treatment of symptomatic carotid stenosis: the International Carotid Stenting Study (ICSS) randomised trial.Lancet. 2015; 385:529–538. doi: 10.1016/S0140-6736(14)61184-3CrossrefMedlineGoogle Scholar
  • 42. Ringleb PA, Allenberg J, Brückmann H, Eckstein HH, Fraedrich G, Hartmann Met al. 30 day results from the SPACE trial of stent-protected angioplasty versus carotid endarterectomy in symptomatic patients: a randomised non-inferiority trial.Lancet. 2006; 368:1239–1247.CrossrefMedlineGoogle Scholar
  • 43. Brott TG, Hobson RW, Howard G, Roubin GS, Clark WM, Brooks W, Mackey A, Hill MD, Leimgruber PP, Sheffet AJet al; CREST Investigators. Stenting versus endarterectomy for treatment of carotid-artery stenosis.N Engl J Med. 2010; 363:11–23. doi: 10.1056/NEJMoa0912321CrossrefMedlineGoogle Scholar
  • 44. Mas JL, Chatellier G, Beyssen B, Branchereau A, Moulin T, Becquemin JP, Larrue V, Lièvre M, Leys D, Bonneville JFet al; EVA-3S Investigators. Endarterectomy versus stenting in patients with symptomatic severe carotid stenosis.N Engl J Med. 2006; 355:1660–1671. doi: 10.1056/NEJMoa061752CrossrefMedlineGoogle Scholar
  • 45. Pothof AB, Zwanenburg ES, Deery SE, O’Donnell TFX, de Borst GJ, Schermerhorn ML. An update on the incidence of perioperative outcomes after carotid endarterectomy, stratified by type of preprocedural neurologic symptom.J Vasc Surg. 2018; 67:785–792. doi: 10.1016/j.jvs.2017.07.132Google Scholar
  • 46. Bond R, Rerkasem K, Rothwell PM. Systematic review of the risks of carotid endarterectomy in relation to the clinical indication for and timing of surgery.Stroke. 2003; 34:2290–2301. doi: 10.1161/01.STR.0000087785.01407.CCLinkGoogle Scholar
  • 47. Savardekar AR, Narayan V, Patra DP, Spetzler RF, Sun H. Timing of carotid endarterectomy for symptomatic carotid stenosis: a snapshot of current trends and systematic review of literature on changing paradigm towards early surgery.Neurosurgery. 2019; 85:E214–E225. doi: 10.1093/neuros/nyy557Google Scholar
  • 48. Orrapin S, Rerkasem K. Carotid endarterectomy for symptomatic carotid stenosis.Cochrane Database Syst Rev. 2017; 6:CD001081. doi: 10.1002/14651858.CD001081.pub3MedlineGoogle Scholar
  • 49. Rothwell PM, Mehta Z, Howard SC, Gutnikov SA, Warlow CP. Treating individuals 3: from subgroups to individuals: general principles and the example of carotid endarterectomy.Lancet. 2005; 365:256–265. doi: 10.1016/S0140-6736(05)17746-0CrossrefMedlineGoogle Scholar
  • 50. Marquardt L, Geraghty OC, Mehta Z, Rothwell PM. Low risk of ipsilateral stroke in patients with asymptomatic carotid stenosis on best medical treatment: a prospective, population-based study.Stroke. 2010; 41:e11–e17. doi: 10.1161/STROKEAHA.109.561837LinkGoogle Scholar
  • 51. Hadar N, Raman G, Moorthy D, O’Donnell TF, Thaler DE, Feldmann E, Lau J, Kitsios GD, Dahabreh IJ. Asymptomatic carotid artery stenosis treated with medical therapy alone: temporal trends and implications for risk assessment and the design of future studies.Cerebrovasc Dis. 2014; 38:163–173. doi: 10.1159/000365206CrossrefMedlineGoogle Scholar
  • 52. Merwick Á, Albers GW, Arsava EM, Ay H, Calvet D, Coutts SB, Cucchiara BL, Demchuk AM, Giles MF, Mas JLet al. Reduction in early stroke risk in carotid stenosis with transient ischemic attack associated with statin treatment.Stroke. 2013; 44:2814–2820. doi: 10.1161/STROKEAHA.113.001576LinkGoogle Scholar
  • 53. Burke JF, Morgenstern LB, Hayward RA. Can risk modelling improve treatment decisions in asymptomatic carotid stenosis?BMC Neurol. 2019; 19:295. doi: 10.1186/s12883-019-1528-7Google Scholar
  • 54. Nicolaides AN, Kakkos SK, Kyriacou E, Griffin M, Sabetai M, Thomas DJ, Tegos T, Geroulakos G, Labropoulos N, Doré CJet al; Asymptomatic Carotid Stenosis and Risk of Stroke (ACSRS) Study Group. Asymptomatic internal carotid artery stenosis and cerebrovascular risk stratification.J Vasc Surg. 2010; 52:1486–1496. doi: 10.1016/j.jvs.2010.07.021CrossrefMedlineGoogle Scholar
  • 55. Altaf N, Kandiyil N, Hosseini A, Mehta R, MacSweeney S, Auer D. Risk factors associated with cerebrovascular recurrence in symptomatic carotid disease: a comparative study of carotid plaque morphology, microemboli assessment and the European Carotid Surgery Trial risk model.J Am Heart Assoc. 2014; 3:e000173. doi: 10.1161/JAHA.113.000173LinkGoogle Scholar
  • 56. Müller MD, von Felten S, Algra A, Becquemin JP, Bulbulia R, Calvet D, Eckstein HH, Fraedrich G, Halliday A, Hendrikse Jet al. Secular trends in procedural stroke or death risks of stenting versus endarterectomy for symptomatic carotid stenosis.Circ Cardiovasc Interv. 2019; 12:e007870. doi: 10.1161/CIRCINTERVENTIONS.119.007870LinkGoogle Scholar
  • 57. Poorthuis MHF, Bulbulia R, Morris DR, Pan H, Rothwell PM, Algra A, Becquemin JP, Bonati LH, Brott TG, Brown MMet al; Carotid Stenosis Trialists’ Collaboration. Timing of procedural stroke and death in asymptomatic patients undergoing carotid endarterectomy: individual patient analysis from four RCTs.Br J Surg. 2020; 107:662–668. doi: 10.1002/bjs.11441Google Scholar
  • 58. Liang P, Solomon Y, Swerdlow NJ, Li C, Varkevisser RRB, de Guerre LEVM, Schermerhorn ML. In-hospital outcomes alone underestimate rates of 30-day major adverse events after carotid artery stenting.J Vasc Surg. 2020; 71:1233–1241. doi: 10.1016/j.jvs.2019.06.201Google Scholar
  • 59. Poorthuis MHF, Brand EC, Halliday A, Bulbulia R, Bots ML, de Borst GJ. High operator and hospital volume are associated with a decreased risk of death and stroke after carotid revascularization: a systematic review and meta-analysis.Ann Surg. 2019; 269:631–641. doi: 10.1097/SLA.0000000000002880Google Scholar
  • 60. Poorthuis MHF, Brand EC, Halliday A, Bulbulia R, Schermerhorn ML, Bots ML, de Borst GJ. A systematic review and meta-analysis of complication rates after carotid procedures performed by different specialties.J Vasc Surg. 2020; 72:335–343. doi: 10.1016/j.jvs.2019.11.061Google Scholar
  • 61. Fokkema M, den Hartog AG, Bots ML, van der Tweel I, Moll FL, de Borst GJ. Stenting versus surgery in patients with carotid stenosis after previous cervical radiation therapy: systematic review and meta-analysis.Stroke. 2012; 43:793–801. doi: 10.1161/STROKEAHA.111.633743LinkGoogle Scholar
  • 62. Fokkema M, Vrijenhoek JE, Den Ruijter HM, Groenwold RH, Schermerhorn ML, Bots ML, Pasterkamp G, Moll FL, De Borst GJ; TREAT CARE Study Group. Stenting versus endarterectomy for restenosis following prior ipsilateral carotid endarterectomy: an individual patient data meta-analysis.Ann Surg. 2015; 261:598–604. doi: 10.1097/SLA.0000000000000799CrossrefMedlineGoogle Scholar
  • 63. Bennett KM, Hoch JR, Scarborough JE. Predictors of 30-day postoperative major adverse clinical events after carotid artery stenting: an analysis of the procedure-targeted American College of Surgeons National Surgical Quality Improvement Program.J Vasc Surg. 2017; 66:1093–1099. doi: 10.1016/j.jvs.2017.04.039Google Scholar
  • 64. Bennett KM, Scarborough JE, Shortell CK. Predictors of 30-day postoperative stroke or death after carotid endarterectomy using the 2012 carotid endarterectomy-targeted American College of Surgeons National Surgical Quality Improvement Program database.J Vasc Surg. 2015; 61:103–111. doi: 10.1016/j.jvs.2014.05.100CrossrefMedlineGoogle Scholar
  • 65. Dasenbrock HH, Smith TR, Gormley WB, Castlen JP, Patel NJ, Frerichs KU, Aziz-Sultan MA, Du R. Predictive score of adverse events after carotid endarterectomy: the NSQIP registry carotid endarterectomy scale.J Am Heart Assoc. 2019; 8:e013412. doi: 10.1161/JAHA.119.013412LinkGoogle Scholar
  • 66. Poorthuis M, Morris D, Gaba K, Howard D, de Borst G, Bulbulia Ret al. Contemporary stroke risks of patients with asymptomatic carotid stenosis: design and characteristics of a large prospective cohort study.Eur J Vasc Endovasc Surg. 2019; 58:e854.Google Scholar
  • 67. AbuRahma AF, DerDerian T, Hariri N, Adams E, AbuRahma J, Dean LS, Nanjundappa A, Stone PA. Anatomical and technical predictors of perioperative clinical outcomes after carotid artery stenting.J Vasc Surg. 2017; 66:423–432. doi: 10.1016/j.jvs.2017.02.057Google Scholar
  • 68. Arhuidese IJ, Nejim B, Chavali S, Locham S, Obeid T, Hicks CW, Malas MB. Endarterectomy versus stenting in patients with prior ipsilateral carotid artery stenting.J Vasc Surg. 2017; 65:1418–1428. doi: 10.1016/j.jvs.2016.11.041CrossrefMedlineGoogle Scholar
  • 69. Arif S, Wojtasik J, Dziewierz A, Bartuś K, Dudek D, Bartuś S. Long-term mortality and follow-up after carotid artery stenting.Hippokratia. 2016; 20:204–208.Google Scholar
  • 70. Basic J, Assadian A, Strassegger J, Senekowitsch C, Wickenhauser G, Koulas S, Waldhör T, Duschek N. Degree of contralateral carotid stenosis improves preoperative risk stratification of patients with asymptomatic ipsilateral carotid stenosis.J Vasc Surg. 2016; 63:82–8.e2. doi: 10.1016/j.jvs.2015.08.067Google Scholar
  • 71. Carmo M, Barbetta I, Bissacco D, Trimarchi S, Catanese V, Bonzini M, Bonardelli S, Settembrini PG. Development and validation of a score to predict life expectancy after carotid endarterectomy in asymptomatic patients.J Vasc Surg. 2018; 67:175–182. doi: 10.1016/j.jvs.2017.05.107Google Scholar
  • 72. Clouse WD, Boitano LT, Ergul EA, Kashyap VS, Malas MB, Goodney PP, Patel VI, Conrad MF. Contralateral occlusion and concomitant procedures drive risk of non-ipsilateral stroke after carotid endarterectomy.Eur J Vasc Endovasc Surg. 2019; 57:619–625. doi: 10.1016/j.ejvs.2018.11.009Google Scholar
  • 73. Dakour-Aridi H, Faateh M, Kuo PL, Zarkowsky DS, Beck A, Malas MB. The Vascular Quality Initiative 30-day stroke/death risk score calculator after transfemoral carotid artery stenting.J Vasc Surg. 2020; 71:526–534. doi: 10.1016/j.jvs.2019.05.051Google Scholar
  • 74. de Waard DD, de Vries EE, Huibers AE, Arnold MM, Nederkoorn PJ, van Dijk LC, van der Lugt A, Koudstaal PJ, Bonati LH, Brown MMet al. A Clinical validation study of anatomical risk scoring for procedural stroke in patients treated by Carotid Artery Stenting in the International Carotid Stenting Study.Eur J Vasc Endovasc Surg. 2019; 58:664–670. doi: 10.1016/j.ejvs.2019.04.035Google Scholar
  • 75. Doig D, Turner EL, Dobson J, Featherstone RL, Lo RT, Gaines PA, Macdonald S, Bonati LH, Clifton A, Brown MM; ICSS Investigators. Predictors of stroke, myocardial infarction or death within 30 days of carotid artery stenting: results from the International Carotid Stenting Study.Eur J Vasc Endovasc Surg. 2016; 51:327–334. doi: 10.1016/j.ejvs.2015.08.013CrossrefMedlineGoogle Scholar
  • 76. Donald GW, Ghaffarian AA, Isaac F, Kraiss LW, Griffin CL, Smith BK, Sarfati MR, Beckstrom JL, Brooke BS. Preoperative frailty assessment predicts loss of independence after vascular surgery.J Vasc Surg. 2018; 68:1382–1389. doi: 10.1016/j.jvs.2018.02.044Google Scholar
  • 77. Dua A, Romanelli M, Upchurch GR, Pan J, Hood D, Hodgson KJ, Desai SS. Predictors of poor outcome after carotid intervention.J Vasc Surg. 2016; 64:663–670. doi: 10.1016/j.jvs.2016.03.428CrossrefMedlineGoogle Scholar
  • 78. Ehlert BA, Najafian A, Orion KC, Malas MB, Black JH, Abularrage CJ. Validation of a modified Frailty Index to predict mortality in vascular surgery patients.J Vasc Surg. 2016; 63:1595–1601. doi: 10.1016/j.jvs.2015.12.023Google Scholar
  • 79. Eslami MH, Saadeddin Z, Farber A, Fish L, Avgerinos ED, Makaroun MS. External validation of the Vascular Study Group of New England carotid endarterectomy risk predictive model using an independent U.S. national surgical database.J Vasc Surg. 2020; 71:1954–1963. doi: 10.1016/j.jvs.2019.04.495Google Scholar
  • 80. Garzon-Muvdi T, Yang W, Rong X, Caplan JM, Ye X, Colby GP, Coon AL, Tamargo RJ, Huang J. Restenosis after carotid endarterectomy: insight into risk factors and modification of postoperative management.World Neurosurg. 2016; 89:159–167. doi: 10.1016/j.wneu.2016.01.028Google Scholar
  • 81. Gavrilenko AV, Kravchenko AA, Kuklin AV, Fomina VV. [Prediction and risk factors of perioperative neurological complications in patients with internal carotid artery stenosis].Khirurgiia. 2017; 10:109–112. doi: 10.17116/hirurgia201710109-112Google Scholar
  • 82. Hicks CW, Nejim B, Locham S, Aridi HD, Schermerhorn ML, Malas MB. Association between Medicare high-risk criteria and outcomes after carotid revascularization procedures.J Vasc Surg. 2018; 67:1752–1761. doi: 10.1016/j.jvs.2017.10.066Google Scholar
  • 83. Hung CS, Lin MS, Chen YH, Huang CC, Li HY, Kao HL. Prognostic factors for neurologic outcome in patients with carotid artery stenting.Acta Cardiol Sin. 2016; 32:205–214. doi: 10.6515/acs20150119hGoogle Scholar
  • 84. Keyhani S, Madden E, Cheng EM, Bravata DM, Halm E, Austin PC, Ghasemiesfe M, Abraham AS, Zhang AJ, Johanning JM. Risk prediction tools to improve patient selection for carotid endarterectomy among patients with asymptomatic carotid stenosis.JAMA Surg. 2019; 154:336–344. doi: 10.1001/jamasurg.2018.5119Google Scholar
  • 85. Luebke T, Brunkwall J. Impact of real-world adherence with best medical treatment on cost-effectiveness of carotid endarterectomy for asymptomatic carotid artery stenosis.Ann Vasc Surg. 2016; 30:236–247. doi: 10.1016/j.avsg.2015.06.098Google Scholar
  • 86. Luebke T, Brunkwall J. Development of a microsimulation model to predict stroke and long-term mortality in adherent and nonadherent medically managed and surgically treated octogenarians with asymptomatic significant carotid artery stenosis.World Neurosurg. 2016; 92:513–520. doi: 10.1016/j.wneu.2016.05.034Google Scholar
  • 87. Moses DA, Johnston LE, Tracci MC, Robinson WP, Cherry KJ, Kern JA, Upchurch GR. Estimating risk of adverse cardiac event after vascular surgery using currently available online calculators.J Vasc Surg. 2018; 67:272–278. doi: 10.1016/j.jvs.2017.06.105Google Scholar
  • 88. Nejim B, Obeid T, Arhuidese I, Hicks C, Wang S, Canner J, Malas M. Predictors of perioperative outcomes after carotid revascularization.J Surg Res. 2016; 204:267–273. doi: 10.1016/j.jss.2016.04.074CrossrefMedlineGoogle Scholar
  • 89. Obeid T, Arhuidese I, Gaidry A, Qazi U, Abularrage C, Goodney P, Cronenwett J, Malas M. Beta-blocker use is associated with lower stroke and death after carotid artery stenting.J Vasc Surg. 2016; 63:363–369. doi: 10.1016/j.jvs.2015.08.108Google Scholar
  • 90. Oksala NKJ, Lindström I, Khan N, Pihlajaniemi VJ, Lyytikäinen LP, Pienimäki JP, Hernesniemi J. Pre-operative masseter area is an independent predictor of long-term survival after carotid endarterectomy.Eur J Vasc Endovasc Surg. 2019; 57:331–338. doi: 10.1016/j.ejvs.2018.11.011Google Scholar
  • 91. Saedon M, Saratzis A, Lee RWS, Hutchinson CE, Imray CHE, Singer DRJ. Registry report on prediction by Pocock cardiovascular score of cerebral microemboli acutely following carotid endarterectomy.Stroke Vasc Neurol. 2018; 3:147–152. doi: 10.1136/svn-2017-000116Google Scholar
  • 92. Stangenberg L, Curran T, Shuja F, Rosenberg R, Mahmood F, Schermerhorn ML. Development of a risk prediction model for transfusion in carotid endarterectomy and demonstration of cost-saving potential by avoidance of “type and screen”.J Vasc Surg. 2016; 64:1711–1718. doi: 10.1016/j.jvs.2016.04.059Google Scholar
  • 93. Tanashian MM, Medvedev RB, Evdokimenko AN, Gemdzhian ÉG, Ckrylev SI, Lagoda OV, Krotenkova MV, Suslin AS. [Prediction of ischaemic lesions of the brain in reconstructive operations on internal carotid arteries].Angiol Sosud Khir. 2017; 23:59–66.Google Scholar
  • 94. Tanaskovic S, Radak D, Aleksic N, Calija B, Maravic-Stojkovic V, Nenezic D, Ilijevski N, Popov P, Vucurevic G, Babic Set al. Scoring system to predict early carotid restenosis after eversion endarterectomy by analysis of inflammatory markers.J Vasc Surg. 2018; 68:118–127. doi: 10.1016/j.jvs.2017.09.054Google Scholar
  • 95. Vatan MB, Acar BA, Aksoy M, Can Y, Varim C, Agac MT, Kilic H, Tatli E, Gunduz H, Akdemir R. Predictors of periprocedural complications of carotid artery stenting - a multivariate analysis of a single-centre experience.Vasa. 2016; 45:387–393. doi: 10.1024/0301-1526/a000550Google Scholar
  • 96. Vinogradov RA, Zebelyan AA. [Risk stratification in carotid artery stenting].Khirurgiia. 2018; 2:93–95. doi: 10.17116/hirurgia2018293-95Google Scholar
  • 97. Yamauchi K, Enomoto Y, Otani K, Egashira Y, Iwama T. Prediction of hyperperfusion phenomenon after carotid artery stenting and carotid angioplasty using quantitative DSA with cerebral circulation time imaging.J Neurointerv Surg. 2018; 10:576–579. doi: 10.1136/neurintsurg-2017-013259Google Scholar
  • 98. Zapata-Arriaza E, Moniche F, González A, Bustamante A, Escudero-Martínez I, De la Torre Laviana FJ, Prieto M, Mancha F, Montaner J. Predictors of restenosis following carotid angioplasty and stenting.Stroke. 2016; 47:2144–2147. doi: 10.1161/STROKEAHA.116.012650LinkGoogle Scholar
  • 99. Zhou H, Shen L, Wei F, Shuai J. Predicting the risk of stroke in chinese internal carotid artery stenosis patients underwent carotid artery stenting: validation and improvement of siena carotid artery stenting risk score.J Stroke Cerebrovasc Dis. 2019; 28:104369. doi: 10.1016/j.jstrokecerebrovasdis.2019.104369Google Scholar
  • 100. Ackerstaff RG, Moons KG, van de Vlasakker CJ, Moll FL, Vermeulen FE, Algra A, Spencer MP. Association of intraoperative transcranial doppler monitoring variables with stroke from carotid endarterectomy.Stroke. 2000; 31:1817–1823. doi: 10.1161/01.str.31.8.1817LinkGoogle Scholar
  • 101. Ackerstaff RG, Suttorp MJ, van den Berg JC, Overtoom TT, Vos JA, Bal ET, Zanen P; Antonius Carotid Endarterectomy, Angioplasty, and Stenting Study Group. Prediction of early cerebral outcome by transcranial Doppler monitoring in carotid bifurcation angioplasty and stenting.J Vasc Surg. 2005; 41:618–624. doi: 10.1016/j.jvs.2005.01.034CrossrefMedlineGoogle Scholar
  • 102. Alcocer F, Mujib M, Lowman B, Patterson MA, Passman MA, Matthews TC, Jordan WD. Risk scoring system to predict 3-year survival in patients treated for asymptomatic carotid stenosis.J Vasc Surg. 2013; 57:1576–1580. doi: 10.1016/j.jvs.2012.12.033CrossrefMedlineGoogle Scholar
  • 103. Aronow HD, Gray WA, Ramee SR, Mishkel GJ, Schreiber TJ, Wang H. Predictors of neurological events associated with carotid artery stenting in high-surgical-risk patients: insights from the Cordis Carotid Stent Collaborative.Circ Cardiovasc Interv. 2010; 3:577–584. doi: 10.1161/CIRCINTERVENTIONS.110.950097LinkGoogle Scholar
  • 104. Bertges DJ, Goodney PP, Zhao Y, Schanzer A, Nolan BW, Likosky DS, Eldrup-Jorgensen J, Cronenwett JL; Vascular Study Group of New England. The Vascular Study Group of New England Cardiac Risk Index (VSG-CRI) predicts cardiac complications more accurately than the Revised Cardiac Risk Index in vascular surgery patients.J Vasc Surg. 2010; 52:674–83, 683. doi: 10.1016/j.jvs.2010.03.031CrossrefMedlineGoogle Scholar
  • 105. Bertges DJ, Neal D, Schanzer A, Scali ST, Goodney PP, Eldrup-Jorgensen J, Cronenwett JL; Vascular Quality Initiative. The Vascular Quality Initiative Cardiac Risk Index for prediction of myocardial infarction after vascular surgery.J Vasc Surg. 2016; 64:1411–1421. doi: 10.1016/j.jvs.2016.04.045CrossrefMedlineGoogle Scholar
  • 106. Cheng CA, Chien WC, Hsu CY, Lin HC, Chiu HW. Risk analysis of carotid stent from a population-based database in Taiwan.Medicine (Baltimore). 2016; 95:e4747. doi: 10.1097/MD.0000000000004747CrossrefMedlineGoogle Scholar
  • 107. Conrad MF, Kang J, Mukhopadhyay S, Patel VI, LaMuraglia GM, Cambria RP. A risk prediction model for determining appropriateness of CEA in patients with asymptomatic carotid artery stenosis.Ann Surg. 2013; 258:534–8. doi: 10.1097/SLA.0b013e3182a5007bCrossrefMedlineGoogle Scholar
  • 108. Fanous AA, Natarajan SK, Jowdy PK, Dumont TM, Mokin M, Yu J, Goldstein A, Wach MM, Budny JL, Hopkins LNet al. High-risk factors in symptomatic patients undergoing carotid artery stenting with distal protection: buffalo risk assessment scale (BRASS).Neurosurgery. 2015; 77:531–542. doi: 10.1227/NEU.0000000000000871Google Scholar
  • 109. Gates L, Botta R, Schlosser F, Goodney P, Fokkema M, Schermerhorn M, Sarac T, Indes J. Characteristics that define high risk in carotid endarterectomy from the Vascular Study Group of New England.J Vasc Surg. 2015; 62:929–936. doi: 10.1016/j.jvs.2015.04.398CrossrefMedlineGoogle Scholar
  • 110. Halm EA, Hannan EL, Rojas M, Tuhrim S, Riles TS, Rockman CB, Chassin MR. Clinical and operative predictors of outcomes of carotid endarterectomy.J Vasc Surg. 2005; 42:420–428. doi: 10.1016/j.jvs.2005.05.029CrossrefMedlineGoogle Scholar
  • 111. Hawkins BM, Kennedy KF, Giri J, Saltzman AJ, Rosenfield K, Drachman DE, White CJ, Spertus JA, Yeh RW. Pre-procedural risk quantification for carotid stenting using the CAS score: a report from the NCDR CARE Registry.J Am Coll Cardiol. 2012; 60:1617–1622. doi: 10.1016/j.jacc.2012.07.026CrossrefMedlineGoogle Scholar
  • 112. Hofmann R, Niessner A, Kypta A, Steinwender C, Kammler J, Kerschner K, Grund M, Leisch F, Huber K. Risk score for peri-interventional complications of carotid artery stenting.Stroke. 2006; 37:2557–2561. doi: 10.1161/01.STR.0000240688.81918.32LinkGoogle Scholar
  • 113. Hoke M, Ljubuncic E, Steinwender C, Huber K, Minar E, Koppensteiner R, Leisch F, Dick P, Kerschner K, Schillinger Met al. A validated risk score to predict outcomes after carotid stenting.Circ Cardiovasc Interv. 2012; 5:841–849. doi: 10.1161/CIRCINTERVENTIONS.112.972430LinkGoogle Scholar
  • 114. Liu J, Xu ZQ, Cui M, Li L, Cheng Y, Zhou HD. Assessing risk factors for major adverse cardiovascular and cerebrovascular events during the perioperative period of carotid angioplasty with stenting patients.Exp Ther Med. 2016; 12:1039–1047. doi: 10.3892/etm.2016.3360Google Scholar
  • 115. McCrory DC, Goldstein LB, Samsa GP, Oddone EZ, Landsman PB, Moore WS, Matchar DB. Predicting complications of carotid endarterectomy.Stroke. 1993; 24:1285–1291. doi: 10.1161/01.str.24.9.1285AbstractGoogle Scholar
  • 116. Morales-Gisbert SM, Zaragozá García JM, Plaza Martínez Á, Gómez Palonés FJ, Ortiz-Monzón E. Development of an individualized scoring system to predict mid-term survival after carotid endarterectomy.J Cardiovasc Surg (Torino). 2017; 58:535–542. doi: 10.23736/S0021-9509.16.08198-2Google Scholar
  • 117. Ruiz-Carmona C, Diaz-Duran C, Sevilla N, Cuadrado E, Clará A. Long-term survival after carotid endarterectomy in a population with a low coronary heart disease fatality: implications for decision making.Ann Vasc Surg. 2016; 36:153–158. doi: 10.1016/j.avsg.2016.01.059Google Scholar
  • 118. Setacci C, Chisci E, Setacci F, Iacoponi F, de Donato G, Rossi A. Siena carotid artery stenting score: a risk modelling study for individual patients.Stroke. 2010; 41:1259–1265. doi: 10.1161/STROKEAHA.110.578583LinkGoogle Scholar
  • 119. Stoner MC, Abbott WM, Wong DR, Hua HT, Lamuraglia GM, Kwolek CJ, Watkins MT, Agnihotri AK, Henderson WG, Khuri Set al. Defining the high-risk patient for carotid endarterectomy: an analysis of the prospective National Surgical Quality Improvement Program database.J Vasc Surg. 2006; 43:285–295. doi: 10.1016/j.jvs.2005.10.069CrossrefMedlineGoogle Scholar
  • 120. van Lammeren GW, Catanzariti LM, Peelen LM, de Vries JP, de Kleijn DP, Moll FL, Pasterkamp G, Bots ML. Clinical prediction rule to estimate the absolute 3-year risk of major cardiovascular events after carotid endarterectomy.Stroke. 2012; 43:1273–1278. doi: 10.1161/STROKEAHA.111.647958LinkGoogle Scholar
  • 121. Wallaert JB, Cronenwett JL, Bertges DJ, Schanzer A, Nolan BW, De Martino R, Eldrup-Jorgensen J, Goodney PP; Vascular Study Group of New England. Optimal selection of asymptomatic patients for carotid endarterectomy based on predicted 5-year survival.J Vasc Surg. 2013; 58:112–118. doi: 10.1016/j.jvs.2012.12.056CrossrefMedlineGoogle Scholar
  • 122. Wallaert JB, Newhall KA, Suckow BD, Brooke BS, Zhang M, Farber AE, Likosky D, Goodney PP; Vascular Quality Initiative. Relationships between 2-year survival, costs, and outcomes following carotid endarterectomy in asymptomatic patients in the vascular quality initiative.Ann Vasc Surg. 2016; 35:174–182. doi: 10.1016/j.avsg.2016.01.024Google Scholar
  • 123. Wimmer NJ, Yeh RW, Cutlip DE, Mauri L. Risk prediction for adverse events after carotid artery stenting in higher surgical risk patients.Stroke. 2012; 43:3218–3224. doi: 10.1161/STROKEAHA.112.673194LinkGoogle Scholar
  • 124. Melin AA, Schmid KK, Lynch TG, Pipinos II, Kappes S, Longo GM, Gupta PK, Johanning JM. Preoperative frailty Risk Analysis Index to stratify patients undergoing carotid endarterectomy.J Vasc Surg. 2015; 61:683–689. doi: 10.1016/j.jvs.2014.10.009Google Scholar
  • 125. Press MJ, Chassin MR, Wang J, Tuhrim S, Halm EA. Predicting medical and surgical complications of carotid endarterectomy: comparing the risk indexes.Arch Intern Med. 2006; 166:914–920. doi: 10.1001/archinte.166.8.914CrossrefMedlineGoogle Scholar
  • 126. Levey AS, Coresh J, Greene T, Stevens LA, Zhang YL, Hendriksen S, Kusek JW, Van Lente F; Chronic Kidney Disease Epidemiology Collaboration. Using standardized serum creatinine values in the modification of diet in renal disease study equation for estimating glomerular filtration rate.Ann Intern Med. 2006; 145:247–254. doi: 10.7326/0003-4819-145-4-200608150-00004CrossrefMedlineGoogle Scholar

eLetters(0)

eLetters should relate to an article recently published in the journal and are not a forum for providing unpublished data. Comments are reviewed for appropriate use of tone and language. Comments are not peer-reviewed. Acceptable comments are posted to the journal website only. Comments are not published in an issue and are not indexed in PubMed. Comments should be no longer than 500 words and will only be posted online. References are limited to 10. Authors of the article cited in the comment will be invited to reply, as appropriate.

Comments and feedback on AHA/ASA Scientific Statements and Guidelines should be directed to the AHA/ASA Manuscript Oversight Committee via its Correspondence page.