Population Risk Prediction Models for Incident Heart Failure
The prevalence of heart failure is expected to significantly rise unless high-risk patients are effectively screened and appropriate, cost-effective prevention interventions are implemented.
Methods and Results—
We performed a systematic review to evaluate the prediction characteristics of the published heart failure risk prediction models as of August 2014 using MEDLINE and EMBASE databases. Eligible studies reported the development, validation, or impact assessment of a model. Two investigators performed independent review to extract data on study design and characteristics, risk predictors, discrimination, calibration, and reclassification ability of models, as well as validation and impact analysis. We included 13 publications reporting on 28 heart failure risk prediction models. Models had acceptable-to-good discriminatory ability (c-statistics, >0.70) in the derivation sample. Calibration was less commonly assessed, but was acceptable when it was. Only 2 models were externally validated more than once, displaying modest-to-acceptable discrimination (c-statistics, 0.61–0.79). When assessed, novel blood and imaging markers modestly improved risk prediction. One model assessed the prediction properties in race-based subgroups, whereas 2 models evaluated sex-based subgroups. Impact analysis found none of the models recommended for use in any clinical practice guideline.
Incident heart failure risk prediction remains at an early stage. The discrimination ability of current models is acceptable in derivation data sets but most models have not been externally validated. It remains unclear which models are cost-effective and best suit population screening needs. The effects of models on clinical and preventative care requires further study.
Heart failure (HF) affects >5 million Americans.1 Unless effective preventive interventions are implemented, it is estimated that by 2030, HF prevalence will grow by 25% and annual costs of care from $21 to $53 billion.2 The prognosis of patients with HF remains poor with a projected 50% 5-year mortality, underscoring the importance of prevention.3 Prevention interventions, however, are unlikely to be cost-effective unless targeted toward reliably identified high-risk populations. Because of the epidemiological paradox, more HF cases occur in patients with the absence of any given individual risk factor, which makes it important to develop a more comprehensive risk assessment scheme than to rely solely on individual risk factors. Unfortunately, the best approach for quantifying overall incident HF risk in individuals remains unclear. In this study, we sought to systematically assess the performance characteristics and the use of the available HF risk models for predicting incident HF in the general population.
Clinical Perspective on p 447
We searched PubMed and EMBASE databases from January 1990 to August 2014 for HF risk prediction models published in English language. We used search terms related to HF and disease prediction (Appendix A in the Data Supplement) and reviewed the reference lists of studies and identified their citations through the ISI Web of Science for additional data. Eligible studies had to report a risk assessment model in a cohort study either by an equation or by a score. Two evaluators (J.B.E. and J.B.) independently identified articles and screened them for the inclusion criteria (Figure) by reviewing text and the Data Supplement. Disagreements were solved by consensus. We excluded studies for models developed in specific population (eg, postoperative patients, patients already diagnosed with HF, studies that reported measures of risk without information on model characteristics, and simulation studies). We extracted information on the area under the receiver-operating characteristic curve (AUC) or c-statistic, reclassification percentage, net reclassification improvement (NRI), and integrated discrimination improvement index (IDI).4
Bias, Clinical Usefulness, and External Validity
Risk for bias in models was assessed per Hayden et al5 recommendations, including 7 categories, such as participation (sampling bias), attrition (attrition bias), prognostic factor selection, prognostic factor measurement, outcome measurement (ascertainment bias), statistical analysis, and reporting model performance. Extent and methods of dealing with missing data, (Table in the Data Supplement), appropriateness of statistical methods, and clinical usefulness as a combination of clinical use and usability were assessed. Clinical use was assessed as whether predictive models proposed clinical scenarios or decision nodes linked to a risk category or threshold where the model may be useful for guiding diagnostic or therapeutic interventions. For usability, we noted whether a calculator or nomogram that enables easier knowledge translation was added. To examine generalizability and external validity, we evaluated whether models had been validated in an independent population in the original or a subsequent publication.
Data Extraction and Quality Assessment
Two reviewers (J.B.E. and J.B.) performed quality assessment and extracted information about design, setting, population characteristics, number of patients in the derivation and validation cohorts, number of outcomes of interest, number of candidate predictor variables, and the type of regression model used. For predictors, we specifically assessed for age, sex, hypertension, left ventricular hypertrophy, myocardial infarction, diabetes mellitus, valvular disease, and overweight/obesity, based on previously reported association of these factors with HF. For discriminative ability, information on the c-statistic, and for calibration, the difference between the observed and predicted rates, as well as the P value of the corresponding test statistic, were extracted. Data relevant to reclassification included NRI and IDI values, as well as their respective 95% confidence intervals and P values, when available. Given the heterogeneity in the type and number of metrics, the number and nature of risk factors included, and the study designs used, our analysis consisted mainly of a narrative synthesis of the evidence.
Effect of HF Risk Models
Effect of studies was identified through scanning the references of all publications and applying search strategies for impact studies proposed by Reilly and Evans,6 which combines the model’s acronym or name of the cohort or first author with specific search term combination (Appendix B in the Data Supplement). We searched relevant practice guidelines to assess the implementation of HF prediction models in countries or regions, in which such models were developed. We also targeted English-language guidelines compiled by a selection of professional organizations known to regularly issue recommendations for the management of HF, including the American College of Cardiology,7 the American Heart Association,7 Heart Failure Society of America,8 and the European Society of Cardiology.9
We screened 413 abstracts, retrieved 48 citations for full-text review, and selected 13 studies that described 28 HF prediction risk models (Figure). The characteristics of the risk prediction models are presented in Table 1.
|Reference and Cohort||Population||Risk Factors||nOutcome Follow-Up, y||Definition||Discrimination||Calibration||Validation||Reclassification|
|NRI, %||IDI, %|
|Kannel et al10 FHS risk score||US: whites||Age, LVH, HR, SBP, vital capacity, CHD, valve disease, DM, cardiomegaly||15 2674864||Framingham criteria||NR||NR||NR||NR||NR|
|Butler et al11Health ABC Score||US: whites and blacks||Age, CHD, smoking, SBP, HR, glucose, creatinine, albumin, and LVH||29352584–9||Physician diagnosis and treatment for HF||0.73 (0.72 after bootstrapping)||HL χ2=6.24 (P=0.621)||ApparentBootstrapping||NR||NR|
|Goyal et al12 Kaiser Permanente||US: mixed but mainly whites||Age, CHD, HTN, DM, AF, and valvular disease||359 94740019.5||ICD-9 code defined HF||Women: 0.886 and men: 0.877||NR||Apparent||NR||NR|
|Velagaleti et al13 FHSRisk score 1||US: mainly whites||Age, HTN, DM, CAD, AF, and valvular disease||2754959.4||Framingham criteria||0.84 (0.838 after jacknife)||HL χ2=2.98 (P=0.94)||ApparentJacknife||NR||NR|
|Velagaleti et al13 FHSRisk score 2||US: mainly whites||Age, HTN, DM, CAD, AF, valvular disease, BNP||2754959.4||Framingham criteria||0.85 (0.862 after jacknife)||HL χ2=3.48 (P=0.90)||ApparentJacknife||NR||NR|
|Velagaleti et al13 FHSRisk score 3||US: mainly whites||Age, HTN, DM, CAD, AF, BNP, albumin/creatinine ratio, valve disease||2754959.4||Framingham criteria||0.86 (0.862 after jacknife)||HL χ2=9.45 (P=0.40)||ApparentJacknife||NR||0.13 (P<0.002)|
|deFilippi, et al14 CHS score 1||US: mainly whites||Age, sex, race, SBP, ECG, glucose, CHD, smoking, creatinine, albumin, HR||42211279||Diagnosis+symptoms/signs+HF therapy||0.752||NR||NR||NR||NR|
|deFilippi, et al14 CHS risk score 2||US: mainly whites||Age, sex, race, SBP, ECG, glucose, CHD, status, creatinine, albumin, HR, cTnT||42211279||Diagnosis+symptoms/signs+HF therapy||0.767||NR||NR||0.043||0.026|
|Smith et al15MDCS score 1||Sweden: mainly whites||Age, sex, SBP, DBP, LDL, MI antihypertensives||518711214||ICD-8, 9, and 10 codes||0.815||NR||NR||NR||NR|
|Smith et al15MDCS score 2||Sweden: mainly whites||Age, sex, SBP, DBP, LDL, MI, antihypertensives, MR-proANP||518711214||ICD-8, 9, and 10 codes||0.824||NR||NR||14% (P=0.005)||0.03 (P<0.001)|
|Smith et al15MDCS score 3||Sweden: mainly whites||Age, sex, SBP, DBP, LDL, MI, antihypertensives, NT-proBNP||518711214||ICD-8, 9, and 10 codes||0.837||NR||NR||16% (P=0.003)||0.03 (P<0.001)|
|Smith et al15MDCS score 4||Sweden: mainly whites||Age, sex, SBP, DBP, LDL, MI, antihypertensives, CRP||518711214||ICD-8, 9, and 10 codes||0.823||NR||NR||7% (P=0.10)||0.01 (P=0.02)|
|Smith et al15MDCS score 5||Sweden: mainly whites||Age, sex, SBP, DBP, LDL, MI, antihypertensives, MR-proANP, NT-proBNP||518711214||ICD-8, 9, and 10 codes||0.838||NR||NR||17% (P=0.002)||0.03 (P<0.001)|
|Smith et al15MDCS score 6||Sweden: mainly whites||Age, sex, SBP, DBP, LDL, MI, antihypertensives NT-proBNP, CRP||518711214||ICD-8, 9, and 10 codes||0.842||NR||NR||19% (P=0.003)||0.04|
|Smith et al15MDCS score 7||Sweden: mainly whites||Age, sex, SBP, DBP, LDL, MI, antihypertensives, MR-proANP, NT-proBNP, CRP||518711214||ICD-8, 9, and 10 codes||0.842||NR||NR||22% (P<0.001)||0.05 (P<0.001))|
|Kalogeropoulos et al16 Health ABC score 2||US: whites and blacks||Age, CHD, smoking, SBP, HR, glucose, LVH, creatinine, albumin, IL-6||26103119.4||Diagnosis+clinical or echo findings+medical therapy||0.734 (0.706–0.763)||NR||NR||NR||NR|
|Kalogeropoulos et al16 Health ABC score 2||US: whites and blacks||Age, CHD, smoking, SBP, HR, glucose, LVH, creatinine, albumin, IL-6, TNF-α, CRP||26103119.4||Diagnosis+clinical or echo findings+medical therapy||0.737 (0.709–0.765)||NR||NR||NR||NR|
|Pfister et al17 PRoactive trial score||19 European countries: whites||Age, CKD, diuretic, MI use, HbA1c, DM, LDL, HR, R/LBBB, pioglitazone, microalbuminuria||49512332.9||Medical record diagnosis||0.75(0.751 after bootstrapping correction)||NR||Apparent||NR||NR|
|Kalogeropoulos et al18 CHS||US: mixed but 86.5% whites||Age, CHD, smoking, SBP, HR, glucose, creatinine, albumin LVH, echo score (EF, LA, and E/A) and NT-proBNP||3752 2865||Diagnosis+clinical or echo findings+medical therapy||0.777||NR||NR||10.6%||0.044|
|Kalogeropoulos et al18 CHS||US: mixed but 86.5% whites||Age, CHD, smoking, SBP, HR, glucose, creatinine, albumin and NT-proBNP||37522865||Diagnosis+clinical or echo findings+medical therapy||0.773||NR||NR||11.3%||0.045|
|Kalogeropoulos et al18 CHS||US: mixed but 86.5% whites||Age, CHD, smoking, SBP, HR, glucose, creatinine, albumin LVH, echo score (EF, LA, and E/A ratio) and NT-proBNP||37522865||Diagnosis+clinical or echo findings+medical therapy||0.789||NR||NR||16.3%||0.064|
|Wang et al19FHS||US: mainly whites||Age, sex, BMI, SBP, HTN, DM, smoking, total and HDL-cholesterol, AF, CVD, EKG LVH, murmur||342816211.3||Framingham criteria||0.846||NR||NR||NR||NR|
|Wang et al19FHS||US: mainly whites||Age, sex, BMI, SBP, HTN, DM, smoking, total and HDL-cholesterol, AF, CVD, EKG LVH, murmur, and sST2, GDF-15, hsTnI, hsCRP, and BNP||342816211.3||Framingham criteria||0.870||NR||NR||0.39 (0.21–0.57)||0.04 (0.02–0.06)|
|Agarwal et al20ARIC||US: biethnic—whites and blacks||Age, race, sex, prevalent CHD, SBP, antihypertensives, DM, smoking, HR, BMI||13 555148710||HF hospitalization or ICD-9 code on the death certificate||NR||NR||NR||NR||NR|
|0.7937 after bootstrap||Bootstrapping|
|Agarwal et al20ARIC||US: whites and blacks||Age, race, sex, NT-proBNP||13 555148710||HF admission or ICD-9 code on death certificate||NR||NR||NR|
|Schnabel et al21FHS||US: whites||Age, BMI, EKG LVH, DM, MI significant murmur||72516110||Framingham criteria||0.71 (0.67–0.75)||HL χ2=7.29, (P=0.61)||NR||NR||NR|
|Wong et al22 NAVIGATOR study||38 Countries mixed||Age, CHD, EKG, waist SBP, AF, COPD, urine albumin/creatinine ratio, hemoglobin||89751246.5||HF hospitalization||0.864 (0.79after bootstrapping)||NR||Apparent||NR||NR|
|Wannamethee et al23 British Regional Heart Study||Mainly whites||Age obesity, HTN, DM, MI, and angina||387025411||Physician diagnosis||0.708 (0.676–0.741)||NR||Apparent||NR|
Populations, Risk Factors, and Outcomes
Eleven models were developed from US cohorts, 1 in Sweden, 1 in Britain, and 2 multicenter cohorts (19 European countries in 1 study17 and 38 countries from America, Europe, Asia, and Africa in another study22). Models were mostly developed in whites; the models based on US populations included non-white participants, mainly blacks. The number of participants ranged from 725 to 359 947 and 18 to 99 years of age. The length of follow-up ranged from 2.9 to 14 years. Three models were developed in specific populations, including diabetes mellitus,17 impaired glucose tolerance,22 and atrial fibrillation.21 The method of HF identification varied, but was mainly based on clinical presentation, hospital discharge records, and cause of death. One study defined HF using outpatient records in addition to hospital and death record12 and another study included an echocardiographic-based definition of HF.18 None of the studies assessed HF with preserved ejection fraction and HF with reduced ejection fraction separately.
One study published the numbers of variables tested for inclusion.22 Thus for most studies, the number of outcomes of interest per candidate variable could not be assessed. The predictors most commonly included in the final models were age, sex, body mass index, diabetes mellitus, systolic blood pressure, creatinine, serum albumin or total protein, LV hypertrophy, and coronary artery disease. Biomarkers were evaluated in several studies, including B-type natriuretic peptide (BNP) or N-terminal pro-BNP (NT-proBNP), midregional proatrial natriuretic peptide, interleukin (IL)-6), tumor necrosis factor (TNF)-α, C-reactive protein (CRP), soluble IL-2 receptor, growth differentiation factor-15, high-sensitivity troponin I (hsTnI), and urinary albumin/creatinine ratio.
Performance of the Models
Table 1 displays the model performance. The AUC ranged from 0.71 to 0.87, indicating a modest-to-good discrimination. Risk scores were internally validated through bootstrapping in 411,17,20,22 and jackknife in 1 study.13 Five models had calibration estimated with the Hosmer–Lemeshow test. Most studies did not test for sex-based10–13,16–18,21,22 or race-based10–13,15–19,21,22 specific performance, except the Health Aging, and Body Composition (ABC) study HF score that predicted HF risk similarly in both sex and white/black race in development cohort,11 but the risk was underestimated in white men, especially in the 5% to 10% risk category, in the validation cohort.24 The AUCs were 0.71, 0.75, 0.74, and 0.72 in white men, white women, black men, and black women, respectively.24
In the Framingham Study, BNP and the urinary albumin/creatinine ratio increased the AUC from 0.84 to 0.8613 and 13% of subjects were appropriately reclassified. In the Health ABC study, the addition of IL-6, TNF-α, and CRP increased AUC from 0.717 to 0.737, mainly because of IL-6 that improved AUC to 0.734 (P=0.001).16 In the Cardiovascular Health Study, NT-proBNP increased AUC from 0.667 to 0.719 (P<0.001), but discrimination remained moderate.25 In the Malmö Diet and Cancer study, midregional proatrial natriuretic peptide, NT-proBNP, and CRP together improved AUC from 0.815 to 0.842 with an IDI of 0.05 (P<0.001) and an NRI of 22% (P<0.001).15 In the Atherosclerosis Risk in Communities study, adding NT-proBNP increased the AUC by 0.032, NRI by 13%, and IDI by 0.057; NRI was higher in women (17.9%) than in men (14.4%).20
Echocardiographic score (left ventricular ejection fraction and mass, E/A ratio, and left atrium), and NT-proBNP improved AUC (echocardiography, 0.031; NT-proBNP, 0.027; combined, 0.043; all P<0.01), and yielded robust IDI (echocardiography, 43.3%; NT-proBNP, 42.2%; combined, 61.7%; all P<0.001), and NRI (echocardiography, 11.3%; NT-proBNP, 10.6%; combined, 16.3%; all P<0.01), when added to the Health ABC model.18 Participants at intermediate risk (5%–20% 5-year risk; 35.7% of the cohort) derived the most reclassification benefit. Echocardiography yielded modest reclassification when used sequentially after NT-proBNP.
A multimarker score with soluble ST2, growth differentiation factor-15, hsTnI, hsCRP, and BNP increased AUC (0.846–0.870), NRI (0.39), and IDI (0.04) in the Framingham study.19 In the Atherosclerosis Risk in Communities study, cardiac troponin T and NT-proBNP improved AUC by 0.040 and 0.057; and NRIs by 50.7% and 54.7% in women and in men, respectively.26 In the Multi-Ethnic Study of Atherosclerosis cohort, NT-proBNP and LV mass index change AUC from 0.81 to 0.89; IDI by 0.046; NRI 6-year risk probability categorized by <3%, 3% to 10%, >10% of 0.175, P=0.019 and category-less NRI by 561, P<0.001.27 In the Framingham cohort, a composite of LV hypertrophy and LV dysfunction increased AUC from 0.765 to 0.770. With growth differentiation factor-15 and hsTnI, the NRI was 0.212 (P<0.0001).28 In the British Heart Regional Study, the addition of NT-proBNP to the Health ABC model improved AUC from 0.706 to 0.764 (P<0.0001).23 In the Prevention of Vascular and Renal End stage Disease (PREVEND) study, NT-proBNP, midregional proatrial natriuretic peptide, hsTnT, hsCRP, cystatin-C and urinary albumin excretion individually modestly improved the AUC of 0.826 by 4.7, 2.2, 2.5, 1.9, 1.0, and 1.6%, respectively.29
Five models were externally validated (Table 2). Of these, the Framingham10 and Health ABC models11 were externally validated more than once. The AUC in validation studies were generally lower and with a poorer calibration, although the latter was not assessed in most of validation studies.
|Author, Reference||Validated Risk Score||Validation Population/Country||Ethnicity||Design||Sample Size||Age, y||Time-Horizon, y||DiscriminationAUC||Calibration||Reclassification|
|NRI, % (95% CI/P Value)||IDI, % (95% CI/P Value)|
|Butler et al11||Framingham HF risk score||USA/Health ABC study||Mainly white||Prospective cohort||Males (n=1315) and female (n=1364)||70–79||5 y||0.735 in men and 0.684 in women||NR||NR||NR|
|Kalogeropoulos et al16||Health ABC score||Cardiovascular Health Study||Mainly white||Prospective cohort||5335 (400 events)||Mean 72.7 (SD: 5.5)||5||0.74 (95% CI, 0.72–0.76)||HL χ2=14.72 (P=0.14)||NA||NA|
|Gupta et al30||Health ABC Composition HF risk score||Dallas Heart Study||Caucasians and Blacks||Prospective cohort||2540||30–65||Stage B HF-BSA, 0.73stage B HF-FFM, 0.75stage B HF-height, 0.64||NR||NA||NA|
|Agarwal et al20||Framingham-published score||ARIC cohort/US||Mixed||Prospective cohort||12 038||0.610||NR||NA||NA|
|Agarwal et al20||Framingham-recalibrated score||ARIC cohort/USA||Mixed||Prospective cohort||4298 for NHANES and 21, 221 for ARIC/CHS||5 (ARIC/CHS cohort)||0.762||NR||NA||NA|
|Agarwal et al20||Health ABC HF recalibrated score||ARIC cohort/USA||Mixed/Mainly white||Prospective cohort||2145 for ENRICHD and 3640 for VISP||NA||0.783||NR||NA||NA|
|Agarwal et al20||ARIC risk score validation 1||ARIC cohort/USA||Mixed/Mainly white||Prospective cohort||5168||4||0.785||NR||NA||NA|
|Agarwal et al20||ARIC risk score validation 2||ARIC cohort/USA||Mixed/Mainly white||Prospective cohort||5168||Mean: 51.2 (SD, 10.5)||4||0.797||NR||NA||NA|
|Wannamethe et al23||Health ABC risk score||British Heart Regional Study||Mainly white||Prospective cohort||3170||40–59||11||0.706||NR||…||…|
Single Biomarker or Imaging Data
In the Cardiovascular Health Study, a comparison of the Health ABC score, NT-proBNP, and echocardiographic data for detecting LV dysfunction indicated >5% risk of HF at 5 years in 44% of the cohort and identified 76% of those who developed HF by 5 years.24,25 An abnormal NT-proBNP was present in 29.5% of the cohort and identified 45% of those who developed HF by 10 years; and baseline echocardiography identified reduced ejection fraction in 7.2% of the population, but identified only 12% of those who developed HF by 10 years.
Depending on the method of indexing LV mass (body surface area, fat-free mass, or height), the AUC for the Health ABC HF score (0.73, 0.75, and 0.64, respectively) was greater than that for BNP (0.62, 0.70, and 0.57, respectively) and NT-pro-BNP (0.62, 0.69, and 0.56, respectively; P<0.01 for all). Similar results were seen in the Framingham score except when LV mass was indexed to fat-free mass (0.65, 0.69 and 0.66, and 0.64, respectively). Addition of BNP resulted in an improvement in discrimination of stage B HF (difference in AUC, 0.01–0.03; P<0.05 for all).30 In an elderly at high risk, NT-proBNP (AUC, 0.74) was superior to the Framingham HF score (AUC, 0.63) in predicting asymptomatic LV systolic dysfunction.31
There were no recommendations for using any risk prediction model to estimate the risk of incident HF in either the clinical or the community settings in any of the existing guidelines. There were no studies assessing the effect of adopting HF risk models in clinical practice on the process of care and outcomes of patients.
This systematic review demonstrates the feasibility of global HF risk prediction in the general population. Compared with coronary artery disease and stroke, prediction of incident HF risk is at an early stage and is not incorporated into contemporary clinical or public health practice. Defining risk can be reliably done using a combination of common variables with an acceptable-to-good discrimination, although correcting for overfitting (internal validation) or testing in a new population (external validation) may attenuate discrimination. Implementation of prediction models may identify a wider segment of at-risk population compared with single risk factors. In addition, biomarkers may be used to further categorize patients into different risk strata in concert with the clinical variables.
HF guidelines advocate prevention of progression from at-risk HF stages (A and B) to symptomatic stages (C and D), thus requiring identification of susceptible patients likely to benefit from interventions,7 including lifestyle changes or more intense risk factor modification. Risk communication to patients may motivate adherence; indeed healthy lifestyle is associated with a lower lifetime risk of HF.32 Furthermore, risk scores can be incorporated into electronic medical records, allowing continual passive screening of high-risk subjects with alerts for patients entering higher-risk strata. These risk models may also circumvent uncertainties surrounding the stage B HF definition, which is currently dictated by what can be measured with imaging rather than a complete understanding of risk. In stepwise screening with models as the first step followed by imaging or biomarkers, those with stage B HF may be more reliably referred for interventions known to halt and possibly reverse structural changes that precede clinical HF. With increasing HF risk, LV mass and concentric remodeling increases, supporting the traditional belief that HF risk correlates with structural heart disease.30 HF models may be useful in assessing the use of novel biomarkers for risk prediction or for selecting patients for inclusion in clinical trials. Indeed, investigating novel markers is warranted, especially given the relatively modest additional discriminative ability of natriuretic peptides (BNP or NT-proBNP) beyond that of clinical variables. This finding is not surprising because factors included in models (eg, plasma creatinine, female sex, and age) may increase these biomarkers independently of left ventricular ejection fraction and filling pressures.33 Moreover, several screening studies have found use of natriuretic peptide level as stand-alone test to lack discriminative ability for detecting structural abnormalities among healthy young adults.34,35
Some of the existing models have been statistically underpowered, especially as consideration was not given to the ratio of outcomes to predictors. Models with an inappropriate ratio (<10 events per variable) are less robust, with a performance that shrinks when applied to other populations. Although the overall study size and the prognostic ability of individual predictors is important, a key driver of statistical power is the number of events. These performance metrics mostly originated from the derivation sample, which are optimistic. Some models were internally validated (bootstrap or jackknife validation), providing an idea about their potential performance when applied to a new population. Calibration was not always assessed. A poorly calibrated model may have the same discrimination as a well-calibrated model, necessitating both of these characteristics to be assessed in concert. Attempts to update models by extra variables have focused on discrimination,13,15,16,18–20,25 and none of the studies reported change in the calibration. Finally, validation is an important step before a model can be recommended for widespread use. This was limited to only 3 models in US populations. Hence, further validations of existing models are needed to guarantee their generalizability.
HF models have mainly been published as mathematical equations. Complex formulas may not be suitable for clinical application. Indeed, previous research suggests that many clinicians find risk calculation too time consuming and are not convinced of the value of information derived from these models.36 Additional translation efforts are needed to convert predictions equations into user-friendly tools to improve uptake.15 Efforts are needed to derive the appropriate context for defining high-risk status to facilitate their use. It is also important to study whether implementing HF risk prediction affects the behavior of healthcare providers and improves clinical outcomes. Few studies examined the incremental predictive value of novel biomarkers (eg, collagen matrix37,38 and omic markers) that may be useful for risk stratification. Clinical risk models provide good basis to assess the incremental value of new discoveries.
Further limitations to existing models exist. Misclassification of HF events may have occurred as there are no universally agreed specific criteria for diagnosing HF, thus affecting the performance of the models. HF is a heterogeneous syndrome, representing a spectrum of pathogenesis, as well as normal to depressed ejection fraction. Existing studies use multiple criteria to define HF39 with little agreement.40 Inconsistencies were indeed observed in the presently reviewed studies, which may have affected model performance. In most studies, diagnosis did not rely on imaging. Consequently, specific forms of HF, such as HF with reduced ejection fraction and HF with preserved ejection fraction, were not considered. Although there is an overlap between predictors of HF with reduced ejection fraction and HF with preserved ejection fraction, there are differences that merit consideration.41,42 Moreover, HF with reduced ejection fraction and HF with preserved ejection fraction are regarded as distinct pathophysiologic entities, as exemplified by the lack of evidence-based disease modifying therapies for the later. Accordingly, it stands to reason that differing preventative strategies and interventions may exist. Furthermore, the majority of participants included were whites. However, HF risk is known to be heightened in specific groups (eg, blacks), thus supporting the need to incorporate more patients of different ethnic backgrounds.
Although existing models may lead to implementation of prevention interventions, such efforts may be premature, particularly in the absence of external validation studies and impact analysis. The lack of studies evaluating the effect of model use on outcomes should be addressed in future studies. Such an undertaking could potentially strengthen the case for their use in routine clinical or public health practice. The absence of existing studies may simply be an indication of the embryonic nature of the HF prediction field, and of HF prevention in general. This limitation is not specific to HF and does not in itself invalidate the future use of the models. Indeed, even with coronary heart disease, for which prediction models have been in existence because the early 1990s, only a limited number of studies have assessed the effect of their implementation, although their use has been consistently recommended in guidelines. We did not explicitly rank or categorize the quality of existing HF models, as there is no agreed-on scientific system for rating risk prediction model quality. Some argue minimizing risk for potential bias is of critical importance, whereas others contend that a risk score should be judged on ability to perform accurately across diverse settings. Finally, our ability to assess publication bias was limited.
In conclusion, risk models for predicting incident HF have acceptable-to-good discriminative ability. These tools need to be better calibrated and externally validated, and the effect of their use on outcomes assessed before incorporation into clinical practice guidelines. With advancement of our understanding of systemic conditions and cardiac and noncardiac structural and functional abnormalities that predispose to HF risk, better approaches for HF prevention are needed. HF prevention extends beyond simply addressing risk factors or precursors, such as diabetes mellitus, hypertension, and coronary artery disease, especially as the latter is not a sine qua non for the development of HF. HF scores may be viewed as a means for selecting a targeted population that would undergo imaging for detection of asymptomatic structural disease (eg, left ventricular systolic dysfunction), a potential therapeutic target in itself. Successful prevention approaches mandate the correct identification of the target population, underscoring the importance of further research in the performance and use of incident HF risk models.
Dr Gheorghiade has been a consultant for Abbott Laboratories, Astellas, AstraZeneca, Bayer HealthCare AG, CorThera, Cytokinetics, DebioPharm S.A., Errekappa Terapeutici, GlaxoSmithKline, Ikaria, Johnson & Johnson, Medtronic, Merck, Novartis Pharma AG, Otsuka Pharmaceuticals, Palatin Technologies, Pericor Therapeutics, Protein Design Laboratories, Sanofi-Aventis, Sigma Tau, Solvay Pharmaceuticals, Takeda Pharmaceutical and Trevena Therapeutics. The remaining authors report no conflicts.
Go AS, Mozaffarian D, Roger VL, Benjamin EJ, Berry JD, Blaha MJ, Dai S, Ford ES, Fox CS, Franco S, Fullerton HJ, Gillespie C, Hailpern SM, Heit JA, Howard VJ, Huffman MD, Judd SE, Kissela BM, Kittner SJ, Lackland DT, Lichtman JH, Lisabeth LD, Mackey RH, Magid DJ, Marcus GM, Marelli A, Matchar DB, McGuire DK, Mohler ER, Moy CS, Mussolino ME, Neumar RW, Nichol G, Pandey DK, Paynter NP, Reeves MJ, Sorlie PD, Stein J, Towfighi A, Turan TN, Virani SS, Wong ND, Woo D, Turner MB; American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Heart disease and stroke statistics–2014 update: a report from the American Heart Association.Circulation. 2014; 129:e28–e292. doi: 10.1161/01.cir.0000441139.02102.80.LinkGoogle Scholar
Heidenreich PA, Albert NM, Allen LA, Bluemke DA, Butler J, Fonarow GC, Ikonomidis JS, Khavjou O, Konstam MA, Maddox TM, Nichol G, Pham M, Piña IL, Trogdon JG; American Heart Association Advocacy Coordinating Committee; Council on Arteriosclerosis, Thrombosis and Vascular Biology; Council on Cardiovascular Radiology and Intervention; Council on Clinical Cardiology; Council on Epidemiology and Prevention; Stroke Council. Forecasting the impact of heart failure in the United States: a policy statement from the American Heart Association.Circ Heart Fail. 2013; 6:606–619. doi: 10.1161/HHF.0b013e318291329a.LinkGoogle Scholar
Levy D, Kenchaiah S, Larson MG, Benjamin EJ, Kupka MJ, Ho KK, Murabito JM, Vasan RSLong-term trends in the incidence of and survival with heart failure.N Engl J Med. 2002; 347:1397–1402. doi: 10.1056/NEJMoa020265.CrossrefMedlineGoogle Scholar
Lloyd-Jones DMCardiovascular risk prediction: basic concepts, current status, and future directions.Circulation. 2010; 121:1768–1777. doi: 10.1161/CIRCULATIONAHA.109.849166.LinkGoogle Scholar
Hayden JA, van der Windt DA, Cartwright JL, Côté P, Bombardier CAssessing bias in studies of prognostic factors.Ann Intern Med. 2013; 158:280–286. doi: 10.7326/0003-4819-158-4-201302190-00009.CrossrefMedlineGoogle Scholar
Reilly BM, Evans ATTranslating clinical research into clinical practice: impact of using prediction rules to make decisions.Ann Intern Med. 2006; 144:201–209.CrossrefMedlineGoogle Scholar
Yancy CW, Jessup M, Bozkurt B, Butler J, Casey DE, Drazner MH, Fonarow GC, Geraci SA, Horwich T, Januzzi JL, Johnson MR, Kasper EK, Levy WC, Masoudi FA, McBride PE, McMurray JJ, Mitchell JE, Peterson PN, Riegel B, Sam F, Stevenson LW, Tang WH, Tsai EJ, Wilkoff BL2013 ACCF/AHA guideline for the management of heart failure: a report of the American College of Cardiology Foundation/American Heart Association Task Force on practice guidelines.Circulation. 2013; 128:e240–e327. doi: 10.1161/CIR.0b013e31829e8807.LinkGoogle Scholar
Lindenfeld J, Albert NM, Boehmer JP, Collins SP, Ezekowitz JA, Givertz MM, Katz SD, Klapholz M, Moser DK, Rogers JG, Starling RC, Stevenson WG, Tang WH, Teerlink JR, Walsh MNHFSA 2010 Comprehensive Heart Failure Practice Guideline.J Card Fail. 2010; 16:e1–194.CrossrefMedlineGoogle Scholar
McMurray JJ, Adamopoulos S, Anker SD, Auricchio A, Bohm M, Dickstein K, Falk V, Filippatos G, Fonseca C, Gomez-Sanchez MA, Jaarsma T, Kober L, Lip GY, Maggioni AP, Parkhomenko A, Pieske BM, Popescu BA, Ronnevik PK, Rutten FH, Schwitter J, Seferovic P, Stepinska J, Trindade PT, Voors AA, Zannad F, Zeiher A, Task Force for the D, Treatment of A, Chronic Heart Failure of the European Society of C, Bax JJ, Baumgartner H, Ceconi C, Dean V, Deaton C, Fagard R, Funck-Brentano C, Hasdai D, Hoes A, Kirchhof P, Knuuti J, Kolh P, McDonagh T, Moulin C, Popescu BA, Reiner Z, Sechtem U, Sirnes PA, Tendera M, Torbicki A, Vahanian A, Windecker S, McDonagh T, Sechtem U, Bonet LA, Avraamides P, Ben Lamin HA, Brignole M, Coca A, Cowburn P, Dargie H, Elliott P, Flachskampf FA, Guida GF, Hardman S, Iung B, Merkely B, Mueller C, Nanas JN, Nielsen OW, Orn S, Parissis JT, Ponikowski PESC guidelines for the diagnosis and treatment of acute and chronic heart failure 2012: The Task Force for the Diagnosis and Treatment of Acute and Chronic Heart Failure 2012 of the European Society of Cardiology. Developed in collaboration with the Heart Failure Association (HFA) of the ESC.Eur J Heart Fail. 2012; 14:803–869. doi: 10.1093/eurjhf/hfs105.CrossrefMedlineGoogle Scholar
Kannel WB, D’Agostino RB, Silbershatz H, Belanger AJ, Wilson PW, Levy DProfile for estimating risk of heart failure.Arch Intern Med. 1999; 159:1197–1204.CrossrefMedlineGoogle Scholar
Butler J, Kalogeropoulos A, Georgiopoulou V, Belue R, Rodondi N, Garcia M, Bauer DC, Satterfield S, Smith AL, Vaccarino V, Newman AB, Harris TB, Wilson PW, Kritchevsky SB; Health ABC Study. Incident heart failure prediction in the elderly: the health ABC heart failure score.Circ Heart Fail. 2008; 1:125–133. doi: 10.1161/CIRCHEARTFAILURE.108.768457.LinkGoogle Scholar
Goyal A, Norton CR, Thomas TN, Davis RL, Butler J, Ashok V, Zhao L, Vaccarino V, Wilson PWPredictors of incident heart failure in a large insured population: a one million person-year follow-up study.Circ Heart Fail. 2010; 3:698–705. doi: 10.1161/CIRCHEARTFAILURE.110.938175.LinkGoogle Scholar
Velagaleti RS, Gona P, Larson MG, Wang TJ, Levy D, Benjamin EJ, Selhub J, Jacques PF, Meigs JB, Tofler GH, Vasan RSMultimarker approach for the prediction of heart failure incidence in the community.Circulation. 2010; 122:1700–1706. doi: 10.1161/CIRCULATIONAHA.109.929661.LinkGoogle Scholar
deFilippi CR, de Lemos JA, Christenson RH, Gottdiener JS, Kop WJ, Zhan M, Seliger SLAssociation of serial measures of cardiac troponin T using a sensitive assay with incident heart failure and cardiovascular mortality in older adults.JAMA. 2010; 304:2494–2502. doi: 10.1001/jama.2010.1708.CrossrefMedlineGoogle Scholar
Smith JG, Newton-Cheh C, Almgren P, Struck J, Morgenthaler NG, Bergmann A, Platonov PG, Hedblad B, Engström G, Wang TJ, Melander OAssessment of conventional cardiovascular risk factors and multiple biomarkers for the prediction of incident heart failure and atrial fibrillation.J Am Coll Cardiol. 2010; 56:1712–1719. doi: 10.1016/j.jacc.2010.05.049.CrossrefMedlineGoogle Scholar
Kalogeropoulos A, Georgiopoulou V, Psaty BM, Rodondi N, Smith AL, Harrison DG, Liu Y, Hoffmann U, Bauer DC, Newman AB, Kritchevsky SB, Harris TB, Butler J; Health ABC Study Investigators. Inflammatory markers and incident heart failure risk in older adults: the Health ABC (Health, Aging, and Body Composition) study.J Am Coll Cardiol. 2010; 55:2129–2137. doi: 10.1016/j.jacc.2009.12.045.CrossrefMedlineGoogle Scholar
Pfister R, Cairns R, Erdmann E, Schneider CAA clinical risk score for heart failure in patients with type 2 diabetes and macrovascular disease: an analysis of the PROactive study.Int J Cardiol. 2013; 162:112–116. doi: 10.1016/j.ijcard.2011.05.056.CrossrefMedlineGoogle Scholar
Kalogeropoulos AP, Georgiopoulou VV, deFilippi CR, Gottdiener JS, Butler JEchocardiography, natriuretic peptides, and risk for incident heart failure in older adults: the Cardiovascular Health Study.J Am Coll Cardiol Img. 2012; 5:131–140. doi: 10.1016/j.jcmg.2011.11.011.CrossrefGoogle Scholar
Wang TJ, Wollert KC, Larson MG, Coglianese E, McCabe EL, Cheng S, Ho JE, Fradley MG, Ghorbani A, Xanthakis V, Kempf T, Benjamin EJ, Levy D, Vasan RS, Januzzi JLPrognostic utility of novel biomarkers of cardiovascular stress: the Framingham Heart Study.Circulation. 2012; 126:1596–1604. doi: 10.1161/CIRCULATIONAHA.112.129437.LinkGoogle Scholar
Agarwal SK, Chambless LE, Ballantyne CM, Astor B, Bertoni AG, Chang PP, Folsom AR, He M, Hoogeveen RC, Ni H, Quibrera PM, Rosamond WD, Russell SD, Shahar E, Heiss GPrediction of incident heart failure in general practice: the Atherosclerosis Risk in Communities (ARIC) Study.Circ Heart Fail. 2012; 5:422–429. doi: 10.1161/CIRCHEARTFAILURE.111.964841.LinkGoogle Scholar
Schnabel RB, Rienstra M, Sullivan LM, Sun JX, Moser CB, Levy D, Pencina MJ, Fontes JD, Magnani JW, McManus DD, Lubitz SA, Tadros TM, Wang TJ, Ellinor PT, Vasan RS, Benjamin EJRisk assessment for incident heart failure in individuals with atrial fibrillation.Eur J Heart Fail. 2013; 15:843–849. doi: 10.1093/eurjhf/hft041.CrossrefMedlineGoogle Scholar
Wong YW, Thomas L, Sun JL, McMurray JJ, Krum H, Hernandez AF, Rutten GE, Leiter LA, Standl E, Haffner SM, Mazzone T, Martinez FA, Tognoni G, Giles T, Califf RMPredictors of incident heart failure hospitalizations among patients with impaired glucose tolerance: insight from the Nateglinide And Valsartan in Impaired Glucose Tolerance Outcomes Research study.Circ Heart Fail. 2013; 6:203–210. doi: 10.1161/CIRCHEARTFAILURE.112.000086.LinkGoogle Scholar
Wannamethee SG, Welsh P, Whincup PH, Lennon L, Papacosta O, Sattar NN-terminal pro brain natriuretic peptide but not copeptin improves prediction of heart failure over other routine clinical risk parameters in older men with and without cardiovascular disease: population-based study.Eur J Heart Fail. 2014; 16:25–32. doi: 10.1093/eurjhf/hft124.CrossrefMedlineGoogle Scholar
Kalogeropoulos A, Psaty BM, Vasan RS, Georgiopoulou V, Smith AL, Smith NL, Kritchevsky SB, Wilson PW, Newman AB, Harris TB, Butler J; Cardiovascular Health Study. Validation of the health ABC heart failure model for incident heart failure risk prediction: the Cardiovascular Health Study.Circ Heart Fail. 2010; 3:495–502. doi: 10.1161/CIRCHEARTFAILURE.109.904300.LinkGoogle Scholar
deFilippi CR, Christenson RH, Kop WJ, Gottdiener JS, Zhan M, Seliger SLLeft ventricular ejection fraction assessment in older adults: an adjunct to natriuretic peptide testing to identify risk of new-onset heart failure and cardiovascular death?J Am Coll Cardiol. 2011; 58:1497–1506. doi: 10.1016/j.jacc.2011.06.042.CrossrefMedlineGoogle Scholar
Nambi V, Liu X, Chambless LE, de Lemos JA, Virani SS, Agarwal S, Boerwinkle E, Hoogeveen RC, Aguilar D, Astor BC, Srinivas PR, Deswal A, Mosley TH, Coresh J, Folsom AR, Heiss G, Ballantyne CMTroponin T and N-terminal pro-B-type natriuretic peptide: a biomarker approach to predict heart failure risk–the atherosclerosis risk in communities study.Clin Chem. 2013; 59:1802–1810. doi: 10.1373/clinchem.2013.203638.CrossrefMedlineGoogle Scholar
Choi EY, Bahrami H, Wu CO, Greenland P, Cushman M, Daniels LB, Almeida AL, Yoneyama K, Opdahl A, Jain A, Criqui MH, Siscovick D, Darwin C, Maisel A, Bluemke DA, Lima JAN-terminal pro-B-type natriuretic peptide, left ventricular mass, and incident heart failure: Multi-Ethnic Study of Atherosclerosis.Circ Heart Fail. 2012; 5:727–734. doi: 10.1161/CIRCHEARTFAILURE.112.968701.LinkGoogle Scholar
Xanthakis V, Larson MG, Wollert KC, Aragam J, Cheng S, Ho J, Coglianese E, Levy D, Colucci WS, Michael Felker G, Benjamin EJ, Januzzi JL, Wang TJ, Vasan RSAssociation of novel biomarkers of cardiovascular stress with left ventricular hypertrophy and dysfunction: implications for screening.J Am Heart Assoc. 2013; 2:e000399. doi: 10.1161/JAHA.113.000399.LinkGoogle Scholar
Brouwers FP, van Gilst WH, Damman K, van den Berg MP, Gansevoort RT, Bakker SJ, Hillege HL, van Veldhuisen DJ, van der Harst P, de Boer RAClinical risk stratification optimizes value of biomarkers to predict new-onset heart failure in a community-based cohort.Circ Heart Fail. 2014; 7:723–731. doi: 10.1161/CIRCHEARTFAILURE.114.001185.LinkGoogle Scholar
Gupta S, Berry JD, Ayers CR, Matulevicius SA, Peshock RM, Patel PC, Markham DW, Drazner MHAssociation of Health Aging and Body Composition (ABC) Heart Failure score with cardiac structural and functional abnormalities in young individuals.Am Heart J. 2010; 159:817–824. doi: 10.1016/j.ahj.2010.02.001.CrossrefMedlineGoogle Scholar
Iacovoni A, De Maria R, Gavazzi A, Grosu A, Fontana A, Guideri S, Gori M, Ferrari P, Filippi A, Rutili MR, Donzelli L, Parolini M, Senni MNatriuretic peptides and the Framingham risk score for screening of asymptomatic left ventricular systolic dysfunction in high-risk patients in primary care. The DAVID-BERG study.Int J Cardiol. 2013; 168:5093–5095. doi: 10.1016/j.ijcard.2013.07.260.CrossrefMedlineGoogle Scholar
Djoussé L, Driver JA, Gaziano JMRelation between modifiable lifestyle factors and lifetime risk of heart failure.JAMA. 2009; 302:394–400. doi: 10.1001/jama.2009.1062.CrossrefMedlineGoogle Scholar
Daniels LB, Maisel ASNatriuretic peptides.J Am Coll Cardiol. 2007; 50:2357–2368. doi: 10.1016/j.jacc.2007.09.021.CrossrefMedlineGoogle Scholar
Vasan RS, Benjamin EJ, Larson MG, Leip EP, Wang TJ, Wilson PW, Levy DPlasma natriuretic peptides for community screening for left ventricular hypertrophy and systolic dysfunction: the Framingham Heart Study.JAMA. 2002; 288:1252–1259.CrossrefMedlineGoogle Scholar
de Lemos JA, McGuire DK, Khera A, Das SR, Murphy SA, Omland T, Drazner MHScreening the population for left ventricular hypertrophy and left ventricular systolic dysfunction using natriuretic peptides: results from the Dallas Heart Study.Am Heart J. 2009; 157:746–753.e2. doi: 10.1016/j.ahj.2008.12.017.CrossrefMedlineGoogle Scholar
Eichler K, Zoller M, Tschudi P, Steurer JBarriers to apply cardiovascular prediction rules in primary care: a postal survey.BMC Fam Pract. 2007; 8:1. doi: 10.1186/1471-2296-8-1.CrossrefMedlineGoogle Scholar
Rocchiccioli JP, McMurray JJ, Dominiczak AFBiomarkers in heart failure: a clinical review.Heart Fail Rev. 2010; 15:251–273. doi: 10.1007/s10741-008-9123-9.CrossrefMedlineGoogle Scholar
van Kimmenade RR, Januzzi JL. Emerging biomarkers in heart failure.Clin Chem. 2012; 58:127–138. doi: 10.1373/clinchem.2011.165720.CrossrefMedlineGoogle Scholar
Zannad F, Stough WG, Pitt B, Cleland JG, Adams KF, Geller NL, Torp-Pedersen C, Kirwan BA, Follath FHeart failure as an endpoint in heart failure and non-heart failure cardiovascular clinical trials: the need for a consensus definition.Eur Heart J. 2008; 29:413–421. doi: 10.1093/eurheartj/ehm603.CrossrefMedlineGoogle Scholar
Rosamond WD, Chang PP, Baggett C, Johnson A, Bertoni AG, Shahar E, Deswal A, Heiss G, Chambless LEClassification of heart failure in the atherosclerosis risk in communities (ARIC) study: a comparison of diagnostic criteria.Circ Heart Fail. 2012; 5:152–159. doi: 10.1161/CIRCHEARTFAILURE.111.963199.LinkGoogle Scholar
Ho JE, Gona P, Pencina MJ, Tu JV, Austin PC, Vasan RS, Kannel WB, D’Agostino RB, Lee DS, Levy DDiscriminating clinical features of heart failure with preserved vs. reduced ejection fraction in the community.Eur Heart J. 2012; 33:1734–1741. doi: 10.1093/eurheartj/ehs070.CrossrefMedlineGoogle Scholar
Ho JE, Lyass A, Lee DS, Vasan RS, Kannel WB, Larson MG, Levy DPredictors of new-onset heart failure: differences in preserved versus reduced ejection fraction.Circ Heart Fail. 2013; 6:279–286. doi: 10.1161/CIRCHEARTFAILURE.112.972828.LinkGoogle Scholar
Risk prediction models for incident heart failure (HF) in asymptomatic populations have the potential to improve risk stratification and augment the practitioner’s ability to prevent HF or to influence its natural history in several ways. These might include communicating risk to patients, offering appropriate patient education, proactive risk factor management, and timely use of imaging to identify asymptomatic structural heart disease for therapeutic intervention, with consequent referral to a cardiologist. Incident HF risk models may also be useful for selecting high-risk patients for inclusion in HF prevention trials or in trials assessing the benefits of therapies for patients with asymptomatic structural heart disease. However, incident HF risk prediction using combinations of clinical and biological variables remains at an early stage in development. Although the discriminative ability of current incident HF prediction models is acceptable, in general, these tools need to be better calibrated and externally validated before incorporation into clinical practice guidelines. It remains unclear which models are cost-effective and best suit population screening needs. The effects of models on clinical and preventative care require further study.