On June 25, 2024, the AHA Journals will be launching a new website design. During the launch process, there may be intermittent outages, and some features (alert sign-ups, article/issue purchases, account customizations/activations, and comment submissions) may be unavailable. This message will be removed when the launch process is complete. Thank you for your patience and we hope that you enjoy the new site!

Skip main navigation

External Performance of the HAVOC Score for the Prediction of New Incident Atrial Fibrillation

Originally publishedhttps://doi.org/10.1161/STROKEAHA.119.027990Stroke. 2020;51:457–461

Abstract

Background and Purpose—

The HAVOC score (hypertension, age, valvular heart disease, peripheral vascular disease, obesity, congestive heart failure, coronary artery disease) was proposed for the prediction of atrial fibrillation (AF) after cryptogenic stroke. It showed good model discrimination (area under the curve, 0.77). Only 2.5% of patients with a low-risk HAVOC score (ie, 0–4) were diagnosed with new incident AF. We aimed to assess its performance in an external cohort of patients with embolic stroke of undetermined source.

Methods—

In the AF-embolic stroke of undetermined source dataset, we assessed the discriminatory power, calibration, specificity, negative predictive value, and accuracy of the HAVOC score to predict new incident AF. Patients with a HAVOC score of 0 to 4 were considered as low-risk, as proposed in its original publication.

Results—

In 658 embolic stroke of undetermined source patients (median age, 67 years; 44% women), the median HAVOC score was 2 (interquartile range, 3). There were 540 (82%) patients with a HAVOC score of 0 to 4 and 118 (18%) with a score of ≥5. New incident AF was diagnosed in 95 (14.4%) patients (28.8% among patients with HAVOC score ≥5 and 11.3% among patients with HAVOC score 0–4 [age- and sex-adjusted odds ratio, 2.29 (95% CI, 1.37–3.82)]). The specificity of low-risk HAVOC score to identify patients without new incident AF was 88.7%. The negative predictive value of low-risk HAVOC score was 85.1%. The accuracy was 78.0%, and the area under the curve was 68.7% (95% CI, 62.1%–73.3%).

Conclusions—

The previously reported low rate of AF among embolic stroke of undetermined source patients with low-risk HAVOC score was not confirmed in our cohort. Further assessment of the HAVOC score is warranted before it is routinely implemented in clinical practice.

Introduction

Atrial fibrillation (AF) is a frequent finding during the diagnostic work-up in patients with embolic stroke of undetermined source (ESUS).1,2 Its incidence depends on the duration and the modality of heart rhythm monitoring and is higher when prolonged continuous cardiac monitoring is applied,3–5 which, however, is not performed routinely after ESUS: in a recent global survey in 61 countries, >24 hours cardiac monitoring was performed only in 17% of participating hospitals, mainly due to limited technical and human resources.6 In this context, it would be useful to have a tool to guide the selection of patients for prolonged automated cardiac monitoring according to their likelihood for AF.

The HAVOC score (hypertension, age, valvular heart disease, peripheral vascular disease, obesity, congestive heart failure, and coronary artery disease) was proposed as a clinical score for the prediction of AF in patients with cryptogenic stroke or transient ischemic attack. It assigns 2 points to hypertension, 2 points to age ≥75 years, 2 points for valve disease, 1 point for peripheral vascular disease, 1 point for obesity, 4 points for congestive heart failure, and 2 points for coronary artery disease. It was developed and internally validated using data from 1995 to 2015 in Stanford Translational Research Integrated Database Environment and showed good model discrimination (area under the curve, 0.77). In particular, only 2.5% of patients with a low-risk HAVOC score (ie, 0–4) were diagnosed with new incident AF during follow-up.7 It was also externally tested in the CRYSTAL-AF cohort (Cryptogenic Stroke and Underlying AF trial), where AF was detected in 18.5% of patients with a low-risk HAVOC score (ie, 0–3).8

The aim of the present analysis was to assess the performance of the HAVOC score in an external independent cohort of patients with ESUS. If it performs well, its use in routine clinical practice to inform selection of patients for prolonged automated cardiac monitoring could be further supported.

Methods

We will make the data, methods used in the analysis, and materials used to conduct the research available to any researcher for purposes of reproducing the results or replicating the procedure on reasonable request.

AF-ESUS Dataset

The AF-ESUS dataset (Prediction of Atrial Fibrillation in Patients With Embolic Stroke of Undetermined Source, https://www.clinicaltrials.gov. Unique identifier: NCT02766205) was used for this analysis. Details about the methodology which was followed in the AF-ESUS study were published previously.9,10 Briefly, the AF-ESUS dataset comprises of consecutive ESUS patients registered in 3 prospective stroke registries: the Acute Stroke Registry and Analysis of Lausanne registry, the Athens Stroke Registry, and the Larissa Stroke Registry.11–13 Study participants provided written or verbal informed consent for the research use of their deidentified data. The use of these registry data for research has been approved by the local ethics committees.

Definitions and Outcomes

ESUS was defined in accordance with the criteria proposed by the Cryptogenic Stroke/ESUS International Working Group as a visualized nonlacunar brain infarct in the absence of all of the following: (1) extracranial atherosclerosis causing ≥50% luminal stenosis in arteries supplying the area of ischemia; (2) major-risk cardioembolic source; and (3) any other specific cause of stroke (eg, arteritis, dissection, migraine/vasospasm, drug misuse).2 For pragmatic reasons, imaging of the intracranial arteries was not required for the definition of ESUS, similar to the approach that was followed in the NAVIGATE ESUS trial (New Approach Rivaroxaban Inhibition of Factor Xa in a Global Trial Versus ASA to Prevent Embolism in Embolic Stroke of Undetermined Source)14 and which can be justified in a Western population by the low prevalence of intracranial stenosis as a stroke mechanism.15

The outcome was new incident AF during follow-up, similar to the outcome assessed in other similar studies.16–18 New incident AF was considered present if confirmed by an ECG performed for any reason including palpitations, irregular pulse on clinical examination, in-hospital surveillance, or portable outpatient long-term monitoring. The assessment of outcome during follow-up was performed according to standard clinical practice during on-site patient visits at the outpatient clinic or by contact with the patient or the next of kin or the patient’s primary physician.

Statistical Analysis

To assess the performance of the score, we examined its discriminatory power and calibration in our dataset.19 Discrimination was defined as the degree to which the prognostic score enables the discrimination between patients with favorable and unfavorable outcome and was assessed by the calculation of the area under the curve. Calibration was defined as the agreement between predicted and actual outcome and was assessed with the use of the Hosmer-Lemeshow goodness-of-fit test with 10 groups.

We also assessed the specificity, negative predictive value, and accuracy of low-risk HAVOC score (ie, 0–4, as proposed in its original publication7) to identify patients without new incident AF during follow-up. The specificity (or else, true negative rate) was defined as the probability that a patient has a score of 0 to 4 if new incident AF is not diagnosed during follow-up and was calculated with the following formula: true negative/(true negative and false positive). The negative predictive value was defined as the probability that new incident AF is not diagnosed during follow-up if the score the patient has a score of 0 to 4 and was calculated with the following formula: true negative/(false negative and true negative). Accuracy was defined as the overall probability that a patient will be correctly classified and was calculated with the following formula: (true negative and true positive)/(true negative and false negative and true positive and false positive).

There was no imputation of missing data. Patient characteristics were described by groups using proportions for discrete variables and medians with interquartile range for continuous. Group differences were summarized by reporting the odds ratio (OR) and 95% CI. Statistical analyses were performed with R package (version 3.5.3).

Results

Baseline Characteristics of the Patients per HAVOC Score

In 658 patients with ESUS (median age: 67 years, 44% women), the median HAVOC score was 2 (interquartile range, 3). There were 540 patients (82%) with a HAVOC score 0 to 4 and 118 (18%) with a score ≥5. Only 2 patients (0.4%) had a HAVOC score ≥10. The baseline characteristics of patients per HAVOC score are summarized in Table 1.

Table 1. Baseline Patient Characteristics per HAVOC Score

VariableHAVOC (0–4; n=540)HAVOC (≥5; n=118)Odds Ratio95% CIP Value
Female sex244/540 (45.2%)49/118 (41.5%)0.860.57–1.290.4
Age, y64.7 (23.2)77.4 (12.5)1.081.06–1.10<0.001
NIHSS6.0 (9.0)6.0 (7.5)1.000.96–1.020.7
Hypertension304/540 (56.3%)113/118 (95.8%)17.547.80–50.22<0.001
Dyslipidemia356/540 (65.9%)90/118 (76.3%)1.661.06–2.670.03
Diabetes mellitus77/540 (14.3%)36/118 (30.5%)2.641.66–4.16<0.001
Smoking219/540 (40.6%)39/118 (33.1%)0.720.47–1.100.13
Coronary artery disease28/540 (5.2%)50/118 (42.4%)13.458.00–23.04<0.001
Previous stroke97/540 (18.0%)21/118 (17.8%)0.990.58–1.640.9
New incident atrial fibrillation during follow-up61/540 (11.3%)34/118 (28.8%)3.181.96–5.11<0.001

HAVOC indicates hypertension, age, valvular heart disease, peripheral vascular disease, obesity, congestive heart failure, coronary artery disease; and NIHSS, National Institutes of Health Stroke Scale.

Baseline Characteristics of the Patients per New Incident AF During Follow-Up

The overall-follow-up was 2340 patient-years corresponding to 3.6 years per patient. In 85% of cases, the assessment of outcome was performed during visits in the outpatient clinic. During follow-up, new incidental AF was diagnosed in 95 patients (14.4%). Patients with incidental AF during follow-up were older (74 versus 66 years, unadjusted OR, 1.04 [95% CI, 1.02–1.06] per year), had higher HAVOC score (median 4 versus 2, unadjusted OR, 1.35 [95% CI, 1.22–1.49]) and higher incidence of arterial hypertension (81.1% versus 60.4%, unadjusted OR, 2.81 [95% CI, 1.67–4.95]) and coronary artery disease (20.0% versus 11.9%, unadjusted OR, 2.14 [95% CI, 1.18–3.72]) compared with patients without new incidental AF. The baseline characteristics of the patients per new incident AF during follow-up are summarized in Table 2.

Table 2. Baseline Patient Characteristics per New Incident AF during Follow-Up Score

VariablePatients Without Atrial Fibrillation (n=563)Patients With Atrial Fibrillation (n=95)Odds Ratio95% CIsP Value
Female sex247/563 (43.9%)46/95 (48.4%)1.200.78–1.860.4
Age, y65.6 (24.4)74.0 (12.6)1.041.02–1.06<0.001
NIHSS6.0 (9.0)5.0 (7.0)0.980.95–1.010.2
Hypertension340/563 (60.4%)77/95 (81.1%)2.811.67–4.95<0.001
Dyslipidemia385/563 (68.4%)61/95 (64.2%)0.830.53–1.320.4
Diabetes mellitus90/563 (16.0%)23/95 (24.2%)1.680.98–2.790.06
Smoking225/563 (40.0%)33/95 (34.7%)0.800.50–1.250.3
Coronary artery disease59/563 (10.5%)19/95 (20.0%)2.141.18–3.720.01
Previous stroke98/563 (17.4%)20/95 (21.1%)1.270.72–2.130.4
HAVOC (continuous)2.0 (4.0)4.0 (3.0)1.351.22–1.49<0.001
HAVOC (≥5)84/563 (14.9%)34/95 (35.8%)3.181.96–5.11<0.001

Continuous parameters are summarized as median value and interquartile range. AF indicates atrial fibrillation; HAVOC, hypertension, age, valvular heart disease, peripheral vascular disease, obesity, congestive heart failure, coronary artery disease; and NIHSS, National Institutes of Health Stroke Scale.

Rate of New Incident AF During Follow-Up per HAVOC Score

The number of patients and the rate of new incident AF during follow-up per HAVOC score are presented in Table 3. The rate of new incident AF was 28.8% among the 118 patients with a HAVOC score of ≥5 and 11.3% among the 540 patients with a HAVOC score of 0 to 4 (age- and sex-adjusted OR, 2.29 [95% CI, 1.37–3.82]). The rate of new incident AF in patients with a HAVOC score of 0 was 4.3%.

Table 3. Number of Patients and Proportion of Patients With New Incident atrial Fibrillation During Follow-Up per HAVOC Score

HAVOC score01234567≥8
Number of patients162271496813445422110
Proportion of patients with new incident fibrillation4.318.510.119.115.724.426.242.930.0

HAVOC indicates hypertension, age, valvular heart disease, peripheral vascular disease, obesity, congestive heart failure, coronary artery disease.

Performance of Low HAVOC Score (0–4) to Identify Patients at Low Risk for New Incident AF

The specificity (or else, true negative rate) of low-risk HAVOC score (ie, 0–4) to identify patients without new incident AF during follow-up was 88.7%; that is, 11.3% of patients with low-risk HAVOC score were diagnosed with new incident AF during follow-up. The negative predictive value of low-risk HAVOC score, or else the probability that new incident AF will not be diagnosed if the HAVOC score is 0 to 4, is 85.1%. The accuracy was 78.0%. The area under the curve of the HAVOC score to predict new incident AF during follow-up was 68.7% (95% CI, 62.1%–73.3%). The Hosmer-Lemeshow X2 value of the logistic model relating new incident AF at follow-up with HAVOC score adjusted for age and sex was estimated at 14.44 (P=0.07), showing that the fit of the model to the observed data was of borderline acceptance.

Discussion

In this large dataset of consecutive ESUS patients, the HAVOC score performed less well than it did in the cohort of its original publication: the area under the curve in our cohort was 68.7% (compared with 77% in the original publication), the negative prognostic value was 85.1% (compared with 97% in the original publication), and the accuracy was 77% (compared with 80% in the original publication).7 Of importance, 11.3% of patients with a low-risk HAVOC score (ie, 0–4) were diagnosed with new incident AF during follow-up in our cohort, compared with 2.5% in its original publication.7 Our results are in line with a recent analysis of the HAVOC score at the CRYSTAL-AF cohort, in which 18.5% of patients with low-risk HAVOC score (ie, 0–3) were detected with AF and the negative prognostic value and accuracy were 84.7% and 73.8% respectively.8 These 2 analyses of the HAVOC score at the AF-ESUS (present analysis) and the CRYSTAL-AF cohorts do not confirm the previously reported low rate of AF among ESUS patients with low-risk HAVOC score.

The ability of the HAVOC score to identify patients with very low risk of AF could be higher if lower thresholds are used. For example, in our cohort, new incident AF was reported only in 4.3% of patients with a HAVOC score of 0; however, the clinical usefulness of these thresholds would be reduced given that in our cohort, this would apply only to 26.4% of patients. On the other side of the spectrum, it could be argued that higher HAVOC scores could perhaps be used also to screen-in patients who would benefit more from prolonged cardiac event monitoring, for example, in our cohort, new incident AF was reported in 28.8% of patients with a HAVOC score ≥5.

Embolism in ESUS patients may be etiologically associated with several conditions like aortic or carotid atherosclerotic plaques causing low-degree stenosis (ie, <50%), covert AF, atrial cardiopathy including other non-AF supraventricular arrhythmias and structural abnormalities of the left atrium, pathologies of the left ventricle, cardiac valvular pathologies, paradoxical embolism through patent foramen ovale or other shunts, cancer and others.2,20 The HAVOC score includes information only for some of these potential causes, which could offer a plausible explanation for the findings of the present and the CRYSTAL-AF cohort analysis. It may be possible that a prognostic tool incorporating information about these parameters could identify more reliably those ESUS patients who are at low risk for AF.

The strengths of the study include the large number of consecutive, well-defined ESUS patients, and its multicenter design. Limitations include the risk of registration bias within and between the participating registries, the retrospective design of the analysis, and differences in the work-up of patients during the in-hospital phase. In addition, it is possible that some of the identified ESUS patients also had symptomatic intracranial stenosis that was not identified radiographically.

In conclusion, the previously reported low rate of AF among ESUS patients with low-risk HAVOC score was not confirmed in our cohort. Further assessment of the HAVOC score is warranted before it is routinely implemented in clinical practice.

Footnotes

Correspondence to George Ntaios, MD, MSc, PhD, Department of Internal Medicine, Faculty of Medicine, School of Health Sciences, University of Thessaly, Biopolis 41110, Larissa, Greece. Email

References

  • 1. Ntaios G, Hart RG. Embolic stroke.Circulation. 2017; 136:2403–2405. doi: 10.1161/CIRCULATIONAHA.117.030509LinkGoogle Scholar
  • 2. Hart RG, Diener HC, Coutts SB, Easton JD, Granger CB, O’Donnell MJ, et al; Cryptogenic Stroke/ESUS International Working Group. Embolic strokes of undetermined source: the case for a new clinical construct.Lancet Neurol. 2014; 13:429–438. doi: 10.1016/S1474-4422(13)70310-7CrossrefMedlineGoogle Scholar
  • 3. Sanna T, Diener HC, Passman RS, Di Lazzaro V, Bernstein RA, Morillo CA, et al; CRYSTAL AF Investigators. Cryptogenic stroke and underlying atrial fibrillation.N Engl J Med. 2014; 370:2478–2486. doi: 10.1056/NEJMoa1313600CrossrefMedlineGoogle Scholar
  • 4. Gladstone DJ, Spring M, Dorian P, Panzov V, Thorpe KE, Hall J, et al; EMBRACE Investigators and Coordinators. Atrial fibrillation in patients with cryptogenic stroke.N Engl J Med. 2014; 370:2467–2477. doi: 10.1056/NEJMoa1311376CrossrefMedlineGoogle Scholar
  • 5. Wachter R, Gröschel K, Gelbrich G, Hamann GF, Kermer P, Liman J, et al; Find-AF(randomised) Investigators and Coordinators. Holter-electrocardiogram-monitoring in patients with acute ischaemic stroke (Find-AFRANDOMISED): an open-label randomised controlled trial.Lancet Neurol. 2017; 16:282–290. doi: 10.1016/S1474-4422(17)30002-9CrossrefMedlineGoogle Scholar
  • 6. Giruparajah M, Bosch J, Vanassche T, Mattina K, Connolly SJ, Pater C, et al. Global survey of the diagnostic evaluation and management of cryptogenic ischemic stroke.Int J Stroke. 2015; 10:1031–1036. doi: 10.1111/ijs.12509CrossrefMedlineGoogle Scholar
  • 7. Kwong C, Ling AY, Crawford MH, Zhao SX, Shah NH. A clinical score for predicting atrial fibrillation in patients with cryptogenic stroke or transient ischemic attack.Cardiology. 2017; 138:133–140. doi: 10.1159/000476030CrossrefMedlineGoogle Scholar
  • 8. Zhao SX, Ziegler PD, Crawford MH, Kwong C, Koehler JL, Passman RS. Evaluation of a clinical score for predicting atrial fibrillation in cryptogenic stroke patients with insertable cardiac monitors: results from the CRYSTAL AF study.Ther Adv Neurol Disord. 2019; 12:1756286419842698. doi: 10.1177/1756286419842698CrossrefGoogle Scholar
  • 9. Ntaios G, Perlepe K, Sirimarco G, Strambo D, Eskandari A, Karagkiozi E, et al. Carotid plaques and detection of atrial fibrillation in embolic stroke of undetermined source.Neurology. 2019; 92:e2644–e2652. doi: 10.1212/WNL.0000000000007611CrossrefMedlineGoogle Scholar
  • 10. Ntaios G, Perlepe K, Lambrou D, Sirimarco G, Strambo D, Eskandari A, et al. Prevalence and overlap of potential embolic sources in patients with embolic stroke of undetermined source.J Am Heart Assoc. 2019; 8:e012858. doi: 10.1161/JAHA.119.012858LinkGoogle Scholar
  • 11. Michel P, Odier C, Rutgers M, Reichhart M, Maeder P, Meuli R, et al. The Acute STroke Registry and Analysis of Lausanne (ASTRAL): design and baseline analysis of an ischemic stroke registry including acute multimodal imaging.Stroke. 2010; 41:2491–2498. doi: 10.1161/STROKEAHA.110.596189LinkGoogle Scholar
  • 12. Vemmos K, Ntaios G, Spengos K, Savvari P, Vemmou A, Pappa T, et al. Association between obesity and mortality after acute first-ever stroke: the obesity-stroke paradox.Stroke. 2011; 42:30–36. doi: 10.1161/STROKEAHA.110.593434LinkGoogle Scholar
  • 13. Ntaios G, Vemmos K, Lip GY, Koroboki E, Manios E, Vemmou A, et al. Risk stratification for recurrence and mortality in embolic stroke of undetermined source.Stroke. 2016; 47:2278–2285. doi: 10.1161/STROKEAHA.116.013713LinkGoogle Scholar
  • 14. Hart RG, Sharma M, Mundl H, Shoamanesh A, Kasner SE, Berkowitz SD, et al. Rivaroxaban for secondary stroke prevention in patients with embolic strokes of undetermined source: design of the NAVIGATE ESUS randomized trial.Eur Stroke J. 2016; 1:146–154. doi: 10.1177/2396987316663049CrossrefMedlineGoogle Scholar
  • 15. Sacco RL, Kargman DE, Gu Q, Zamanillo MC. Race-ethnicity and determinants of intracranial atherosclerotic cerebral infarction. The Northern Manhattan Stroke Study.Stroke. 1995; 26:14–20. doi: 10.1161/01.str.26.1.14LinkGoogle Scholar
  • 16. Fauchier L, Clementy N, Pelade C, Collignon C, Nicolle E, Lip GY. Patients with ischemic stroke and incident atrial fibrillation: a nationwide Cohort Study.Stroke. 2015; 46:2432–2437. doi: 10.1161/STROKEAHA.115.010270LinkGoogle Scholar
  • 17. Li YG, Bisson A, Bodin A, Herbert J, Grammatico-Guillon L, Joung B, et al. C2 HEST score and prediction of incident atrial fibrillation in poststroke patients: a French Nationwide Study.J Am Heart Assoc. 2019; 8:e012546. doi: 10.1161/JAHA.119.012546LinkGoogle Scholar
  • 18. Li YG, Pastori D, Farcomeni A, Yang PS, Jang E, Joung B, et al. A simple clinical risk score (C2HEST) for predicting incident atrial fibrillation in Asian subjects: derivation in 471,446 Chinese subjects, with internal validation and external application in 451,199 Korean subjects.Chest. 2019; 155:510–518. doi: 10.1016/j.chest.2018.09.011CrossrefMedlineGoogle Scholar
  • 19. Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model.BMJ. 2009; 338:b605. doi: 10.1136/bmj.b605CrossrefMedlineGoogle Scholar
  • 20. Hart RG, Catanese L, Perera KS, Ntaios G, Connolly SJ. Embolic stroke of undetermined source: a systematic review and clinical update.Stroke. 2017; 48:867–872. doi: 10.1161/STROKEAHA.116.016414LinkGoogle Scholar

eLetters(0)

eLetters should relate to an article recently published in the journal and are not a forum for providing unpublished data. Comments are reviewed for appropriate use of tone and language. Comments are not peer-reviewed. Acceptable comments are posted to the journal website only. Comments are not published in an issue and are not indexed in PubMed. Comments should be no longer than 500 words and will only be posted online. References are limited to 10. Authors of the article cited in the comment will be invited to reply, as appropriate.

Comments and feedback on AHA/ASA Scientific Statements and Guidelines should be directed to the AHA/ASA Manuscript Oversight Committee via its Correspondence page.