Artificial Intelligence-Enabled ECG Algorithm to Identify Patients With Left Ventricular Systolic Dysfunction Presenting to the Emergency Department With Dyspnea

Introduction

See Editorial by Haq et al

Dyspnea is a frequent complaint in the emergency department (ED) accounting for ≈1.2 million annual ED visits in the United States.¹ Ascertaining the cause of dyspnea can be challenging given the variability in clinical symptoms and presentation. Heart failure is a common cause of cardiac-related dyspnea among ED patients, and accurate identification of these patients is essential for acute management and final disposition. Heart failure is also one of the most common diagnoses assigned to ED patients who become hospitalized.²

Identification of left ventricular systolic dysfunction (LVSD) in the ED can effectively diagnose heart failure and appropriately guide management in the acute setting. A left ventricular ejection fraction (LVEF) ≤35% is associated with increased mortality in patients with and without heart failure.^3–5

We hypothesized that the application of an artificial intelligence-enabled ECG (AI-ECG) algorithm may provide for a rapid and inexpensive means to effectively identify LVSD in patients presenting to the ED with dyspnea. This AI-ECG algorithm has been previously demonstrated to be effective in identification of LVSD.⁶ To test this hypothesis, we performed a retrospective study of patients presenting to the Mayo Clinic ED with acute dyspnea.

Methods

The data, analytic methods, and study materials that support the findings of this study are available from the corresponding author upon reasonable request.

Study Population

We identified adult patients 18 years and older (n=21 309) presenting to the ED at all Mayo Clinic sites (AZ, FL, and MN) and Mayo Clinic Health Systems with a complaint of dyspnea between May 2018 and February 2019 who had at least one standard 10-second 12-lead ECG performed within 24 hours of ED visit. Reported dyspnea was identified based on International Classification of Diseases, Ninth Revision(ICD-9) codes: 786.0 (including 786.00–786.09) and ICD-10 codes: R06 (R06.00–R06.9). For patients with multiple ED visits, the first ED visit was selected as the index visit. The first ECG performed within 24 hours of index ED visit was selected when multiple ECGs were available. From this cohort, we subsequently identified patients who had a comprehensive 2-dimensional echocardiogram performed within 30 days of index ED visit. All ECGs were acquired at a sampling rate of 500 Hz using a GE Marquette ECG machine (Marquette, WI) and stored using the MUSE ECG data management system (GE Healthcare, Chicago, IL). Quantitative data from echocardiography performed are recorded at the time of the acquisition in a Mayo Clinic developed, custom database (Echo Image Management System, Rochester, MN). LVEF was estimated using standard methods recommended by the American Society of Echocardiography.⁷

Exclusion

We excluded patients with a known prior diagnosis of systolic, diastolic, or unspecified heart failure using ICD-10 diagnosis codes I11, I13, I50, I27.22, and Z95.811 (n=4812) within 2 years before index ED visit. We also excluded patients without an echocardiogram within 30 days of index ED visit (n=14 072), prior echocardiogram demonstrating LVEF ≤35% (n=20), ECGs not performed on the date of ED visit (n=101), ECGs not readily accessible in MUSE (n=444), and those without readily accessible echocardiogram accession numbers (n=254). Our final sample size included 1606 patients (7.5% of patients meeting inclusion criteria; Figure 1). The study was approved by the Mayo Clinic Internal Review Board including a waiver of informed consent.

• Download figure
• Download PowerPoint

Primary and Secondary Outcomes

Our primary outcome was the identification of patients with new LVSD defined as LVEF ≤35% within 30 days of the ED visit using a deep learning network for ECGs performed at the time of ED visit. Our secondary outcome was the identification of LVEF EF <50% within 30 days of the ED visit using the same deep learning network. For diagnostic accuracy assessment, the gold standard was LVEF as measured on a 2-dimensional echocardiogram.

AI Model

We used a previously described AI-ECG algorithm developed and validated for identification of LVEF ≤35%^6,8 with no additional training or optimization. This algorithm used a convolutional neural network trained with Keras with a Tensorflow (Google, Mountain View, CA). Details of the algorithm derivation have been previously published.^6,8 There was no overlap in the study data used and published in Nature Medicine.

Statistical Analyses

The primary global measure of model performance for this study was the area under the receiver operating characteristics curve (AUC) formed by modeling the AI-ECG algorithm prediction of the probability of LVSD in relationship to the clinically determined diagnosis of LVSD within 30 days of the ED presentation. Routine measures of diagnostic performance based on dichotomized predictions (eg, sensitivity and specificity) were obtained using a previously determined threshold of ≥0.256 to indicate a positive screen. Ninety-five percent exact CI were calculated for all measures of diagnostic performance except for AUC. In this case, the large sample approximation of the DeLong method with optimization by Sun and Xu was used.^9,10 The AI-ECG algorithm’s performance was tested against NT-proBNP (N-terminal pro-B-type natriuretic peptide) on subjects that had NT-proBNP measured at time of ED visit. Summaries for the AUC of NT-proBNP alone and in combination with the AI-ECG algorithm were developed. The incremental benefit of the AI-ECG prediction of LVSD was tested using the DeLong test.

To provide some general description about comorbidities the sample had at time of ED presentation, the following approach was taken. Using standardized code sets of ICD-9 and ICD-10 codes per diagnosis, the Mayo Clinic Unified Data Platform was queried for the presence of at least one code within the code set within 30 days post the ECG. If a single code was found, the patient was considered positive for the condition. Statistical analyses were computed using R version 3.6.2.

Results

Study Population Characteristics

A total of 1606 patients were included. Overall, the median age of patients evaluated in the ED for dyspnea was 68 years, approximately half were female (47%), and the majority were white (91%). The median time to echocardiogram following ED visit was 1 day (Q1: 1, Q3: 2). Only 54% of patients had NT-proBNP levels assessed, 43% had high-sensitivity troponin values, and 63% had serum creatinine at the index ED visit (Table).

Left Ventricular Ejection Fraction ≤35%

Utilization of an AI-ECG algorithm for identification of new LVSD among ED patients presenting with dyspnea achieved an AUC of 0.89 (95% CI, 0.86–0.91) with an accuracy of 85.9% (95% CI, 84.1%–87.6%). Sensitivity, specificity, negative predictive value, and positive predictive value were 74%, 87%, 97%, and 40%, respectively (Figure 2).

• Download figure
• Download PowerPoint

Left Ventricular Ejection Fraction <50%

For our secondary outcome, the AI-ECG algorithm achieved an AUC of 0.85 (95% CI, 0.83–0.88) with an accuracy of 86% (95% CI, 84.2%–87.7%). Sensitivity, specificity, negative predictive value, and positive predictive value were 63%, 91%, 92%, and 62%, respectively (Figure 3).

• Download figure
• Download PowerPoint

Subpopulations

The AI-ECG algorithm appeared to have slightly better performance characteristics for identification of LVSD (EF≤35%) in younger (AUC=0.91) and female (AUC 0.90) patients, albeit with less precision (Figure 4). Overall, the AI-ECG algorithm’s diagnostic accuracy was similar across the subgroups examined. We also evaluated its performance for identification of EF<50% and its performance was similar across patient subgroups (Figure 5).

• Download figure
• Download PowerPoint

• Download figure
• Download PowerPoint

Comparing AI-ECG Versus NT-proBNP in the ED

In a subsample of our patient population who had NT-proBNP values available (54%, n=866), NT-proBNP alone at a cutoff value >800 identified new LVSD (EF≤35%) with an AUC of 0.80 (95% CI, 0.76–0.84) demonstrating a superior diagnostic value of a single AI-ECG algorithm in this patient cohort (P<0.0001). Addition of NT-proBNP to the AI-ECG algorithm added a marginal incremental value with improvement in AUC from 0.89 to 0.91 (P=0.091).

Thirty-Day Clinical Outcomes

Patients with new LVSD identified by echocardiography were significantly more likely to be rehospitalized with heart failure (32% versus 10%, P≤0.001). There was, however, no difference in 30-day all-cause rehospitalizations or repeat ED visits.

Discussion

In this study, we demonstrate that an AI-ECG algorithm can be an effective tool for rapid detection of LVSD in patients presenting to the ED with acute dyspnea with an AUC of 0.89. Our study provides evidence to support real-world application of an AI-ECG algorithm in routine clinical practice.

This study is particularly important as its goal is to identify patients with significant depression in LV systolic function in the ED. Early identification of these patients provides a potential opportunity for linkage to essential cardiovascular care for appropriate management, follow-up diagnostic testing as appropriate, medication optimization, and device-based therapies such as implantable cardioverter-defibrillator and cardiac resynchronization therapy.⁴ The ECG is inexpensive, ubiquitous, painless, quickly obtained, and can be performed with minimal training. The American College of Cardiology/American Heart Association guidelines identified dyspnea as the most common angina equivalent and recommend obtaining a 12-lead ECG in all adult patients presenting to the ED with dyspnea,¹¹ which makes the ECG an appropriate screening tool. The cost of the ECG compared with an echocardiogram likely makes it more cost-effective for screening patients in the ED for LVSD.

Although not all patients with cardiac-related dyspnea have LVSD. Some patients may present with dyspnea secondary to heart failure with preserved EF, angina presenting as dyspnea, or pulmonary edema due to acute valvular heart disease or hypertensive crisis. However, given the high mortality risk in patients with significant LVSD (EF≤35%),⁴ identifying this subgroup in the ED provides a unique avenue for early linkage to cardiovascular care and device-based therapies.

Echocardiography is considered the ideal noninvasive test for evaluation of left ventricular systolic function,⁷ but this procedure is highly operator dependent, requires significant training, and may not be readily available in the ED.¹² Formal echocardiography requires review and interpretation by a trained cardiologist which takes considerable time and may not be feasible for making acute care decisions.

Studies have examined various point-of-care tests for differentiating between cardiac and noncardiac causes of dyspnea in the ED. Natriuretic peptides are by far the most widely used and the only biomarkers recommended by the American College of Cardiology/American Heart Association heart failure guidelines (Class IA) for initial evaluation of heart failure in patients presenting with dyspnea.¹³ The Breathing Not Properly study was the landmark clinical trial that established the utility of BNP (B-type natriuretic peptide) values in identifying patients presenting to the ED with heart failure-related dyspnea.¹⁴ The final diagnosis of heart failure in this study was adjudicated based on a combination of clinical symptoms, physical examination findings, and diagnostic tests reviewed by cardiologists. However, only 77% of those deemed to have heart failure-related dyspnea had echocardiograms performed.¹⁴ Several other studies have also demonstrated the utility of point-of-care BNP and NT-proBNP testing in the acute care setting.^15–21 However, the misclassification rate with BNP testing has been reported to be as high as 14% to 29%.² Multiple factors are also known to affect natriuretic peptide levels, such as obesity, age, chronic kidney disease, hemodialysis, pulmonary hypertension, sepsis, chronic atrial fibrillation, and ARNI (angiotensin receptor–neprilysin inhibitor).^13,22–27 In addition, the measurement of BNP has not been shown to have any effect on clinical outcomes or medications administered.^13,28 Our study demonstrated the AI-ECG algorithm outperforms NT-proBNP alone for identifying patients with new LVSD (AUC 0.89 versus 0.80).

Other rapid diagnostic modalities previously evaluated in the ED or acute care setting for evaluation of dyspnea include: chest radiograph,²⁹ inferior vena cava diameter on ultrasound,³⁰ lung ultrasound,^29,31–33 partial pressure of end-tidal CO2,³⁴ bioimpedance,³⁵ plasma volume status,³⁶ and focused cardiac ultrasound,³⁷ all with associated limitations. Making a diagnosis of acute heart failure in the ED using a combination of history, physical examination, chest radiograph, and ECG was noted to be discordant with the final diagnosis 25% of the time.² The use of cardiopulmonary ultrasound in the ED has been shown to be superior to a combination of clinical examination, NT-proBNP, and chest radiograph for establishing the cause of acute dyspnea³⁸; however, performance of a cardiopulmonary ultrasound also requires additional training in ultrasonography. A meta-analysis of various studies using these rapid diagnostic modalities found bedside lung ultrasound and echocardiography to be the most useful tests for confirming acute heart failure while use of natriuretic peptides were more effective for diagnostic exclusion.²

Clinical diagnoses made in the ED often remain in patient’s medical records even if the final diagnosis changes with the availability of additional imaging tests or upon further evaluation by the admitting or consulting physician. An analysis of the National Hospital Discharge Survey noted the number of hospitalizations with any mention heart failure tripled between 1979 and 2004. However, only 30% of these patients had heart failure listed as the first or primary diagnosis,³⁹ suggesting that heart failure may not be the main clinical diagnosis or may not be acute (ie, chronic heart failure). Therefore, the additional information provided by an AI-ECG algorithm may assist in improving diagnostic accuracy and documentation in the ED and patient disposition. While not assessed in this work, we intend to study this prospectively. In addition, AI-ECG may be particularly helpful and potentially add incremental valuable information as a screening tool in community EDs without ready access to cardiologists or echocardiography.

We also observed that patients with LVSD were significantly more likely to be rehospitalized with heart failure. These findings suggest that the use of an AI-ECG algorithm could potentially identify patients at risk for repeat heart failure hospitalizations and provides a unique opportunity to implement specific interventions to prevent this while in the ED including early follow-up with a cardiologist, initiation of guideline-directed medical therapy, and social work if needed.

A recent analysis of Medicare beneficiaries revealed a diagnosis of heart failure had the highest preventable healthcare spending among elderly patients in the acute care setting above bacterial pneumonia, urinary tract infections, and diabetes mellitus complications.⁴⁰ In the United States, health care costs for patients with heart failure are projected to increase from $20.9 billion in 2012 to $53.1 billion by 2030.⁴¹ As such, it is imperative that patients with heart failure-related dyspnea are accurately and efficiently identified in the ED and appropriate therapies initiated early to potentially reduce associated health care costs due to readmissions related to delays in appropriate cardiovascular care.

Limitations

Our study used ICD diagnosis codes for excluding patients with prior heart failure as such, some patients may have been missed or inappropriately excluded. In addition, we only included patients who had a confirmatory echocardiogram performed within 30 days of index ED visit but may have inadvertently excluded patients with new heart failure who did not have a follow-up echocardiogram within 30 days or those who had an echocardiogram performed at a different facility. The omission of the confirmatory echocardiogram required to diagnosis LVSD may result in biased measures of positive and negative predictive value given the true underlying disease prevalence may be different from what was observed in the analysis.

Strengths

A major strength of this study is the ability to utilize an existing, rapid, noninvasive test-the ECG, to provide valuable additional information in the care of patients with dyspnea in the ED. This study adds to the growing body of literature demonstrating practical applications of AI-based algorithms in the field of cardiovascular medicine, such as rapid determination of implantable cardiac device type and model using images from a chest radiograph,⁴² heart failure mortality risk prediction,⁴³ and prognostication using cardiopulmonary exercise testing in heart failure.⁴⁴

Conclusions

An AI-ECG algorithm was able to identify LVSD with high accuracy in patients presenting to the ED with dyspnea. The application of an AI-ECG algorithm in the ED could improve diagnostic accuracy, facilitate appropriate disposition, and provide an avenue to identify high-risk patients early and link them to appropriate cardiovascular care. Prospective studies are needed to further evaluate the effectiveness of this algorithm, practicality, effect on real-time improvement in diagnostic evaluation, cost-effectiveness, and its association with long-term clinical outcomes.

Footnotes