Machine Learning–Enabled Multimodal Fusion of Intra-Atrial and Body Surface Signals in Prediction of Atrial Fibrillation Ablation Outcomes

This is a retrospective analysis of consecutive adult patients with paroxysmal or persistent AF who underwent catheter ablation between 2015 and 2017 at a tertiary referral center by 5 providers. To be included, patients were required to have panoramic left atrial electrograms recorded before ablation and clinical follow-up for at least 12 months following ablation for accurate assessment of their AF ablation procedure outcomes. All patients had pulmonary vein isolation as a part of the AF ablation procedure; additional ablation lesions per the operating physicians’ discretion were allowed. This comprised of ablation of localized AF sources via focal impulse and rotor mapping (FIRM, in 100% of patients), ablation of left atrial linear lesions (in 24% of patients) and cavotricuspid isthmus ablation (in 27% of patients).

Clinical and demographic data were obtained from electronic medical records. Twelve-lead ECGs in sinus rhythm obtained within 1 year of the ablation procedure were included. Patients with no 12-lead ECG available (n=3) were excluded from the ECG-only model and were imputed with the means of the other patients’ ECG features in the fusion models. This study protocol was approved by the Institutional Review Board of Stanford University. Due to the retrospective nature of the study, no informed consent was required. The corresponding author had full access to all the data in the study and took responsibility for its integrity and the data analysis.

All procedures were performed under general anesthesia. Various ablation catheters were used to achieve PVI, which included point-by-point radiofrequency ablation with a contact force sensing 3.5 mm tip irrigated catheter (Biosense Webster; Abbott) or cryoballoon (Arctic Front, Medtronic). Unipolar panoramic intracardiac signals used for ML analysis were obtained before any ablation with a 64-pole basket catheter (FIRMap catheter, Abbott) during AF. If patients presented to the electrophysiology laboratory in normal sinus rhythm, AF was induced with burst pacing.

Patients were followed up routinely in the outpatient setting, and all had 3-month evaluations for at least 1 year, which included rhythm assessment with 12-lead ECGs at 3 and 6 months and a 14-day event monitor at 1 year. AF recurrence was defined as >30 second duration episodes on ECG monitoring, or >1% AF burden on device interrogation for the patients with implantable monitors. In this study, we focus on the outcome of whether a patient has recurrent AF within 1 year after catheter ablation.

The demographic variables extracted from electronic health records included patients’ age at the time of ablation, sex, height, weight, body mass index, race, and ethnicity. Clinical comorbidities such as presence of hypertension, hyperlipidemia, transient ischemic attack, stroke (CVA), coronary artery disease, diabetes (DM), chronic kidney disease, congestive heart failure, and obstructive sleep apnea were collected. Arrhythmia characteristics such as type of AF (paroxysmal, persistent or long standing persistent), and history of prior AF ablation were recorded. Structural features extracted from imaging studies included left ventricular ejection fraction and left atrial diameter from transthoracic echocardiograms; and left atrial volume, surface area and sphericity index from CT scans that were routinely obtained within 1 year before AF ablation. These variables were selected based on the literature on known factors which could impact AF ablation outcomes.^{3–5,8,13–16} A complete list of clinical features and number of missing values is shown in Table S1. Missing values were imputed with the most frequent value of the feature. Table S2 shows model performance with different missing value imputation techniques, and Table S3 shows model performance in patients without missing values.

As a baseline method, we built a classifier for predicting 1-year AF recurrence from demographic and clinical features. For each patient, a multi-dimensional feature vector was constructed from the clinical and demographic features, where continuous variables were normalized to have zero mean and unit variance and categorical variables were one-hot encoded. We used the categorical boosting (CatBoost) classifier,¹⁷ a state-of-the-art, gradient boosted decision tree-based ML algorithm, for AF recurrence prediction. Briefly, CatBoost sequentially builds many weak learners (ie, decision trees) and creates a strong predictive model by greedy search and ensembling. We chose CatBoost because it has been shown to outperform other gradient boosted decision tree-based algorithms and naturally handles both continuous and categorical variables.¹⁷

In each patient, unipolar left atrial intracardiac electrograms were recorded during AF. Unipolar signals were recorded from a 64-pole basket catheter positioned in the mid left atrium (LA) before any ablation were exported. Preprocessing of electrogram signals included QRS subtraction and resampling to 200 Hz. See Supplemental Methods for details.

Preprocessing of ECG signals included a bandpass filtering of 0.05 to 100 Hz and resampling to 200 Hz. Eight independent ECG channels were used (channels I, II, and V1-6) as any linear dependency can be naturally learned by deep neural networks (ie, channel III can be derived vectorially from channels I and II).

Each electrogram and ECG signal was augmented by dividing into 5-sec windows with a 4-sec overlap between consecutive windows, resulting in a 1000×64 matrix for each input electrogram data point and a 1000×8 matrix for each input ECG data point.

We developed a convolutional neural network (CNN) for predicting 1-year AF recurrence from electrogram or ECG signals.

Similar to Attia et al,¹⁴ our CNN consisted of several layers of bottleneck blocks with 1-dimensional (1D) convolutions operating on the time dimension, followed by a 1D convolutional layer operating on the channel dimension. Intuitively, the time-dimension convolutional layers capture the temporal dependency in the signal by extracting features from signals within one channel, whereas the final channel-dimension convolutional layer aggregates the features across channels to obtain a spatial representation of the signal. Details of the CNN can be found in Supplemental Methods and Figure S1.

Finally, we developed a multimodal fusion framework that leverages more than one modality to improve the prediction of AF recurrence (Figure 1B).

First, electrogram features were extracted from the CNN that was trained on electrogram signals only. All the features from the electrogram signals from the same patient were averaged to obtain a single electrogram feature representation for each patient. ECG features for each patient were extracted in a similar way. Next, for each patient, the feature vectors of the fused modalities (ie, electrogram features, ECG features, and clinical features) were concatenated to form a multimodal feature vector. Last, a classifier was trained on the patients’ multimodal feature vectors for predicting 1-year AF recurrence. As a fair comparison to clinical feature-based models, we also applied the CatBoost¹⁷ classifier in the fusion framework.

As ablation experiments, we also validated fusion of 2 modalities (ie, electrogram and clinical features, ECG and clinical features, or electrogram and ECG features) and compared the results to fusion of 3 modalities (electrogram, ECG, and clinical features).

Stratified 10-fold cross-validation (patient-wise split) was used to train and validate each of the models described above. Specifically, all patients were randomly divided into 10 groups (ie, folds) with the same proportion of AF recurrence in each fold (ie, stratified 10-fold). At the i-th cross-validation step, the i-th fold was used to test the model and the remaining 9 folds were used to train the model. This above process was repeated 10 times, such that each patient only appeared in one of the test folds.

To mitigate overfitting, data augmentation was applied during training. We designed 5 data augmentation methods using electrophysiology domain knowledge: (1) randomly shift (forward or backward in time) each 5-sec window by up to 2.5-sec, (2) randomly scale the raw signal by a factor within range 0.5 to 2, (3) randomly shift the DC value within range −10 to 10 microvolts, (4) randomly masking with zeros for up to 25% of the 5-sec window, (5) randomly add gaussian noise with zero mean and a SD<0.2. Importantly, these data augmentations did not result in invalid signals but naturally increased the variability of the training data, which could mitigate overfitting of deep neural networks.

Training for the CNNs on electrogram and ECG signals was accomplished using the Adam optimizer¹⁸ in PyTorch on a single NVIDIA P100 GPU. For CNNs, we followed the same model architecture configuration as that in Attia et al¹⁴ (except for reducing the number of bottleneck blocks from 9 to 6 in ECG-based CNN) and did not tune the model hyperparameters. Training for CatBoost was done using the CatBoost Python package,¹⁷ and CatBoost hyperparameters were tuned using grid search (see Supplemental Methods for details). All models were trained to optimize AUROC. We assessed the model’s ability to predict 1-year AF recurrence using AUROC, sensitivity, specificity, accuracy, and F1-scores. To derive sensitivity, specificity, accuracy, and F1-scores, a probability threshold was selected based on the highest F1-score on the 10 fold test sets.

For population characteristics, continuous data are reported as mean±SD, unless otherwise stated, and are tested for normality using the Shapiro-Wilk test (P>0.05). Independent samples t test and Mann-Whitney U test were run to determine if there were differences in mean values between cohorts for analysis of continuous data. Categorical variables were compared using the Pearson χ² test or Fisher exact test where expected frequencies were <5. For model evaluation, we report the mean and SD of AUROC, sensitivity, specificity, accuracy, and F1-scores of the 10-fold test results. In addition, we measure the calibration of the models using Brier score¹⁹ and expected calibration error (ECE).²⁰ Briefly, the Brier score measures the mean squared difference between the predicted probability assigned to the possible label and the actual label. The ECE approximates the expectation between model confidence and accuracy by binning the predictions into equally-spaced bins and taking a weighted average of the bin’s accuracy and confidence difference. For both Brier score and ECE, lower values indicate better calibrated models. A statistical significance threshold (α) of 0.05 was used for all the reported tests.

Between 2015 and 2017, 226 consecutive AF ablations were done using a 64-pole basket catheter that recorded simultaneous panoramic unipolar electrograms from the left and the right atria. Of these, 161 had left atrial signals recorded before any ablation. Five were excluded due to poor signal quality, leaving 156 patients to be analyzed for this study. Baseline characteristics of these patients are shown in Table 1. PVI was done using radiofrequency in 118 patients (76%), cryoballoon in 38 patients (24%). Thirty-four patients (21.8%) were on an antiarrhythmic drugs (AADs) at the time of follow up (10.2% on class IC agents, 3.9% on class III agents [sotalol or dofetilide], 8.3% on amiodarone and 1.9% on dronedarone). Additional ablation lesions beyond PVI and ablation of localized sources are presented in Table 1.

On follow-up at 1 year, 112 (72%) patients remained free of AF. Patients with and without recurrence had a similar age, body mass index and comorbidities (Table 1). AAD use was not different among groups. 28% of the patients had a prior history of AF ablation. Presence of hyperlipidemia and diabetes correlated with AF recurrence (P=0.04) in univariate analysis. Ablation of additional left atrial lines did not correlate with AF ablation outcomes.

First, we validated 2 existing clinical feature-based prediction scores, APPLE³ and CHA2DS2-VaSC,⁴ for 1-year AF recurrence prediction using CatBoost.¹⁷ Detailed formulation of APPLE and CHA2DS2-VaSC scores can be found in Supplemental Methods.

The CatBoost classifier achieved an AUROC of 0.644 (SD=0.129) on APPLE scores and an AUROC of 0.650 (SD=0.133) on CHA2DS2-VASc scores (Table 2, first and second rows).

Using clinical features, the CatBoost classifier achieved an AUROC of 0.755 (SD=0.093; Table 2, third row), outperforming the performance of the CatBoost classifier trained on APPLE and CHA2DS2-VASc scores. This performance improvement is expected given that multiple clinical features were used, whereas APPLE and CHA2DS2-VASc scores only accounted for 5 and 7 clinical features, respectively.

Figure 2 shows the model interpretation of the clinical features that contribute the most to AF recurrence prediction in our clinical feature-based model, where the 5 most important features are left ventricular ejection fraction, height, body mass index, weight, left atria volume from CT, and left atria surface area; which have previously been reported to correlate with development of incident AF^15,21 or poorer outcomes following AF ablation.^16,22

Using electrogram signals only, the CNN achieved an AUROC of 0.731 (SD=0.105) for AF recurrence prediction (Table 2, 4th row); using ECG signals only, the CNN achieved an AUROC of 0.767 (SD=0.122; Table 2, 5th row), both of which outperform APPLE and CHA2DS2-VASc scores.

In addition, we visualize examples of electrogram and ECG learned by the CNNs using the Uniform Manifold Approximation and Projection²³ (UMAP) dimensionality reduction technique. As shown in Figure S2, the same patient’s electrogram features are clustered together, whereas different patients’ electrogram features are further apart. Moreover, electrogram/ECG features of patients with AF recurrence are further away from features of patients without AF recurrence, suggesting that the CNNs are able to learn distinct patterns in patients with different outcomes.

Our final fusion model that combines electrogram, ECG, and clinical features achieved an AUROC of 0.859 (SD=0.082; Table 2, last row), outperforming the APPLE scores, CHA2DS2-VASc scores, and ECG or electrogram signals alone, suggesting the effectiveness of our fusion framework.

Figure 3 shows the ROC curves of the clinical feature-based models, the signal-based CNN models, and the fusion model. At a low false positive rate, such as 20% false positive rate, our fusion model had a true positive rate (TPR) of 80%, which translates clinically to missing 20% recurrent AF patients with 20% of the predicted recurrent AF being false positives. In contrast, the CHA2DS2-VASc score-based classifier and the clinical feature-based classifier only achieved a TPR of 40% and 58%, respectively, which translates to missing 60% and 42% recurrent AF patients, respectively, with the same number of false positives.

Moreover, combining 2 modalities performed better than single modalities (Table 2, 6th–8th rows), which is intuitive given that 2 modalities encode additional features than a single modality. Model performance in various subgroups are provided in Supplemental Results and Tables S4 through S6.

In addition to discriminative measures (eg, AUROC, sensitivity, and specificity), we evaluate the calibration of the models using Brier score¹⁹ and expected calibration error.²⁰ See Supplemental Results and Table S7 for details.

This study was performed at a single center, involves a small cohort with underrepresentation of women, and results have not been validated externally. Heterogeneity in ablation approaches may limit generalizability of the findings to specific ablation strategies. Despite this limitation, all the patients underwent PVI, and evidence of benefit of further ablation beyond PVI, including linear ablation and ablation of sites of organized rotational or focal activation, has not been proven consistently in multicenter randomized studies.^24,29,30 All patients in this study underwent FIRM mapping and ablation that formed the basis of the unipolar EGMs used in the model. The necessity of the use of FIRM mapping is a limitation to this study, as this is not a widely used catheter or mapping strategy in the community.

Freedom from AF appears higher for a mixed cohort of patients but is consistent with other studies that used intermittent monitoring rather than implanted loop recorders. Intermittent monitoring of AF recurrence with 12-lead ECGs and 14-day event monitors likely underrepresents true AF recurrence, which could affect the accuracy of our predictive model. The retrospective nature of the data limited strict guidelines over AAD use in follow up, that is, for certain patients, preprocedure AADs were continued postablation due to patient or provider preference regardless of procedure outcome. Twenty-eight percent of patients had prior AF ablation, which may have impacted intracardiac signal characteristics. Twelve-lead ECGs in sinus rhythm before ablation were not available in all patients. When a patient’s 12-lead ECGs in sinus rhythm before ablation was not available, a 12-lead ECG in sinus rhythm immediately after ablation was used for analysis, which could result in bias in analyses. While we show that when evaluating the trained models on patients whose preablation 12-lead ECGs are available, the model performance did not differ significantly from our original analysis (ie, post-ablation 12-lead ECGs are used for patients whose preablation ECGs are not available), we did not re-train the models on preablation ECGs only due to the limited size of our cohort (n=107 patients with preablation ECGs). Majority of these patients had a 12-lead ECG in sinus rhythm before ablation that was not performed at our center in an electronic format that could be exported for analysis, due to the tertiary referral center status where the study was conducted. Some of the data that were used in the models to predict ablation success, including intracardiac signals, are obtained at the time of ablation, and may not help in patient selection for ablation procedure, but can rather guide medical management and expectations following the procedure. Furthermore, while we show that most of the trained models perform similarly on patient subgroups (patients with paroxysmal versus nonparoxysmal AF; patients with cryoablation versus radiofrequency ablation), future study with a larger cohort that trains models on these subgroups independently is needed to further compare these subgroups. Last, while we show that our CNNs and fusion model are better calibrated than the existing APPLE and CHASDS2-VASc scores (Table S7), the Brier scores and expected calibration errors are still relatively high; advanced calibration techniques³¹ for deep neural networks need to be incorporated in the future to produce better calibrated models.

Our machine learning approach provides an automatic technique to predict freedom from atrial arrhythmias in patients undergoing AF ablation, outperforming traditional scoring systems. Larger datasets are needed in the future to train and validate this approach even further to help develop personalized ablation strategies for patients with AF.

Supplemental Methods

Supplemental Results

Tables S1–S7

Figures S1 and S2

References^32–35