Chess and Coronary Artery Ischemia: Clinical Implications of Machine-Learning Applications

In December of 2017, DeepMind—an artificial intelligence company owned by Alphabet—released a software program called AlphaZero, a machine-learning (ML) approach to playing the game of chess.¹ Although chess-playing artificial intelligence computers have been showcased for >2 decades before the release of AlphaZero—beginning with IBM’s DeepBlue notorious win against Grandmaster chess champion Garry Kasparov in 1996—the excitement surrounding AlphaZero was for its distinct approach from that taken by DeepBlue, which allowed AlphaZero to master chess with incomparable rapidity and efficiency.² Within 1 day of its release, AlphZero realized superhuman levels of chess play, capable of beating both humans as well as championship chess software programs, such Stockfish, elmo, and AlphaGo Zero. Notably, AlphaZero was not taught through review of prior games, or by books on chess strategy, or by championship players; but rather learned to play chess by simply self play. After only 4 hours of this self play training, AlphaZero triumphed over Stockfish 8—the leading championship chess software—in a 100-game match, where it won 28, lost 0, and drew 72 games.³ During these games, AlphaZero was allowed to think for up to 1 minute per move and achieved its efficiency in decision making by simultaneous searching of >800-fold fewer moves than traditional software algorithms and counterbalancing this parsimony through the use of deep neural networks for better selection of promising moves and a particularly aggressive style of play.

Similar to chess, there has been much discussion of late of the ability of artificial intelligence to disrupt healthcare, with much of the positive speculation by technology experts. As an example, Vinod Khosla, venture capitalist and founder of Sun Microsystems, presages that “by 2025, 80% of the functions doctors do will be done much better and much more cheaply by machines and machine learned algorithms.”⁴ These types of predictions often focus on medical imaging—given the particular proclivity of ML algorithms to excel at computer vision problems—with Geoffrey Hinton, a Google scholar and the godfather of deep learning, intimating that ML algorithms will soon be superior to physicians for medical imaging and that “we should stop training radiologists right now.”⁵ Although only time will tell whether these predictions will hold true, there is certainty that there is an exponentially growing interest in applications of ML in medical imaging—a field wherein computers are allowed to learn without being explicitly programmed–with a >25-fold increase in the annual number of peer-reviewed published papers that involve ML and medical imaging in the past 10 years alone.

One such article is the study published by Coenen et al⁶ in this issue of the Circulation: Cardiovascular Imaging, a large, well-performed 5-center international study of patients who underwent coronary computed tomographic (CT) angiography, invasive coronary angiography, and invasive fractional flow reserve (FFR). Coenen et al⁶ compared the diagnostic performance of a noninvasive method for determining FFR from CT by ML (ML-based CT-FFR) against a more traditional—albeit still generally new—approach that determines FFR from CT using computational fluid dynamics (CFD-based CT-FFR). In this study of 351 patients for which 525 vessels were directly interrogated by FFR, they observed a vessel-based ischemia prevalence of 47%. Compared with CFD-based CT-FFR, ML-based CT-FFR performed well, with a correlation of 0.997, and similarly outperforming measures of high-grade coronary stenosis severity at the 50% stenosis threshold by CT alone for determining vessel-specific ischemia; both ML-based and CFD-based CT-FFR exhibited an area under the curve of 0.84. At the per-patient and per-vessel level, the overall diagnostic accuracy for ischemia diagnosis was 85% versus 71%, respectively, and 78% versus 58%, respectively, for ML-based CT-FFR versus CT. The investigators further noted that ML-based CT-FFR allowed correct reclassification of 73% (62 of 85) of false-positive CT interpretations by increasing diagnostic specificity.

In the past several years, FFR derived from CT has been a topic of great interest in the field of cardiac imaging, with numerous software approaches reported.^7–9 Among these, the method that has been investigated the most has been an off-site software-as-a-service approach that uses 3-dimensional CFD methods to estimate rest and hyperemic pressure across the coronary vascular bed using a supercomputer (FFR_CT, HeartFlow, Redwood City, CA). In 3 multicenter clinical trials, per-patient accuracy of FFR_CT compared with invasive FFR ranged between 73% and 81%, with the most recent software version requiring up to 4 hours for FFR_CT analysis.^10–12 In an effort to increase efficiency and workflow, a more contemporary on-site software CFD solution has been developed that determines FFR from CT using reduced order CFD modeling, and which can determine FFR from CT in 30 minutes using a standard desktop computer (Siemens Healthineers, Erlangen, Germany).¹³ Most recently, an FFR algorithm has been developed that determines the FFR from CT using ML methods. The rationale behind this development is to reduce the requirements for human input and to improve the efficiency and accuracy of FFR derived from CT by integrating an array of imaging features that are not generally accounted for in traditional clinical CT interpretation. These features total 28 in number and include not only the % stenosis of a coronary lesion but also vital angiographic parameters that upstream of a stenosis, downstream of a stenosis, as well as global in nature. Examples of upstream coronary variables include the % diameter stenosis, length, and minimum radius of the most significant stenosis upstream—to account for diffuse atherosclerosis proximal to a lesion that may contribute to its ischemia-causing nature—while downstream lesion features include such factors as the total ischemic contribution score wherein a lesion can be generally mapped to the amount of myocardium it subtends. The latter requires the input of global variables, such as total left ventricular mass, which will allow for proper fractionation of the myocardium in accordance to vessel size. Although it is possible for a human reader to easily identify these features, their measurements are time intensive, and the ability to integrate the multitude of features into a binary output (eg, ischemic versus nonischemic vessel) is nearly impossible. As such, the application of ML to this particular clinical question is highly pertinent.

The authors should be congratulated for their efforts because they serve as leaders in fostering the transition of imaging analytics from traditional to more contemporary methods in a manner that may improve efficiency, reduce computational complexity, and maintain high diagnostic accuracy. By performance of their study in a large multinational registry, they further establish the generalizability of their technique, which offers the prospect of near-instantaneous calculations of FFR from CT in an unbiased fashion. As the authors have noted, however, coronary artery segmentation—which is required for both the CFD-based and ML-based CT-FFR—took readers 30 to 60 minutes of image analysis time to enable processing for FFR, which is likely preclusive for widespread daily clinical adoption.⁶ As this present study employed site readers (rather than core laboratory readers), this amount of time may represent a ceiling for analytic times that may become shorter as readers become more adept. These findings nevertheless substantiate the need for improved solutions for extracting the coronary artery lumen in a rapid manner for widespread clinical use.

That the discriminatory performance achieved by the ML-based CT-FFR—as represented by areas under the receiver operating characteristics curve—was identical to the CFD-based CT-FFR is generally unsurprising as it simply highlights the effectiveness of state-of-the-art ML algorithms to identify strong patterns embedded within the sufficiently large data set of the MACHINE consortium study (Machine Learning Based CT Angiography Derived FFR: A Multi-Center Registry) highlighted in this issue of the Journal.¹⁴ However, it is unlikely that this particular version of ML-based CT-FFR will ever exceed the performance of CFD-based FFR as it was trained through a supervised learning approach to simply emulate CFD-based FFR rather than be better than it. This is not to say that ML-based CT-FFR performance cannot be improved. Indeed, future generations of ML-based CT-FFR softwares hold the potential to substantially improve on the current performance by increasing the quantity, as well as the quality of the input features to the ML algorithms. At present, ML-based CT-FFR has been trained on 12 000 coronary arterial geometries through examination of 28 different angiographic variables. In recent years, however, an array of high quality prospective clinical studies have evaluated CT variables beyond luminal features alone and have identified atherosclerotic plaque characteristics (APCs) to be robust indicators of the ischemia-causing nature of coronary artery lesions.^14,15 These APCs include aggregate plaque volume (plaque burden), as well as high-risk plaque features (such as low attenuation plaque, positive arterial remodeling, and spotty calcifications). These APCs enable robust determination of coronary artery ischemia and, if coupled to the 28 angiographic variables used in the present ML-based CT-FFR, may allow for exchange of a relatively sparse feature set to a rich one that is replete with information about coronary atherosclerosis and its effects on the coronary lumen. Future efforts integrating angiographic measures of coronary artery disease with APCs into a single ML algorithm now seem warranted.

Given the generally large size of the present study, this reader is hopeful that there will be an opportunity to evaluate the prognostic significance of their findings. At present, only early reports have been published that have evaluated the ability of CT-FFR to offer measures of risk stratification above and beyond conventional coronary stenosis.¹⁶ Whether CT-FFR can effectively prognosticate outcomes in this larger study—or whether it is additive beyond APCs for risk stratification—remains unknown and an important consideration point for early adoption of the technology. Furthermore, the combination of high diagnostic performance with prognostic risk stratification of an ML-based CT-FFR may allow for guidance to therapeutic decision making, and the ability of this software solution to do so in a manner that improves event-free survival represents a large and exciting opportunity for these investigators who are leading the way in early technology adoption.

As with all studies, this investigation has limitations that should be noted. Similar to all of the studies that have preceded it, this study comprised patients undergoing cardiac catheterization with clinically indicated invasive FFR. As such, a bias of selection is present, and the results of this study applied to all comers undergoing noninvasive CT imaging should be done with caution. Given the population, the prevalence of vessels exhibiting ischemia approached ≈50%, and the diagnostic accuracy of the ML-based CT-FFR in populations with a lower prevalence should be carefully considered. Furthermore, it is noteworthy to point out ML-based CT-FFR is dependent on high image quality. Similar to rates reported for CFD-based FFR, 16% of the patients were excluded from analysis, with approximately half of these because of inadequate CT image quality.^10–12 These findings highlight the importance of maintaining excellent image quality, which is required to achieve accurate postprocessing algorithms. Finally, it would have been interesting if the authors had reported out the weighted contribution of the 28 angiographic factors that contributed to the likelihood of a vessel exhibiting ischemia or not. Contemporary algorithms allow for determination of the weighted importance of any given feature, and these data would be helpful to clinicians who may consider medical therapy versus revascularization of an individual based on the algorithmic output. In this regard, future evaluation of the current data set to determine the most contributory features toward ischemia would be of great interest.

In the near-term future, the field of cardiac imaging will undoubtedly be witness to a wealth of new ML algorithms embedded into image analytic software platforms that hold the promise of improving accuracy, increasing efficiency, and enabling integration of vast amounts of data into digestible outputs that may enhance therapeutic decision making. The current study represents a large step forward in this regard and may serve as a barometer for the type of high quality research necessary to proving the value of these ML algorithms in clinical practice. Invariably accompanying these softwares will be much debate and discussion as to the most ideal way to incorporate ML algorithms into daily cardiac imaging interpretation and application. The example studied by Coenen et al⁶ may represent a useful first use case as there is a widely accepted threshold of ischemia that can offer guidance toward a clinical decision-making process that is generally binary (ie, medical therapy versus revascularization). Yet, given the ability of contemporary ML algorithms to digest and analyze vast amounts of data, we will be confronted with future algorithms where the outputs may be necessarily more complex or where there is no widely accepted threshold to guide therapy choices. Similar to AlphaZero, these algorithms may quickly find efficiencies of diagnosis or prognosis using different approaches that we may not have thought of or do not use today; and these algorithms may be so multidimensional in nature so as to prevent our understanding if viewed from historical ways of medical thinking. To be certain that these algorithms do not confound clinical decision making will require great care for judicious evaluations of outputs such that they encourage direct application to patient care in a manner that allows us to make clear and informed decisions that improve the outcomes of our patients.