Confidence Weighting for Robust Automated Measurements of Popliteal Vessel Wall Magnetic Resonance Imaging

We recently introduced a novel approach to assessing atherosclerosis: popliteal artery wall analysis from knee magnetic resonance imaging.¹ Specifically, the OAI (Osteoarthritis Initiative) is a multicenter, prospective cohort study of knee osteoarthritis (NCT00080171) acquiring multiple biomarkers longitudinally in 4796 subjects for a decade, including serial high-resolution 3-dimensional double-echo steady-state magnetic resonance imaging of the knee.² The popliteal artery wall is well seen in these high-quality MR images, allowing quantitative assessment of atherosclerotic progression/regression. However, the large number of images poses a formidable challenge. Manual segmentation of 1 knee/artery across 60 axial slices takes about 3 to 4 hours for an expert human reader, so around 2 decades of continuous effort would be needed to manually segment all 3 million slices in the OAI database.

To address this, we developed a fully automated deep learning–based algorithm called fully automated and robust analysis technique for popliteal artery evaluation (FRAPPE) to segment and quantify the popliteal artery wall.³ Institutional review boards approved the Health Insurance Portability and Accountability Act-compliant OAI study protocol, and all participants provided written informed consent. Anonymized OAI questionnaire and MR imaging data are publicly available at https://nda.nih.gov/oai/. The additional data that support the findings of this study are available from the corresponding author on reasonable request.

FRAPPE reduced the segmentation time from 4 hours to 8 minutes on graphical processing unit–equipped high-performance environments such as the American Heart Association Precision Medicine Platform or equivalent workstations. Segmentation quality is generally good, with only occasional discrepancies due to image quality and the overall complexity of the case. For example, segmentation may be impacted by incomplete blood suppression, nonuniform fat suppression, complicated bifurcations/trifurcations/branches, and the presence of adjacent vein, which may resemble the artery. We require that our findings not be confounded by these aberrant images, but unfortunately, the large total number of images precludes checking and revising segmentations manually. We, therefore, seek an objective, unbiased, and fully automatic method to minimize the contribution of poor segmentations to the conclusions. Here, we employ confidence weights from FRAPPE’s own internal assessment during subsequent statistical analysis.

FRAPPE provides 2 outputs for each cross-sectional image: a map of the probability that each image pixel is artery wall; and pixel locations of lumen and outer wall contours. FRAPPE segmentation confidence is derived from these 2 outputs by summing the artery wall pixel probabilities within the artery wall contours (P_in) and outside of the artery wall (P_out), then dividing (P_in−P_out) by the number of pixels within the artery wall contours. The FRAPPE segmentation confidence is maximally 1 (highest confidence) when all pixels within the artery wall contours have probability 1 and all others have probability 0. However, FRAPPE segmentation confidence can be <0 when segmentation confidence is particularly low (P_in<P_out).

FRAPPE segmentation confidence can be converted into a slice-level statistical weight (SLW) by replacing negative confidence values with 0. The SLWs, therefore, range from 0 (lowest confidence) to 1 (highest confidence). SLWs are then used when aggregating measurements from the slice level to the artery level, such as calculating mean wall thickness or mean lumen area as a weighted average instead of a typical unweighted average of the slice-level measurements. An artery-level weight (ALW; range, 0–1) is calculated by summing the SLWs across all cross-sectional images and dividing by the number of images. The ALWs can be used in any statistical routine that supports weighting, including linear regression, generalized linear models, etc. Using these weights (SLW/ALW) in the analysis will reduce the impact of measurements with lower confidence on the final results in an objective, unbiased, and fully automatic way.

To assess this scheme, we analyzed the repeatability of FRAPPE measurements with and without weighting applied. Seven hundred seventy subjects with scan-rescan pairs (a 6-month interval assumed to involve negligible progression) provided repeated imaging of 1484 knees. Of these, 46% were men; age, 61±9 years; body mass index, 30±5 kg/m²; 73% were hypertensive, 9% were current smokers, 10% had diabetes mellitus; and 30% took lipid-lowering medications. Mean wall thickness was 0.83±0.09 mm, and mean lumen area was 30±10 mm². ALWs ranged from 0.05 to 0.81 (median, 0.76), although 92% had ALW >0.7. The overall coefficients of variation for repeatability of mean wall thickness and mean lumen area were 4.7% and 4.6%, respectively. After applying weighting, the overall coefficients of variation decreased to 4.0% and 4.3%, respectively—a 14% to 16% improvement. When grouped by ALW (Figure), the 2 lowest confidence groups showed the most improvement.

After applying confidence weights, FRAPPE scan-rescan coefficients of variation statistics for popliteal artery morphology measurements were similar to those from manual segmentation of carotid and intracranial arteries.^4,5 This suggests that applying confidence weighting to automated FRAPPE measurements can improve measurement precision, allowing large-scale and reliable analyses of the OAI and similar cohorts. Coupled with the rich demographic, clinical, and lifestyle covariates collected over the follow-up period, this enables an in-depth assessment of popliteal plaque progression and related risk factors, providing a valuable opportunity to improve our understanding of the natural history of atherosclerosis. Automated FRAPPE segmentation and confidence weighting can be extended to other vascular beds and large-scale vessel wall magnetic resonance imaging studies for a comprehensive assessment of atherosclerosis.