Acute Infarct Segmentation on Diffusion-Weighted Imaging Using Deep Learning Algorithm and RAPID MRI
Article information
Dear Sir:
Because of its excellent sensitivity to acute lesions, diffusion-weighted imaging (DWI) is essential for the diagnosis of ischemic stroke. Although computed tomography (CT) perfusion- or CT angiography-based decisions can reduce door-to-puncture time, magnetic resonance image (MRI)-based endovascular treatment (EVT) has become a popular alternative due to the development of fast multimodal MRI protocol [1] and image resolution upscaling techniques [2].
Manual DWI lesion segmentation is time-consuming and prone to inconsistencies. Currently, available automatic tools using apparent diffusion coefficients (ADC) may yield inconsistent results in individual patients [3], raising concerns about their reliability in estimating the infarct core in EVT candidates. Deep learning algorithms have demonstrated encouraging results in DWI infarct segmentation [4]. In this study, we compared two commercial DWI infarct segmentation programs, JBS-01K (JLK Inc., Seoul, Korea) and RAPID MRI (iSchemaView Inc., Menlo Park, CA, USA), in a comprehensive stroke center.
From August 2020 to April 2021, we screened all patients with ischemic stroke hospitalized within 7 days of symptom onset (n=673). Among them, 259 patients were excluded (details of the exclusion process are summarized in Supplementary Figure 1). The Institutional Review Board of Chonnam National University Hospital approved the study protocol (CNUH-2023-092). The requirement for written informed consent from the study participants was waived because of the anonymity of the individuals and the minimal risk to them. The Digital Imaging and Communications in Medicine (DICOM) images were randomly divided into two groups and manually segmented by two experienced vascular neurologists (JTK and WSR) using a custom-built web application (https://aix.medihub.ai). Inter-rater agreement was examined in 20 randomly selected patients. Large-vessel occlusion (LVO) was defined as occlusion of the internal carotid artery or middle cerebral artery (M1 or M2) using magnetic resonance (MR) angiography. RAPID MRI is an automatic infarct segmentation program based on the ADC threshold value, whereas JBS-01K is an automatic ischemic lesion segmentation software that uses three-dimensional (3D) U-Net deep learning. Additional details are provided in the Supplementary Materials. The data in the present study were not used for the deep learning training. JBS-01K utilized only the DWI b1000 image as input data, and the mean execution time without a GPU (Intel i5-8250 and 8 GB RAM) for 100 random samples was 13.3±2.1 seconds.
We used the dice similarity coefficient (DSC) to evaluate the inter-rater agreement and segmentation performance of JBS-01K compared with manual segmentation. A Bland-Altman plot and R-squared, root mean squared error, Akaike information criteria, and log-likelihood following linear regression analysis were used to automatically and manually compare the segmented volumes. After stratification according to the LVO, infarct volume, infarct location, and time from the last known well (LKW) to imaging, we reanalyzed the data. To evaluate the accuracy of patient classification based on the EVT clinical trial criteria, we categorized patients using the time from LKW to imaging and manually segmented infarct volume and compared the frequency of correct sorting between RAPID MRI and JBS-01K using the chi-square test. Furthermore, for participants who underwent DWI prior to EVT (n=35), we compared the estimated infarct volumes using RAPID MRI and JBS-01K with manually segmented volumes using the Bland-Altman plot and parameters from a linear regression analysis.
A comparison of the included and excluded patients is shown in Supplementary Table 1. Inter-rater DSC was 0.68±0.16, and mean volume difference was 2.82±4.36 mL. Estimated infarct volumes by RAPID MRI and JBS-01K were correlated with manual segmentation (rho=0.98 both) (Figure 1A). The mean DSC between JBS-01K and manual segmentation was 0.74. RAPID MRI tended to underestimate the true infarct volume in small infarcts (<10 mL) compared with JBS-01K (Figure 1A, inlet). Additionally, RAPID MRI and JBS-01K failed to detect infarcts in 254 (61.4%) and 8 (1.9%; P<0.001) (Supplementary Figure 2 and Supplementary Discussion) of the 414 patients, respectively. The mean percent differences in JBS-01K and RAPID MRI compared to the ground truth were 20.3% (95% confidence interval [CI], -77.4% to 118.1%) and 110.5% (95% CI, -113.6% to 334.6%), respectively (Figure 1B and C).
Comparison of software-estimated and manually segmented infarct volumes. (A) Ground truth infarct volume by manual segmentation vs. estimated infarct volume by RAPID MRI (red) and JBS-01K (blue). The right lower figure enlarged the gray shaded (<10 mL) area. Black dotted lines indicate lesion volume cutoffs of 21, 31, and 51 mL. Green dotted line and shared areas indicate lesion volume cutoff of 70 mL. The threshold of 21, 31, or 51 mL was used in the DAWN trial (DWI or CTP Assessment With Clinical Mismatch in the Triage of Wake-Up and Late Presenting Strokes Undergoing Neurointervention With Trevo) for inclusion of patients, whereas the 70 mL threshold was used in the DEFUSE-3 (Endovascular Therapy Following Imaging Evaluation for Ischemic Stroke) and EXTEND-IA (Extending the Time for Thrombolysis in Emergency Neurological Deficits With Intra-Arterial Therapy) trials. (B) Bland-Altman plot between ground truth infarct volume and estimated infarct volume by JBS-01K. (C) Bland-Altman plot between ground truth infarct volume and estimated infarct volume by RAPID MRI. Green line and black dotted lines indicate mean percentage difference and its 95% limit. Note that if the software did not find infarct, the percent difference was calculated as 200%. DWI, diffusion-weighted imaging; CTP, computed tomography perfusion.
JBS-01K demonstrated more accurate segmentation of infarct lesions than RAPID MRI in both groups, with and without LVO (Table 1). RAPID MRI could not estimate the infarct volume in all small infarcts (<0.87 mL). In both the medium (0.89–6.13 mL) and large (6.21–393.4 mL) infarct groups, JBS-01K performed better than RAPID MRI (Table 1). After stratification by infarct location and onset-to-imaging (Supplementary Figure 3), JBS-01K consistently showed lower root-mean-square errors and log-likelihoods than RAPID MRI. The overall accuracy of patient classification in major clinical trials on EVT was superior for JBS-01K than for RAPID MRI (P=0.002) (Supplementary Table 2). Among the 35 patients who underwent DWI before EVT, RAPID MRI and JBS-01K did not estimate the infarct volume in seven (20%) and two (5.7%) patients, respectively (Supplementary Table 3). In patients with infarct volumes of <10 mL by manual segmentation, JBS-01K markedly outperformed RAPID MRI (Supplementary Figure 4).
Comparison between JBS-01K and RAPID MRI stratified by large vessel occlusion, infarct volume, and location using linear regression
In summary, automated ischemic lesion segmentation using deep learning (JBS-01K) outperforms RAPID MRI in patients with acute ischemic stroke. Notably, RAPID MRI underestimated or failed to detect small lesions compared with JBS-01K (Supplementary Figure 5), which aligns with a recent study [5]. As EVT extends to distal middle cerebral artery occlusion [6] and basilar artery occlusion [7], more precise infarct segmentation is required. The deep learning solution outperformed RAPID MRI in detecting small infarct cores and infratentorial strokes, suggesting that it may be superior to the ADC threshold-based solution for detecting infarct cores in MRI-guided EVT decisions.
To date, clinical research on infarct progression using serial DWIs has not been feasible owing to inter- or intra-rater variability, particularly in patients with small infarcts. Consistent and precise measurements achieved through deep learning can bolster clinical studies using serial DWIs. Additionally, a large dataset and precise infarct segmentation using deep learning can expand our understanding of DWI in terms of functional outcomes [8], early neurological deterioration [9], and recurrent stroke prediction [10].
Our study has several limitations. The short interval between LKW and imaging may underestimate the performance of RAPID MRI, which utilizes an ADC threshold. Moreover, the analogous racial, demographic, and clinical characteristics of the patients used in developing JBS-01K and those in the present study contribute to JBS-01K’s superior performance to RAPID MRI. Multinational and multiethnic research is necessary to further validate these findings.
Supplementary materials
Supplementary materials related to this article can be found online at https://doi.org/10.5853/jos.2023.02145.
Baseline characteristics comparison between included and excluded patients
Accuracy of patient classification according to cutoff using endovascular trials
Comparison between JBS-01K and RAPID MRI in patient undergoing endovascular treatment using linear regression (n=35)
Study flow chart. DWI, diffusion-weighted imaging; MRI, magnetic resonance imaging.
Infarct lesions on diffusion-weighted images missed by JBS-01K. (A-H) Eight diffusion-weighted images with infarcts that were missed by JBS-01K (Dice coefficient similarity of 0). Red arrows indicate presumed infarct areas. Except for image (C), the other seven images showed small and faint lesions.
Bland-Altman plots between estimated infarct using the software and ground truth infarct stratified by last known well (LKW) to image. (A) Patients with diffusion-weighted imaging (DWI) <6 h after the LKW. (B) Patients with DWI between 6 and 24 h after LKW. (C) Patients with DWI findings >24 h after LKW. The green and black dotted lines indicate the mean percentage difference and 95% limit, respectively. If the software did not find an infarct, the percentage difference was calculated as 200%.
Comparison of software-estimated and manually segmented infarct volumes in patients undergoing diffusion-weighted image before endovascular treatment (n=35). (A) Ground truth infarct volume by manual segmentation vs. the infarct volume estimated by RAPID MRI (red) and JBS- 01K (blue). The gray-shaded (<30 mL) area is enlarged in the lower-right figure. Black dotted lines indicate lesion volume cutoffs of 21, 31, and 51 mL. The green dotted line and shared areas indicate a lesion volume cutoff of 70 mL. Thresholds of 21, 31, or 51 mL were used in the DAWN trial3 (DWI or CTP Assessment With Clinical Mismatch in the Triage of Wake-Up and Late Presenting Strokes Undergoing Neurointervention With Trevo) for the inclusion of patients, whereas the 70 mL threshold was used in the DEFUSE-32 (Endovascular Therapy Following Imaging Evaluation for Ischemic Stroke) and EXTEND-IA4 (Extending the Time for Thrombolysis in Emergency Neurological Deficits With Intra-Arterial Therapy) trials. (B) Bland-Altman plot between the ground truth infarct volume and the infarct volume estimated by JBS-01K. (C) Bland-Altman plot between the ground truth infarct volume and estimated infarct volume by RAPID MRI. The green and black dotted lines indicate the mean percentage difference and 95% limit, respectively. If the software did not find an infarct, the percentage difference was calculated as 200%. DWI, diffusion-weighted imaging; CTP, computed tomography perfusion.
Representative cases segmented by JBS-01K and RAPID MRI. (A) Large middle cerebral artery territory infarction. Time since the last well-known imaging was 12.5 h. The infarct volumes obtained by manual segmentation, JBS-01K, and RAPID MRI were 135, 128, and 139 mL, respectively. (B) Infarct area in the middle cerebellar peduncle. The time from the last well-known imaging was 7 h. The infarct volumes obtained using manual segmentation, JBS-01K, and RAPID MRI were 2.2, 1.8, and 0 mL, respectively. (C) A small lesion in the thalamus. Time since the last well-known imaging was 2.3 h. The infarct volumes obtained using manual segmentation, JBS-01K, and RAPID MRI were 0.09, 0.07, and 0 mL, respectively. DWI, diffusion-weighted imaging; ADC, apparent diffusion coefficients.
Notes
Funding statement
This research was supported by a grant from the Korea Health Technology R&D Project through the Korea Health Industry Development Institute, funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HR20C0021), and a Multiministry Grant for Medical Device Development (KMDF_PR_1711195742) of the National Research Foundation, funded by the Korean government.
Conflicts of interest
Wi-Sun Ryu, Yoon-Gon Noh, Jong-Hyeok Park, and Dongmin Kim are employees of JLK Inc.
Author contribution
Conceptualization: WSR, JTK. Study design: WSR, JTK. Data collection: WSR, YGN, JHP, DK. Statistical analysis: WSR, YGN, JHP, DK. Writing—original draft: WSR, YRK, JTK. Writing—review & editing: all authors. Approval of final manuscript: all authors.
Acknowledgements
For deep learning model training, diffusion-weighted image dataset were acquired from Dongguk University Ilsan Hospital (Goyang, Korea), Inje University Ilsan Paik Hospital (Goyang, Korea), Seoul National University Bundang Hospital (Seoungnam, Korea), Soonchunhyang University Hospital (Seoul, Korea), Seoul Medical Center (Seoul, Korea), Nowon Eulji Medical Center (Seoul, Korea), Hallym University Sacred Heart Hospital (Anyang, Korea), Eulji University Hospital (Deajeon, Korea), Dong-A University Hospital (Busan, Korea), and Yeungnam University Hospital (Daegu, Korea).