Development of a multimodal machine-learning fusion model to non-invasively assess ileal Crohn's disease endoscopic activity

Itai Guez*, Gili Focht, Mary Louise C. Greer, Ruth Cytter-Kuint, Li Tal Pratt, Denise A. Castro, Dan Turner, Anne M. Griffiths, Moti Freiman

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


Background and Objective: Recurrent attentive non-invasive observation of intestinal inflammation is essential for the proper management of Crohn's disease (CD). The goal of this study was to develop and evaluate a multi-modal machine-learning (ML) model to assess ileal CD endoscopic activity by integrating information from Magnetic Resonance Enterography (MRE) and biochemical biomarkers. Methods: We obtained MRE, biochemical and ileocolonoscopy data from the multi-center ImageKids study database. We developed an optimized multimodal fusion ML model to non-invasively assess terminal ileum (TI) endoscopic disease activity in CD from MRE data. We determined the most informative features for model development using a permutation feature importance technique. We assessed model performance in comparison to the clinically recommended linear-regression MRE model in an experimental setup that consisted of stratified 2-fold validation, repeated 50 times, with the ileocolonoscopy-based Simple Endoscopic Score for CD at the TI (TI SES-CD) as a reference. We used the predictions’ mean-squared-error (MSE) and the receiver operation characteristics (ROC) area under curve (AUC) for active disease classification (TI SEC-CD≥3) as performance metrics. Results: 121 subjects out of the 240 subjects in the ImageKids study cohort had all required information (Non-active CD: 62 [51%], active CD: 59 [49%]). Length of disease segment and normalized biochemical biomarkers were the most informative features. The optimized fusion model performed better than the clinically recommended model determined by both a better median test MSE distribution (7.73 vs. 8.8, Wilcoxon test, p<1e-5) and a better aggregated AUC over the folds (0.84 vs. 0.8, DeLong's test, p<1e-9). Conclusions: Optimized ML models for ileal CD endoscopic activity assessment have the potential to enable accurate and non-invasive attentive observation of intestinal inflammation in CD patients. The presented model is available at

Original languageAmerican English
Article number107207
JournalComputer Methods and Programs in Biomedicine
StatePublished - Dec 2022
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2022 Elsevier B.V.


  • Crohn's disease
  • Machine-learning
  • Magnetic Resonance Enterography
  • Multimodal Learning in Medical Imaging and Informatics


Dive into the research topics of 'Development of a multimodal machine-learning fusion model to non-invasively assess ileal Crohn's disease endoscopic activity'. Together they form a unique fingerprint.

Cite this