Evaluation of endoscopic disease activity for patients with ulcerative colitis (UC) is important when determining the treatment of choice. However, endoscopists require a certain period of training to evaluate the activity of inflammation properly, and interobserver variability exists. Therefore, we constructed a computer-assisted diagnosis (CAD) system using a convolutional neural network (CNN) and evaluated its performance using a large dataset of endoscopic images from patients with UC.

A CNN-based CAD system was constructed based on GoogLeNet architecture. The CNN was trained using 26,304 colonoscopy images from a cumulative total of 841 patients with UC, which were tagged with anatomic locations and Mayo endoscopic scores. The performance of the CNN in identifying normal mucosa (Mayo 0) and mucosal healing state (Mayo 0–1) was evaluated in an independent test set of 3981 images from 114 patients with UC, by calculating the areas under the receiver operating characteristic curves (AUROCs). In addition, AUROCs in the right side of the colon, left side of the colon, and rectum were evaluated.

The CNN-based CAD system showed a high level of performance with AUROCs of 0.86 and 0.98 to identify Mayo 0 and 0–1, respectively. The performance of the CNN was better for the rectum than for the right side and left side of the colon when identifying Mayo 0 (AUROC = 0.92, 0.83, and 0.83, respectively).

The performance of the CNN-based CAD system was robust when used to identify endoscopic inflammation severity in patients with UC, highlighting its promising role in supporting less-experienced endoscopists and reducing interobserver variability.