Digital single-operator cholangioscopy (d-SOC) with cholangioscopic biopsy sampling has shown promise in the evaluation of indeterminate biliary strictures. Some studies have suggested higher sensitivity for visual impression compared with biopsy sampling, although assessors were not blinded to previous investigations. We aimed to investigate the diagnostic accuracy and interobserver agreement (IOA) of d-SOC in the visual appraisal of biliary strictures when blinded to additional information.

A multicenter, international cohort study was performed. Cholangioscopic videos in patients with a known final diagnosis were systematically scored. Pseudonymized videos were reviewed by 19 experts in 2 steps: blinded for patient history and investigations and unblinded.

Forty-four high-quality videos were reviewed of 19 benign and 25 malignant strictures. The sensitivity and specificity for the diagnosis of malignancy was 74.2% and 46.9% (blinded) and 72.7% and 62.5% (unblinded). Cholangioscopic certainty of a malignant diagnosis led to overdiagnosis (sensitivity, 90.6%; specificity, 33%), especially if no additional information was provided. The IOA for the presence of malignancy was fair for both assessments (Fleiss’ κ = .245 [blinded] and κ = .321 [unblended]). For individual visual features, the IOA ranged from slight to moderate for both assessments (κ = .059-.400 vs κ = .031-.452).

This study showed low sensitivity and specificity for blinded and unblinded d-SOC video appraisal of indeterminate biliary strictures, with considerable interobserver variation. Although reaching a consensus on the optical features of biliary strictures remains important, optimizing visually directed biopsy sampling may be the most important role of cholangioscopy in biliary stricture assessment.