Thyroid eye disease (TED) can be difficult to manage. The range of available treatments is expanding rapidly; however, cost is a concern and some patients do not respond. The Clinical Activity Score (CAS) was devised as a measure of disease activity and a potential predictor of response to anti-inflammatory treatment. Despite the widespread use of the CAS, inter-observer variability has not been investigated. The aim of the study was to determine the inter-observer variability of the CAS in patients with TED.

Prospective reliability analysis.

Nine patients with a spectrum of clinical features of TED were assessed by 6 experienced observers on the same day. Agreement among the observers was analyzed using the Krippendorff alpha.

The Krippendorff alpha for the total CAS was 0.532 (95% CI = 0.199-0.665), whereas alpha values for the individual components of the CAS varied between 0.171 (CI = 0.000-0.334) for lid redness and 0.671 (CI = 0.294-1.000) for spontaneous pain. Assuming that a CAS value ≥3 implies suitability of the patient for anti-inflammatory treatment, the calculated Krippendorff alpha for agreement among assessors on whether treatment should be given or not given was 0.332 (95% CI = 0.0011-0.5862).

This study has shown unreliable inter-observer variability in total CAS and most individual CAS components, thus highlighting the need for improving the performance of the CAS or seeking other methods to assess activity.