Interobserver agreement issues in radiology - 19/09/20

Doi : 10.1016/j.diii.2020.09.001

M. Benchoufi ^a,^b, E. Matzner-Lober ^c, N. Molinari ^d, A.-S. Jannot ^b,^e,^f, P. Soyer ^b,^g,⁎
^a Center of Research in Epidemiology and Statistics (CRESS), French Institute of Health and Medical Research (inserm), National Institute of Agricultural Research (INRA), Paris, France
^b Université de Paris, 75006 Paris, France
^c CREST ENSAE, UMR 9194, 91120 Palaiseau, France
^d IMAG, CNRS, Univ Montpellier, Department of Statistics, CHU Montpellier, 34000 Montpellier, France
^e Centre de Recherche des Cordeliers, Sorbonne Université, Inserm, 75006 Paris, France
^f Assistance Publique-Hôpitaux de Paris (AP–HP), Department of Biostatistics, Medical Informatics and Public Health, Hôpital Européen Georges Pompidou, 75015 Paris, France
^g Assistance Publique-Hôpitaux de Paris (AP–HP), Department of Radiology, Hôpital Cochin, 75014 Paris, France

^⁎Corresponding author at: Hôpital Cochin-AP–HP, Department of Radiology, Université de Paris-Descartes Paris 5, 27, rue du Faubourg Saint-Jacques, 75014 Paris, France.Hôpital Cochin-AP–HP, Department of Radiology, Université de Paris-Descartes Paris 527, rue du Faubourg Saint-JacquesParis75014France

Sous presse. Épreuves corrigées par l'auteur. Disponible en ligne depuis le Saturday 19 September 2020
Cet article a été publié dans un numéro de la revue, cliquez ici pour y accéder

Highlights

•	Intra- and inter observer agreement is a critical issue in imaging and this must be assessed with the most appropriate test.
•	Cohen's kappa test should be used to evaluate inter-rater consistency (i.e., inter-rater reliability) for qualitative (nominal or ordinal) variables.
•	Intraclass correlation coefficient should be used to assess agreement for qualitative variables.
•	The Bland-Altman method can be used to assess consistency and conformity but its use should be restricted to comparison of two raters.

Le texte complet de cet article est disponible en PDF.

Abstract

Agreement between observers (i.e., inter-rater agreement) can be quantified with various criteria but their appropriate selections are critical. When the measure is qualitative (nominal or ordinal), the proportion of agreement or the kappa coefficient should be used to evaluate inter-rater consistency (i.e., inter-rater reliability). The kappa coefficient is more meaningful that the raw percentage of agreement, because the latter does not account for agreements due to chance alone. When the measures are quantitative, the intraclass correlation coefficient (ICC) should be used to assess agreement but this should be done with care because there are different ICCs so that it is important to describe the model and type of ICC being used. The Bland-Altman method can be used to assess consistency and conformity but its use should be restricted to comparison of two raters.

Le texte complet de cet article est disponible en PDF.

Keywords : Reproducibility of results, Interobserver agreement, Radiology, Kappa test, Intraclass correlation coefficient