Inter-rater reliability during live pediatric formative Objective Structured Clinical Examination (OSCE) in a French University - 03/11/26

Doi : 10.1016/j.arcped.2026.105614

Aurélie Pham ^a,^⁎ , Florian Blanchard ^b, Antoine Monsel ^c,^d,^e,^f, Manon Allaire ^c,^g,^h, Alain Carrié ^c,^i,^j, Jean-Michel Davaine ^c,^k, Maxens Decavele ^c,^l,^m, Jean-Philippe Foy ^c,ⁿ, Kamila Kolanska ^c,^o, Sohela Moussaoui ^c,^p, Nadia Nathan ^c,^q, Marie-Christine Renaud ^c, Laure Serresse ^c,^r, Yanis Tamzali ^c,^s, Jessica Taytard ^c,^t,^u, Nada Sabourdin ^c,^v,^w

^a Sorbonne University, AP-HP, Department of Neonatology, Armand Trousseau Hospital, Paris 75012, France

^b Sorbonne University, GRC 29, AP-HP, DMU DREAM, Department of Anesthesiology and Critical Care, Pitié-Salpêtrière Hospital, Paris 75013, France

^c Sorbonne University, OSCE Research Group, Faculté de santé Sorbonne Université, Paris, France

^d Sorbonne University, AP-HP, Multidisciplinary Intensive Care Unit, Department of Anesthesiology and Critical Care, La Pitié-Salpêtrière Hospital, Paris 75013, France

^e Institut National de La Santé et de La Recherche Médicale (INSERM), Unité mixte de recherche (UMR)-S 959, Immunology-Immunopathology-Immunotherapy (I3), Paris, France

^f AP-HP, Biotherapy (CIC-BTi) and Inflammation-Immunopathology-Biotherapy Department (DHU i2B), Hôpital Pitié Salpêtrière, Paris, France

^g AP-HP, Department of Hepato-gastroenterology, Hôpital Pitié-Salpêtrière, Paris, France

^h INSERM UMRS 1138, Team Proliferation, Stress and Liver Physiopathology, Centre de Recherche des Cordeliers, Paris, France

ⁱ Inserm, UMR_S1166 ICAN, Sorbonne Université, Paris, France

^j Sorbonne University, AP-HP, UF de génétique de l’Obésité et des Dyslipidémies, service de Biochimie Endocrinienne et Oncologique, Hôpital de la Pitié-Salpêtrière, Paris 75013, France

^k Sorbonne University, Vascular Surgery Department, Hôpital Européen Georges Pompidou, Paris, France

^l INSERM, UMRS1158 Neurophysiologie Respiratoire Expérimentale et Clinique, Sorbonne Université, Paris, France

^m Sorbonne University, AP-HP, Service de Pneumologie, Médecine Intensive et Réanimation (Département R3S), Hôpital Pitié Salpétrière, Paris 75013, France

ⁿ Sorbonne University, AP-HP, Department of Maxillo-facial Surgery, Hôpital Pitié-Salpêtrière, GRC 33 (RIC), INSERM UMRS 938, Centre de Recherche de Saint Antoine, Team Cancer Biology and Therapeutics, Paris, France

^o Sorbonne University, INSERM UMRS_938 CRSA & AP-HP, Service de Gynécologie Obstétrique et Médecine de la Reproduction, Hôpital Tenon, Paris, France

^p Sorbonne University, Department of General Practice, Paris, France

^q Sorbonne University, AP-HP, Pediatric Pulmonology Department, Laboratory of Childhood Genetic Diseases, Inserm UMR_S933, Armand Trousseau Hospital, Paris 75012, France

^r INSERM, UMRS1158 Neurophysiologie Respiratoire Expérimentale et Clinique; Groupe Hospitalier Universitaire APHP Sorbonne Université, Sorbonne Université, Service hexa-site Equipe Soins Palliatif, Accompagnement et Soins de Support, Hôpitaux Charles-Foix, Pitié-Salpêtrière, Rothschild, Tenon, Trousseau/La Roche-Guyon, Paris, France

^s Inserm, UMR_S1135 CIMI, AP-HP Hôpital de la Pitié-Salpêtrière, Département de Néphrologie, Unité de Transplantation Rénale, Sorbonne Université, Paris, France

^t Sorbonne Université, AP-HP, service d’explorations fonctionnelles respiratoires - somnologie, Trousseau, Paris 75012, France

^u Inserm UMRS1158, neurophysiologique respiratoire expérimentale et clinique, Sorbonne Université, Paris, France

^v Sorbonne University, AP-HP, GRC 29, DMU DREAM, Department of Anesthesiology and Critical Care, CHU Armand Trousseau, Paris, France

^w University of Paris Cité EA 7323: Pharmacologie et Evaluation des Thérapeutiques chez L’enfant et la Femme Enceinte, Paris, France

^⁎ Corresponding author.

Abstract

Background and objective

In France, OSCE (Objective Structured Clinical Examination) have recently been incorporated as an official assessment method for the final National Ranking Examination of undergraduate medical students. Given these high stakes, it is of paramount importance to provide high-quality OSCE training in every discipline. Our pediatrics department has developed an OSCE training program covering all areas of pediatric specialties. Students are evaluated by two independent assessors for each station. As part of a quality assessment process, this study investigated inter-rater reliability in OSCE grading between the two assessors.

Materials and methods

This prospective observational study was conducted from March to June 2023 in the Department of Pediatrics of Sorbonne University (Paris, France). Undergraduate medical students were evaluated during OSCE training sessions. Each student had to complete two consecutive stations. Each station was evaluated by two independent assessors. The scoring checklist was divided into two parts: "clinical skills" and "general skills," assessing the student's behavior and relational aptitudes. Evaluation scales were dichotomous for all items.

Results

About 206 fifth-year medical students participated in four distinct OSCE sessions. A total of 411 stations were evaluated by 20 pairs of assessors, resulting in 822 different scores. The intraclass correlation coefficients of “total score,” “clinical skills,” and “general skills” scores between both assessors were 0.83 [0.64–0.93], 0.85 [0.68–0.94], and 0.50 [0.14–0.75], respectively. Inter-rater reliability was influenced by overall OSCE score and varied significantly between OSCE scenarios. Variability was greater in pediatric tuberculosis and hemolytic uremic syndrome scenarios, compared to bronchiolitis.

Conclusions

During live pediatric training OSCE, inter-rater reliability was good regarding clinical skills, and moderate for general skills. Inter-rater variability might be more pronounced for students with lower overall OSCE score. Our study emphasizes the importance of a careful drafting of OSCE cases with precise and clear-cut scoring checklist, especially in stations focusing on very specialized clinical situations.

Le texte complet de cet article est disponible en PDF.

Keywords : objective structured clinical examination (OSCE), Undergraduate medical students, National ranking exam, Reliability

Plan

Introduction

Methods

Study design and population

Inter-rater reliability

Inter-rater absolute difference

Factors influencing inter-rater reliability

Multivariate analysis

Discussion

Conclusion

Availability of data and materials

Funding

Export

Vol 33 - N° 8

Article 105614- novembre 2026 Retour au numéro

Article précédent

Serum IL-17A levels and their association with disease activity in Egyptian children with juvenile idiopathic arthritis: A cross-sectional study
Naglaa S. Osman, Menna K. Tolba, Hanan H. Abd-Elatif, Khaled Saad, Eman R. Badawy

| Article suivant

Acute neuro-inflammatory presentation of ATP1A3 variants: Case reports of three patients with different phenotypes
Capucine Glasson, Marc Gibaud, Marie Alesandrini, Floriane Lebricquir, Lucile Altenburger, Mathilde Nizon, Capucine de Lattre, Guy Letellier, Magalie Barth, Marie Hully, Mélodie Aubart, Patrick Van Bogaert

Bienvenue sur EM-consulte, la référence des professionnels de santé.
L’accès au texte intégral de cet article nécessite un abonnement.

Déjà abonné à cette revue ?

connectez-vous ou créez un compte

Inter-rater reliability during live pediatric formative Objective Structured Clinical Examination (OSCE) in a French University - 03/11/26

Abstract

Background and objective

Materials and methods

Results

Conclusions

Plan

Export citations

Fichier

Contenu

Accès rapides

Mon compte

Aide & support

Plateformes Elsevier Masson

Déclaration CNIL