Completeness, safety, and readability of large language model-generated informed consent materials for orthognathic and craniofacial surgery: a procedure-specific content analysis - 15/11/26
Abstract |
Background |
Procedure-specific completeness of large language model (LLM)-generated orthognathic and craniofacial consent materials remains uncertain.
Methods |
A fixed corpus of 132 outputs from four default/free consumer interfaces (ChatGPT, Gemini, DeepSeek and Grok; 11 procedures; three new-chat repetitions) underwent revision-stage independent scoring by three oral and maxillofacial surgeons using a locked checklist. Final outcomes used item-level two-of-three majorities. Coverage comparisons used 11 paired procedure blocks; critical omissions were analysed with a procedure-random-intercept logistic model. Readability comparisons were descriptive.
Results |
Mean coverage was 77.81% for ChatGPT, 72.94% for Gemini, 83.15% for DeepSeek and 83.27% for Grok (Friedman χ²(3)=22.223, P<0.001). Holm-adjusted comparisons supported higher coverage for DeepSeek than ChatGPT and Gemini, and for Grok than Gemini. Outputs with at least one critical omission numbered 15/33, 17/33, 1/33 and 1/33, respectively. Of 34 affected outputs, 28 met general-consent/task criteria and six met procedure-specific criteria. Item-level Gwet AC1 was 0.892 (95% CI 0.885–0.899); derived critical-status AC1 was 0.583 (0.477–0.689). Six outputs contained incorrect/unsupported-statement flags, two over-reassurance flags and one format failure. Mean Flesch–Kincaid Grade Level ranged from 10.22 to 13.89. Excluding the format failure left coverage comparisons unchanged.
Conclusion |
Coverage and critical-omission frequency differed across these sampled outputs. Aggregate coverage did not establish completeness or clinical safety. The findings identify content requiring scrutiny before any proposed clinical use; patient comprehension, clinical utility and the effectiveness of checklist-assisted verification were not tested.
Le texte complet de cet article est disponible en PDF.Keywords : Informed consent, Orthognathic surgery, Craniofacial surgery, Large language models, Content analysis, Readability
Plan
Vol 127 - N° 6
Article 103077- décembre 2026 Retour au numéroBienvenue sur EM-consulte, la référence des professionnels de santé.
L’accès au texte intégral de cet article nécessite un abonnement.
Déjà abonné à cette revue ?

