Validity Evidence for a Novel, Comprehensive Bag–Mask Ventilation Assessment Tool - 16/06/22
, Matthew H. Merves, MD 2, Priyanka Kharayat, MD, MBBS 3, James S. Barry, MD 4, Kristen M. Glass, MD 5, Robert A. Berg, MD 6, 7, Taylor Sawyer, DO, MEd, CHSE-A 8, Vinay Nadkarni, MD, MS 6, 7, Donald L. Boyer, MD, MSEd 6, 7, Akira Nishisaki, MD, MSCE 6, 7Abstract |
Objective |
To develop a comprehensive competency assessment tool for pediatric bag–mask ventilation (pBMV) and demonstrate multidimensional validity evidence for this tool.
Study design |
A novel pBMV assessment tool was developed consisting of 3 components: a 22-item-based checklist (trichotomized response), global rating scale (GRS, 5-point), and entrustment assessment (4-point). Participants' performance in a realistic simulation scenario was video-recorded and assessed by blinded raters. Multidimensional validity evidence for procedural assessment, including evidence for content, response-process, internal structure, and relation to other variables, was assessed. The scores of each scale were compared with training level. Item-based checklist scores also were correlated with GRS and entrustment scores.
Results |
Fifty-eight participants (9 medical students, 10 pediatric residents, 18 critical care/neonatology fellows, 21 critical care/neonatology attendings) were evaluated. The pBMV tool was supported by high internal consistency (Cronbach α = 0.867). Inter-rater reliability for the item-based checklist component was acceptable (r = 0.65, P < .0001). The item-based checklist scores differentiated between medical students and other providers (P < .0001), but not by other trainee level. GRS and entrustment scores significantly differentiated between training levels (P < .001). Correlation between skill item-based checklist and GRS was r = 0.489 (P = .0001) and between item-based checklist and entrustment score was r = 0.52 (P < .001). This moderate correlation suggested each component measures pBMV skills differently. The GRS and entrustment scores demonstrated moderate inter-rater reliability (0.42 and 0.46).
Conclusions |
We established evidence of multidimensional validity for a novel entrustment-based pBMV competence assessment tool, incorporating global and entrustment-based assessments. This comprehensive tool can provide learner feedback and aid in entrustment decisions as learners progress through training.
Le texte complet de cet article est disponible en PDF.Keywords : simulation, medical education, checklist, assessment, workplace-based assessment, procedural competency
Abbreviations : BMV, EPA, GRS, INSPIRE, pBMV
Plan
| Supported by a Section on Critical Care Small Projects Grant (A.W., 2017) from the American Academy of Pediatrics. V.N. and A.N. are supported by the Agency for Healthcare Research and Quality (AHRQ R03HS021583, R18 HS022464, R18HS024511). V.N. is supported by the Endowed Chair in Critical Care Medicine at Children’s Hospital of Philadelphia. The Children's Hospital of Philadelphia is supported by Nihon Kohden, Inc, for the development of Respiratory Function Monitor. D.B. receives funding from the University of Pennsylvania Graduate School of Education. The other authors have indicated they have no financial relationships relevant to this article to disclose. |
|
| Portions of this study were presented as an oral abstract during the 49th Critical Care Congress of the Society of Critical Care Medicine, February 16-19, 2020, Orlando, Florida. |
Vol 245
P. 165 - juin 2022 Retour au numéroBienvenue sur EM-consulte, la référence des professionnels de santé.
L’accès au texte intégral de cet article nécessite un abonnement.
Déjà abonné à cette revue ?
