Measurement Under the Microscope: High Variability and Limited Construct Validity in Emergency Department Patient-Experience Scores - 23/04/18
Abstract |
Study objective |
We evaluate variability and construct validity in commercially generated patient-experience survey data in a large sample of US emergency departments (EDs).
Methods |
We used Press Ganey patient-experience data from a national emergency medicine group from 2012 to 2015 across 42 facilities and 242 physicians. We estimated variability as month-to-month changes in percentile scores and through intraclass correlations. Construct validity was assessed with linear regression analysis for monthly facility- and physician-level percentile scores.
Results |
A total of 1,758 facility-months and 10,328 physician-months of data were included. Across facility-months, 40.8% had greater than 10 points of percentile change, 14.7% changed greater than 20 points, and 4.4% changed greater than 30. Across physician-months, 31.9% changed greater than 20 points, 21.5% changed greater than 30, and 13.6% changed greater than 40. Intraclass correlation estimates demonstrated similar variability; however, this was reduced as data were aggregated over fixed time increments. For facility-level construct validity, several facility factors predicted higher scores: teaching status; more older, male, and discharged patients without Medicaid insurance; lower patient volume; less requirement for physician night coverage; and shorter lengths of stay for discharged patients. For physician-level construct validity, younger physician age, participating in satisfaction training, increasing relative value units per visit, more commercially insured patients, higher computed tomography or magnetic resonance imaging use, working during less crowded times, and fewer night shifts predicted higher scores.
Conclusion |
In this sample, both physician- and facility-level patient-experience data varied greatly month to month, with physician variability being considerably higher. Facility-level scores have greater construct validity than physician-level ones. Optimizing data gathering may reduce variability in ED patient-experience data and better inform decisionmaking, quality measurement, and pay for performance.
Le texte complet de cet article est disponible en PDF.Plan
| Please see page 546 for the Editor’s Capsule Summary of this article. |
|
| Supervising editor: Stephen Schenkel, MD, MPP |
|
| Author contributions: JMP, JNC, OC, CKC, and AV conceived the study. JNC, CKC, and AV supervised data collection. PP performed the statistical analysis. JMP, PP, SA, and AV drafted the article, and all authors contributed substantially to its revision. AV takes responsibility for the paper as a whole. |
|
| All authors attest to meeting the four ICMJE.org authorship criteria: (1) Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND (2) Drafting the work or revising it critically for important intellectual content; AND (3) Final approval of the version to be published; AND (4) Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. |
|
| Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje.org/). The authors have stated that no such relationships exist. |
|
| US Acute Care Solutions provided the data for this study to Carnegie Mellon University under a data use agreement. US Acute Care Solutions does not exercise any control over the analysis, conclusions, or decision to publish the analysis presented. |
|
| Readers: click on the link to go directly to a survey in which you can provide M8DYYHV to Annals on this particular article. |
|
| A podcast for this article is available at www.annemergmed.com. |
Vol 71 - N° 5
P. 545 - mai 2018 Retour au numéro
Article précédent
- Information for Readers
- Bouncing Back Elsewhere: Multilevel Analysis of Return Visits to the Same or a Different Hospital After Initial Emergency Department Presentation
- Bradley D. Shy, George T. Loo, Tina Lowry, Eugene Y. Kim, Ula Hwang, Lynne D. Richardson, Jason S. Shapiro
Bienvenue sur EM-consulte, la référence des professionnels de santé.
L’accès au texte intégral de cet article nécessite un abonnement.
Déjà abonné à cette revue ?