Operational data are often used to make systems changes in real time. Inaccurate data, however, transiently, can result in inappropriate operational decision making. Implementing electronic health records (EHRs) is fraught with the possibility of data errors, but the frequency and magnitude of transient errors during this fast-evolving systems upheaval are unknown. This study was done to assess operational data quality in an emergency department (ED) immediately before and after an EHR implementation.

Direct observations of standard ED timestamps (arrival, bed placement, clinician evaluation, disposition decision, and exit from ED) were conducted in a suburban ED for 4 weeks immediately before and 4 weeks after EHR implementation. Direct observations were compared with electronic timestamps to assess data quality. Differences in proportions and medians with 95% confidence intervals (CIs) were used to estimate the magnitude of effect.

There were 260 observations: 122 before and 138 after implementation. We found that more systematic data errors were introduced after EHR implementation. The proportion of discrepancies where the observed and electronic timestamp differed by more than 10 minutes was reduced for the disposition timestamp (29.3% vs 16.1%; difference in proportions, −13.2%; 95% CI, −24.4% to −1.9%). The accuracy of the clinician-evaluation timestamp was reduced after implementation (median difference of 3 minutes earlier than observed; 95% CI, −5.02 to −0.98). Multiple service intervals were less accurate after implementation.

This single-center study raises questions about operational data quality in the peri-implementation period of EHRs. Using electronic timestamps for operational assessment and decision making following implementation should recognize the magnitude and compounding of errors when computing service times.