The aim of the study was to describe a paper-based, template-driven and an electronic medical record used for capturing emergency care clinical information and to compare the accuracy of these documentation systems for coding patient encounters using the American Medical Association Current Procedural Terminology-2004 (AMA CPT-2004) evaluation and management codes intended for provider reimbursement.

A retrospective, cross-sectional study of 4-consecutive-day samples of ED patient encounter records from 2 similar community hospitals was done. For clinical documentation, hospital A uses an electronic medical record, whereas hospital B uses a paper-based template-driven record. Using a simple analytic model, expert coders A and B, respectively, coded the records from hospitals A and B for completeness. First, power analysis determined the acceptability of the patient record sample sizes (1 − β = .90 at 1% significance level), and the frequency of AMA CPT-2004 primary evaluation and management codes 99281 through 99285 was calculated. Second, the completeness discrepancy rates for hospitals A and B were compared to determine the accuracy of both the paper-based, template-driven record and the electronic medical record in documenting and representing the clinical encounter. Third, interrater reliability between expert coders A and B was calculated to assess the level of agreement between each expert coder in determining the completeness discrepancy rates between hospitals A and B. Finally, the frequency of primary evaluation and management codes was analyzed to determine if there was a statistically significant difference between the paper-based, template-driven record and the electronic medical record representation of the clinical information, and if that difference could be attributable to the differing clinical documentation systems used in hospitals A and B.

First, descriptive display demonstrated a difference in the frequency of the primary evaluation and management codes 99283 and 99284 within hospital A (expert coder A assessment, 36.1% vs 39.1%; expert coder B assessment, 36.6% vs 38.7%) and hospital B (expert coder A assessment, 47.8% vs 21.9%; expert coder B assessment, 48.6% vs 21.4%) was noted with the median, primary evaluation, and management code for hospital A of 99284 and the median, primary evaluation, and management code for hospital B of 99283. Second, Fisher exact test compared the completeness discrepancy rates between hospitals A and B as assessed by each expert coder and demonstrated no statistically significant difference in the completeness discrepancy rates (accuracy) between the paper-based, template-driven record and the electronic medical record documentation and coding system when assessed by either expert coder A (P = .370) or expert coder B (P = .819). Third, interrater reliability between expert coders A and B was evaluated using Cohen's κ statistic. When evaluated both individually and jointly with respect to hospitals A and B, expert coders A and B had a good strength of agreement in their assessments of the accuracy of the documentation and coding system for hospital A (κ = 0.6200) and hospital B (κ = 0.6906) as well as for both hospitals evaluated together (κ = 0.6616). Finally, interhospital differences in the frequency of primary evaluation and management codes were evaluated using Pearson χ² test with 3 df. The results for expert coder A (χ² = 47.4160; P < .001) and expert coder B (χ² = 46.5946; P < .001) recognize that there is a statistically significant degree of difference between hospitals A and B in the frequency distribution of primary evaluation and management codes, probably because of the dispersion of codes 99283 and 99284.

A keystroke-driven, electronic medical record that resides on a knowledge platform that incorporates a clinical structured terminology, administrative coding schemata, AMA CPT-2004 codes and uses object-oriented, open-ended, branching chain clinical algorithms that “force” physician documentation of the clinical elements provides an equally accurate capture and representation of ED clinical encounter data as a paper-based, template-driven documentation system both in terms of the presence or absence of both the medically necessary, discrete data elements and the textual documentation-dependent, medical decision-making elements.