Study objectives: To determine (1) reliability and validity estimates of three modalities used to assess open thoracotomy procedural competency and (2) the effect of computer practice on procedural performance as measured by the three assessment modalities.

Methods: An experimental, sequential assessment design with volunteer examinees completing all three assessment modalities (paper, computer, pig model) was implemented at the animal support facilities of a university medical school with an affiliated emergency medicine residency program. Level of physician training (student, resident, faculty) and type of computer practice (thoracotomy, cricothyrotomy) were independent variables. Procedural competency scores were determined for each modality; scores were defined in terms of performance time and performance accuracy for three thoracotomy procedures (opening the chest, pericardiotomy, and aortic cross-clamping).

Results: Thoracotomy performance on the pig reliably discriminated among examinees known to differ in level of training. However, computer simulation performance did not significantly differ among examinees with different levels of training. Computer simulation practice significantly improved later performance on the computer assessment (P<.05) but not on the pig assessment. The greatest predictor of procedural competency (time and accuracy) on the pig assessment was the ability to sequentially order procedural steps.

Conclusion: This study establishes the pig model as superior to the paper and computer models as the criterion standard for open thoracotomy assessment. Psychometric properties support the pig model as the most reliable and valid model yet described for assessing thoracotomy procedural competency. Computer simulation practice using visual images (complex anatomy) and the sequential ordering of procedural steps through paper modeling show promise for teaching and assessment of prerequisite skills required to develop psychomotor procedural competency. [Chapman DM, Rhee KJ, Marx JA, Honigman B, Panacek EA, Martinez D, Brofeldt BT, Cavanaugh SH: Open thoracotomy procedural competency: A validity study of teaching and assessment modalities. Ann Emerg Med December 1996;28:641-647.]