Study objectives: We implement and describe a process for graduating emergency medicine residents from a new training program in a developing postwar region with no legacy of emergency medicine.

Methods: We describe the testing process for the first graduating emergency medicine residency class at the University Hospital in Pristina, Kosovo. The 19 residents had completed 4 years of emergency medicine training. Given the lack of existing emergency medicine specialists in Kosovo, the Ministry of Health required a process for certifying residency graduates as competent to practice independently. Our team developed and conducted that examination process in Kosovo. The 3-day examination process was modeled after the American Board of Emergency Medicine (ABEM) examination process. Residents received a 110-question written examination to be taken over a 3-hour period. The questions, adapted from emergency medicine board review materials, were translated into the predominant local language, Albanian, before the team's arrival. A 70% correct response rate was required to pass the written examination, which would be validated on site. Residents were also required to manage oral case simulations. A cardiac, trauma, pediatric, and toxicology case were each prepared by the team before arrival. Because local practice relies heavily on decentralized specialty clinics, the residents had minimal if any pediatric case experience, and this case was eliminated. The oral examination was conducted over a 2-day period. Each case was delivered by either an emergency medicine or emergency medicine–pediatrics ABEM diplomate, whereas residents' performances were judged by a panel composed of an emergency physician and 2 local physicians practicing another specialty. Residents would be scored on a continuum (fail, borderline, acceptable, excellent). Although the Kosovar emergency medicine residents had received 10 weekly sets of practice written questions, they had no experience with oral case simulations.

Results: All 19 residents completed the written examination within the allotted time. The median preliminary score for the written examination was 77% (range 67% to 82%). During 4 hours, the team assessed each item's correct response rate. Any question answered incorrectly by more than 60% of the residents was reviewed by the team. Twenty-two items met this criterion and were reviewed separately in the original English and by back-translation from Albanian. On review, 9 items were either judged to be poorly constructed or were not applicable to local realities, whereas 8 were inadequately translated. These 17 items were removed from the numerator and the denominator for scoring. The remaining 5 questions passed review and were retained. The mean score was 88% correct (range 77% to 94%). Although it was intended that each resident receive 3 case simulations, because of the novelty of oral case simulations and the time required for translation, this was impossible. Instead, each resident conducted only 1 case simulation, which would be graded on a pass/fail basis instead of a continuum. Three residents who did not complete required actions or whose actions were considered potentially dangerous or negligent received a second case in an effort to determine whether those deficiencies were due to true poor judgment or lack of familiarity with the examination format. Each received a passing evaluation on the second case. The outbreak of civil unrest on the second day of oral examinations, resulting in an urgent need for physicians, required an expedited completion of the final 2 examination cases and provided the unexpected opportunity for direct observation of the residents' clinical competency, which the team perceived to be evident.

Conclusion: The certification process used in the United States may be successfully applied in developing and rebuilding regions, provided that process is flexibly adapted to local realities. The resumption of ethnic hostilities exemplified why emergency medicine is necessary in such regions, just as the disparity between oral case simulation and actual clinical performance serves as a warning. Examination methods that are valid in the United States may be insensitive in discerning competency in other settings, particularly in the absence of extensive experience with those methods. Emergency medicine in the developing world is fragile and often unfamiliar or even threatening to other specialty physicians. A rigid approach to preconceived performance standards, without sensitivity to the limitations of those standards, risks crippling a nascent emergency medicine specialty. However, by including other specialists in a credible certification process and by flexibly demonstrating firm standards, US emergency physicians can lend invaluable credibility and support to the introduction of emergency medicine to developing or rebuilding regions.