Objective Structured Assessment of technical skill (OSATS) in the Surgical Skills and Technology Elective Program (SSTEP): Comparison of peer and expert raters - 10/02/22

Doi : 10.1016/j.amjsurg.2021.03.064

Hamza Asif ^a, Carter McInnis ^a, Frances Dang ^a, Henry Ajzenberg ^a, Peter L. Wang ^a,^b, Adam Mosa ^a, Gary Ko ^a, Boris Zevin ^c, Stephen Mann ^c,^⁎ , Andrea Winthrop ^c
^a Undergraduate Medical Education Program, Queen’s University, Kingston, Ontario, Canada
^b Department of Medicine, Queen’s University, Kingston, Ontario, Canada
^c Department of Surgery, Kingston General Hospital, Kingston, Ontario, Canada

^∗Corresponding author. Department of Surgery, Victory 3, Kingston General Hospital, 76 Stuart St, Kingston, ON K7L 2V7, Canada.Department of SurgeryVictory 3Kingston General Hospital76 Stuart StKingstonONK7L 2V7Canada

Abstract

Background

A major challenge with Competency Based Medical Education (CBME) is that of increased assessment burden on faculty. To reduce this burden, the accuracy and reliability of peer-assessment for surgical skills requires further exploration.

Methods

Forty-two second year medical students were video recorded while performing a simple interrupted suture and an instrument tie. Four novice raters underwent a short training session on the use of the Objective Structured Assessment of Technical Skills (OSATS) checklists. Videos of the suturing task were then independently assessed by the four novice raters and two expert raters on two occasions. Agreement between novice and expert rater scores was calculated using the intraclass correlation coefficient (ICC).

Results

For both simple interrupted suturing (ICC = 0.78, CI = 0.66–0.86, p < 0.001) and instrument ties (ICC = 0.87, CI = 0.80–0.92, p < 0.001), there was good agreement between novice and expert raters.

Conclusions

Novice raters can be taught to use the OSATS checklists to assess peers on simple suturing and instrument tying tasks.

Le texte complet de cet article est disponible en PDF.

Highlights

•	Medical students (novice raters) were trained to use OSATS checklists to achieve good agreement with surgeons (expert raters).
•	Novice raters received a one-time 30-min tutorial by an attending surgeon for instrument ties and simple interrupted suturing.
•	Novice raters had good agreement on OSATS checklists when rating videos of peers performing these two tasks.

Le texte complet de cet article est disponible en PDF.

Keywords : Medical education, Surgical skills, OSATS checklists, Peer-assessment, CBME