S. Ahad1, M. Sheley2, B. Dyniewski2, C. J. Schwind2, M. L. Boehler2, I. Hassan1 1University Of Iowa,General Surgery,Iowa City, IA, USA 2Southern Illinois University School Of Medicine,Surgery,Springfield, IL, USA
Introduction: The phase 1 curriculum of the American College of Surgeons (ACS) and the Association of Program Directors in Surgery (APDS) includes eleven basic surgical skills training modules. After completing training, performances are intended to be scored by surgical faculty through direct observation or video-taping using the ACS/APDS verification of proficiency (VOP) checklist. We hypothesized that, if trained by an expert, an inexperienced non-physician could reliably evaluate these basic technical skills using the VOP checklist.
Methods: PGY1 resident videos from one-handed surface knot tying (n=24), simple interrupted suturing (n=17) and central line venous access (CVL) (n=18) were evaluated using the ACS/APDS VOP checklists by one expert and four inexperienced non-physicians (2 medical students and 2 clinical nurse instructors). Evaluations (n=295) were compared using chi-square, analysis of variance and intraclass correlation (ICC).
Results: We found near perfect inter-rater agreement for determining overall proficiency for suturing and CVL (ICC=0.96) (P<.001, 95%CI 0.79-0.99) and very good agreement for knot tying proficiency (ICC=0.76) (P<.001, 95%CI 0.48-0.86). Individual checklist items that showed agreement were objective in nature (i.e. hands crossed, accurate placement of suture, appropriate site for venipuncture) (ICC range from 0.64-0.96). While individual items that required judgment and interpretation of quality (e.g. smooth transition between knots, appropriate tension, economy of motion) had little to no inter-rater agreement (ICC range -0.06-0.18).
Conclusion: With training, inexperienced non-physicians can adequately determine overall global proficiency but they may lack necessary experience to recognize some of the more subjective items measuring proficiency on the ACS/APDS VOP checklists.