J. WEBER1, S. SHEBRAIN1, A. WOODWYK1, G. MUNENE1 1WESTERN MICHIGAN HOMER STRYKER SCHOOL OF MEDICINE,GENERAL SURGERY,KALAMAZOO, MICHIGAN, USA
Introduction:
The interviewing of applicants for surgery residency is considered an important determinant of which and how applicants get ranked. The objective of this study is to evaluate the reliability and validity of a standardized interviewing assessment instrument
Methods:
A prospective analysis of interviewees using a standardized assessment instrument at a university affiliated general surgery program. Applicants were evaluated using a standardized scoring instrument. (6 GS faculty, including the program director, and 3 GS residents), evaluated applicants on 6 characteristics (subjective and objective) using a 5-point Likert scale and a total score. Applicants were ranked at the conclusion of their interviews
Results:
There were a total of 432 assessments representing 51 applicants. The inter rater reliability (IRR) was good in all six domains (Cronbach’s alpha coefficient > 0.7). The IRR of the interviewees rank was good for the faculty, residents and the entire group (Cronbach’s alpha coefficient >0.7). The spearman correlation between the total score and the final rank was -0.296, -0.366 and -0.294 respectively for faculty, residents and the entire group.
Conclusion:
Despite good IRR in several domains evaluated by the standardized interview instrument the total score had a low correlation with the final rank. IRR of the applicants rank was excellent despite the low correlation with the total score suggesting shared biases play a significant role in the ranking process. Adoption of best practices in the interviewing process may eliminate some of these biases