A. Nelson1, M. Baker1, J. Cummings3, M. McCrum1, T. Varghese2 1University Of Utah, Division Of General Surgery, Department Of Surgery, Salt Lake City, UT, USA 2University Of Utah, Division Of Cardiothoracic Surgery, Department Of Surgery, Salt Lake City, UT, USA 3University of Utah, David Eccles School Of Business, Salt Lake City, UT, USA
Introduction:
The COVID-19 pandemic necessitated the use of virtual interviews in Graduate Medical Education application and resulted in an increase in the average number of residency programs applied to. Simultaneous questions regarding the predictive value of standardized tests have driven a transition to pass/fail grading of board exams. Such trends may interfere with ideal matching of residents to training programs. Few empiric studies evaluate efficacy of current selection practice or compare current to more traditional resident hiring practices. Effective practice regarding resident selection is not well defined.
Methods:
A literature search using scoping review methodology, augmented by literature review AI tool Elicit, was performed across PubMed, Embase, CINAHL and Semantic Scholar to explore the GME resident selection process from application review to rank list development. Eligible studies included reviews, observational studies, surveys, and narrative reviews between 1984 and 2023. There were no RCTs. Further analysis was done to map these to traditional hiring practices.
Results:
A total of 2,287 publications were identified, of which 241 articles were included after title and abstract screening, and 223 were included for data synthesis after full text review. These studies evaluated holistic review, interview styles, standardized questions, pre-interview assessments, the role of USMLE scores, virtual interviews, promoting diversity equity and inclusion (DEI), post-interview correspondence, predictors of resident performance, and work sample testing. 31.1% of studies were surveys of program directors or applicants, 37.4% were observational studies, 13.5% were review articles, and 18.0% were narrative reviews. 35% of studies examined residents across all specialties. Studies evaluating new approaches to resident interview identified holistic review, standardized interviews, diversity goals, blinding, and work sample testing as effective applicant evaluation strategies. The intervention most commonly cited as effective was holistic review, which was found to improve applicant fit and program diversity in 12.6% of papers reviewed.
Conclusion:
While holistic application review has emerged as a method to increase diversity in recent years, there remains a paucity of research regarding specific best practices for application review, standardizing interviews, and rank list formation. Evidence suggests that both blinding and standardized interview techniques significantly improve inter-rater reliability and DEI. There are mixed opinions among applicants and program directors regarding preference for virtual versus in person interviews. In general, methods for evaluating applicants by virtual platform are currently rudimentary relative to best practices in the business community. Both virtual and in-person work sample testing techniques show promise. More research will be required to determine the best practices for resident selection moving forward.