Implementation of a Holistic Review Pipeline of Radiation Oncology Residency Applicants: Initial Evaluation and Next Steps
Faculty and Abstracts
Purpose: The purpose of this study is to evaluate the efficacy of a tool and workflow for holistic review of radiation oncology (RO) residency applicants to enhance equity within the physician workforce. Physician demographics have been shown unequivocally to impact health inequities in America. While the ACGME’s institutional requirements state that residency programs should focus on recruitment of “a diverse and inclusive workforce of residents/fellows,” there continues to be a lack of sex, racial, and ethnic diversity within radiation oncologists. Here, we demonstrate implementation of a holistic review pipeline for practical use during residency recruitment to facilitate consistency amongst reviewers and ensure comprehensive assessment.
Methodology: At our institution, residency application review has historically been ‘holistic’. Specifically, every aspect of an applicant’s file (recommendation letters, grades/scores, research, community work) is reviewed by program leadership; no applicants are rejected at face value due to low scores. The proposed tool seeks to balance this evaluation across multiple reviewers, to ensure that each measure of the applicant is taken into careful consideration, and to quantify the evaluation for equitable comparison. In addition, a new category has been incorporated: “life experiences that may affect opportunities for development as a physician.” This evaluates non-academic factors such as inherent systemic bias due to underrepresentation in medicine, economic disadvantage, living with disabilities, and other such circumstances. After an initial screening (“first pass”) of applicants, we also added a “second pass” assessment into the pipeline, in which these life experiences are evaluated by members of the Department of Radiation Oncology’s Equity, Diversity, and Inclusion (EDI) Committee.
Results: Of the 138 applicants evaluated by the “first pass,” 67% were identified as having experienced a significant experience, whether or not it had an impact on their life opportunities. These applications were given a blinded “second pass” by the EDI Committee; of these, 55% of applicants were found to have a higher/more favorable score by the “first pass”, while 7% of applicants were given a higher score by the “second pass”. 38% yielded concordance between both groups. All applicants were then reviewed by program leadership, taking into consideration the balance of the two preliminary scores, and a top set of applicants were invited for interview.
Conclusions: Given the majority (93%) of applicants who received a higher or equivalent score by a group of reviewers not intimately involved in structural EDI initiatives, we have demonstrated our Department’s commitment to expand residency education to include more diverse voices. The second pass by the EDI Committee offers even more careful examination of applicants holistically. Implementing this pipeline will help to mitigate fluctuating differences in scoring amongst evaluators. We will next utilize the scoring tool within the interview process and gauge its utility amongst an even larger cohort of reviewers.
