United States

Background: 
Increasing application numbers constrain residency programs from conducting a holistic review of applicants. Pilot programs to improve holistic review are underway aimed at appraising applicants rapidly. Despite this, assessing narrative components of residency applications remains a bottleneck during applicant review, with many programs resorting to filters based on academic metrics and attributes to invite applicants that are known to have racial, gender, and other inequities. To address this gap and automate the process, the authors used natural language processing (NLP) to predict an interview invite based solely on their narrative experiences.

Methods: 
Narrative experience entries were extracted from 6500 residency applications across three application cycles (2017-2019) at one academic internal medicine program and paired with the interview invitation decision made by program directors. NLP was used to process the experiences’ entries; the importance of each word or phrase (called “tokens”) was calculated using the term frequency-inverse document frequency (TF-IDF). A logistic regression model was developed to determine words or phrases that predicted an interview invitation. Models were also built using structured application data such as USMLE scores and graduation year – either alone or combined with the NLP analysis. Model performance was evaluated on never-before-seen data using the area under the receiver operating characteristic and precision-recall curves (AUROC and AUPRC, respectively). The most influential words or phrases impacting interview invitation were examined.

Results: 
The NLP-only model had an AUROC of 0.79 and AUPRC of 0.48; phrases such as “student-run”, “pre-clinical clerkships”, “analyze data”, and “social determinants of health” increased the probability of receiving an interview invitation. Adding structured data significantly improved prediction, with an AUROC of 0.93 and AUPRC of 0.74. The variable representing the predicted probability based on NLP of the narrative experiences had the highest coefficient magnitude in this combined model. This emphasized the importance of experiences during the resident selection process.

Conclusion: 
To our knowledge, this is the first report on applying NLP to residency application materials to predict an interview invitation. NLP may provide a promising approach for a rapid holistic review and could advance the aims of equity and diversity in selection by expanding currently used metrics.


AMA Research Challenge 2022

Enhancing Holistic Review During Residency Selection: Using Natural Language Processing of Applicants’ Experiences to Predict an Interview Invitation

Welcome to the largest national, multi-specialty research event **featuring research from 750 medical students, residents and fellows, and international medical graduates.**


#### Event Details
The 2022 American Medical Association Research Challenge includes a poster symposium hall and a semifinals hall with the top scored research. Log in at any time throughout the event to:
* **View posters and presentations** on a variety of topics and specialties
* **Score posters in the semifinals** to help decide the top five finalists (AMA member exclusive opportunity)
* **Connect with participants** and provide feedback to help advance their research

At the conclusion of this event, the AMA will announce the top five finalists who will advance to the AMA Research Challenge finals, present their research to an elite panel of judges and compete to win a $10,000 grand prize sponsored by Laurel Road.
 

#### Attend Live Forums
**Oct. 20, 6 — 7 p.m. Central**
 **How to plan a career in academic medicine** 
 Learn how to plan a career in academic medicine and get your questions answered. [Access Here](https://underline.io/events/363/lounge)
 
 **Speakers**
![](https://assets.underline.io/uploads/markdown_image/1/image/d5adb133dfc58eb0a6d469e9cedcd5d4.jpg)
Ricardo Correa, MD, EdD
![](https://assets.underline.io/uploads/markdown_image/1/image/5f9437ab87861538bf02b9ca7c0d926e.jpg)
Stuart M. Greenstein, MD
![](https://assets.underline.io/uploads/markdown_image/1/image/05b1b637c99286d558c371f8e8b185ce.jpg)
Stephanie Strohbeen, MD
 
 **Oct. 22, 11 a.m. — Noon Central**
 **How to get published**
 Learn tips to get your research published and get your questions answered. [Access Here](https://underline.io/events/363/lounge)
 
 **Speakers**
![](https://assets.underline.io/uploads/markdown_image/1/image/8ce6235e957d31cb5e6f9097cc6f6972.jpg)
Jessica Li, MD/PhD Candidate
![](https://assets.underline.io/uploads/markdown_image/1/image/9950ef2e0237f54cb3d8280334d42491.jpg)
Charles Lopresto, DO

### Sponsor
AMA would like to thank Laurel Road for sponsoring a $10,000 grand prize for the winner of the AMA Research Challenge. Laurel Road is not involved in the judging or scoring process to select the winner. The winner will be announced on Dec. 7 during the AMA Research Challenge finals. Payment of the grand prize will be directly from Laurel Road. See the AMA Research Challenge Official Rules for terms and conditions.

![](https://assets.underline.io/uploads/markdown_image/1/image/611e331d4a0eba0920f240cfc04b4d77.png)

**There is no registration fee for this event! **

Register to explore research in a variety of topics. AMA members get exclusive access to score posters in the semifinals.

**Please use the link below to register for free if you have not already done so:**

https://www.ama-assn.org/about/events/ama-research-challenge

Registration Is Required

Welcome to the largest national, multi-specialty research event featuring research from 750 medical students, residents and fellows, and international medical graduates.


poster

Background: Throughout the COVID-19 pandemic, countless research publications were released regarding SARS-CoV-2 and its variants, suggested treatments, and vaccine safety and efficacy. Daily publication of new research became overwhelming for health professionals and the public to stay informed. In recent years, there has been increasing interest in the use of social media for dissemination of veritable information in academic medicine. A group of 70 students across the four colleges at Rush University created the “Rush University COVID-19 Journal Club.” Following expansion of the journal club to social media, audience opinion and engagement was surveyed. 
Methods: Easily accessible and understandable summaries of the newest research were written by students and sent to faculty sponsors for feedback. Following revision, summaries were published weekly on the Rush University COVID-19 Journal Club website for clinicians and students to use for reference. An Instagram page was then created and information was further condensed into succinct posts to address COVID-19 “FAQs.” Next, a survey was distributed to followers of the Instagram page with questions meant to assess the effectiveness of the platform and gain feedback. A 5 point Likert scale was used as the primary question format. 
Results: The Instagram page accrued 749 followers and posted 52 unique posts over a 2 year period. Preliminary results from the surveys demonstrate that over 80% of respondents strongly agree that the Instagram posts 1) are an effective platform for the public presentation of factual COVID-19-related information; 2) provide relevant and valuable information; 3) provide information that is clear, concise, and can be easily understood. 
Conclusion: These results suggest that the Rush COVID-19 Journal Club was able to successfully create a social media presence and convey information without sacrificing scholarly integrity. Other academic institutions may benefit from the application of this model to help students and clinicians with the interpretation and evaluation of research topics with large bodies of evidence.

Going Viral: Expanding a Student-Run COVID-19 Journal Club to Social Media

Background
Across the nation, racism has been declared a public health crisis. To provide high value healthcare, providers and medical student must define and recognize manifestations of stress related to racism and discrimination. By student request, we aimed to provide knowledge of the physiological effects of stress from perceived discrimination, screening tools, and dialogue frameworks for clinical discussions. 

Methods
Second year medical students and neurobiology faculty developed an hour-long teaching session addressing the stress response. To prepare for the session, learners were assigned readings, a video lecture, and tasked to create an HPI on a prompted scenario. The session acknowledged current systems of oppression in society and established ground rules for psychological safety. Students traded self-created cases between groups and discussed each clinical scenario identifying stress-related responses in different patient populations. After the session, a debriefing period was offered to address lasting concerns. 

Results
Students identified the impact of acute and chronic stress on various body systems through an understanding of physiological mediators of the stress response. They discussed the various health related consequences of racism and other forms of discrimination concerning social identity and intersectionality. Groups demonstrated structural competency in the clinical impact of the stress response through the creation of clinical case-based scenarios, group discussion and a debrief including the importance of trauma-informed language for both provider and patient.

Strengths included a peer-created, peer-delivered course curriculum where second year medical students provided didactic instruction, facilitated small group discussion, and led a debrief session for first-year students. Ideally, a topic of this sensitivity would include additional instruction from an expert in the field of healthcare advocacy or discrimination.

Conclusion
Our medical education innovation highlights the importance of using student feedback in the curriculum to address the biochemical response to social determinants of health. Peer course delivery and interactive clinical case creation facilitated reflective conversations amongst students and faculty. Nationally, healthcare becomes more equitable when providers are able to recognize and respond appropriately to the stress response resulting from racism and discrimination. 


Implementation of a Peer-Led and Peer-Facilitated Interactive Course on Racism-Related Stress and Culturally-Responsive Interactions with Patients in Clinical Settings

Abstract Title: 
Medical student reported patterns, attitudes, and challenges of interpreter use throughout core clinical clerkships

Background 
In the United States, around 30 million people report speaking English less than “very well.” This Limited English Proficiency (LEP) impacts all aspects of daily life, including medical care. LEP is associated with reduced access to care, higher rates of preventable errors, and impaired understanding of health overall. Despite the impact that LEP status has on health access and outcomes, medical students, who will become the next generation of doctors, continue to feel unprepared to care for the growing LEP patient population. This study sought to understand the practices, patterns, and attitudes of interpreter use encountered by medical students at the University of Pittsburgh on their core clinical rotations, with the goal of pinpointing challenges in accessing language services to be addressed in future additions to the pre-clinical curriculum. 

Methods
We developed a 17-item, novel survey instrument that assessed patterns of interpreter use in clinical settings, team attitudes towards interpreter use, and student-reported confidence and success in accessing interpreter services. The survey was developed using input from content experts and literature review and underwent pre-distribution testing and revision. It was administered electronically to 150 medical students after their first year of clinical rotations. Results were analyzed using descriptive statistics. 

Results
Of the 37 respondents, 97% of students reported encountering obstacles to accessing and providing interpretation services, citing reasons such as interpreter availability and interface unfamiliarity. 30% of students cited the supervising physician’s resistance to using an interpreter as a barrier that prevented access. Additionally, 65% of students reported that they felt the pre-clinical curriculum did not adequately prepare them to work with patients with LEP, and 35% of students stated that they did not feel as comfortable caring for patients with LEP as English-speaking patients.

Conclusion
Our findings suggest clear obstacles to interpreter use in the clinical setting, as well as a lack of confidence in implementing the skills needed to provide language-concordant care. At the same time, students recognized this deficiency and expressed desire to better learn these skills. These results will be used to shape a pre-clinical curriculum directed at teaching tangible skills to address these gaps prior to clinical training.


Medical Student Reported Patterns, Attitudes, and Challenges of Interpreter Use throughout Core Clinical Clerkships

LGBTQ+ health concerns have become increasingly prevalent in medical education, however, intersex individuals and their respective healthcare needs are frequently lumped into this category and not appropriately represented as standalone content in standard curricula. Intersex education is typically taught in conjunction with disorders of sexual development – an outdated and stigmatized term. The authors, who represent medical student and faculty educator stakeholders, aimed to develop a discrete intersex educational pilot training session to better inform how conversations supporting intersex individuals can be facilitated in undergraduate medical education.

Medical Student Sensitivity Training on the Differences of Sex Development

Melanated Suture Pad Implementation into Medical Education for Culturally Competent Curriculum Reform

Following the murder of George Floyd in Minneapolis on 5/25/20, and in the absence of a University-issued response in the days that followed, five Minnesotan medical students designed a dense one-week block of medical DEI curriculum to replace the previously scheduled content. The curriculum was formally proposed, and conditionally approved, and a few supportive faculty joined with the students to build the curriculum into Canvas. Each student assumed responsibility for one topic to be delivered over the course of one day and agreed upon a set or parameters. Though rapid, this was a highly collaborative and iterative process. We assessed student satisfaction through a course evaluation and found exceptionally high satisfaction, engagement, and perceived relevance. Through careful documentation and focused review, we identify core elements necessary for the success of this curriculum and we present a template for use in broader settings.


Minnesota 2020 Student-Driven Diversity, Equity, Inclusion, Social Justice Curriculum

Author name(s) and title(s):
Benjamin Johnson BS; Varun Rao, BS; Genaro DeLeon, MS

Background
Medical education currently stresses the importance of being aware of healthcare disparities and barriers to care through reflection papers and discussions. While these assignments play a valuable role in shaping awareness of social determinants of health awareness in future physicians, real action to combat these issues are often sidelined until students complete their education. Through Hoosier Health Hub a student led, pro-bono consulting group, students partnered with Super Shot, a local Fort Wayne, IN vaccine clinic for the underinsured and uninsured to gather information on reasons for missed vaccine appointments in hopes of putting a system in place that improves no-show rates. 

Methods
Hoosier Health Hub and Super Shot met together to discuss no show rates and current reminder processes in place. It was decided that Hoosier Health Hub could help Super Shot if more information was known about why patients missed their appointments and how they would prefer to be reminded. Hoosier Health Hub created a survey to be administered to patients who had no showed that was geared at understanding any barriers to arriving at appointments, difficulty scheduling, and preferred methods of communication for reminders. A group of first and second year medical students from Indiana University School of Medicine comprised the team. In their own time, students administered the survey, then compiled and analyzed the data. The information gathered was presented in a meeting with Super Shot over a zoom meeting.

Results
While Hoosier Health Hub did not seek IRB approval for the research of this project, Super Shot was pleased with the information we were able to provide them about their patient population. 

Conclusion
Student led pro-bono consulting is a direct way for students to actively address inequalities in healthcare. This project proved valuable to medical education by allowing students to learn first-hand how to help health care programs that are created to help those who are underinsured or uninsured who many times fall through the cracks of American health care. Valuable information was provided for Super Shot that lead the organization to ask for an on-going partnership to continue to survey their patient population so that they can continue to improve operations and meet the needs of their patients. American medical education could be enhanced by adopting a more hands on approach to addressing the needs of local communities while simultaneously providing students with valuable perspective that they can carry into their future practice. 


Pro-Bono Student Consulting Project Addressing Patient No-Show Rates for Underinsured and Uninsured Populations

Background
Race in medical education is often used as a proxy for a genetic ancestry or is otherwise lacking vital social or epidemiological context. As a result, medical training can reinforce false beliefs about race as a biomedical rather than social category. One study found that half of all trainees surveyed believed Black people possess less sensitive nerve endings, thicker skin, and more coagulable blood than White people. Yet, it remains unclear when such views arise and how they endure over time. The rationale for this study is to analyze medical students’ perception of race at two time points in the pre-clerkship phase and to explore strategies for instilling a holistic understanding of race and ethnicity in preclinical curricula.

Methods
First-year medical students at Medical College of Georgia completed a nine-question survey in September 2021 that called for Likert scale and short-answer responses. Data was deidentified, descriptive statistics were performed in Qualtrics, and responses were analyzed qualitatively in NVivo by coding keywords to identify relevant themes. Repeat survey is to be administered to same cohort in September 2022 followed by voluntary post-survey focus group.

Results
Likert (1 – 5) data indicates students (n = 97) moderately believe in a biological basis for race (3.45 ± 1.19) and somewhat believe in biological difference as a causal factor for racial health disparities (2.80 ± 1.18). Qualitative responses showed that 37 students believe race is relevant to diagnostic reasoning, while 50 students believe considering race can introduce anchoring and other biases.

Conclusion
Preliminary findings suggest students lack a cohesive understanding of race and ethnicity in the context of medical science. Though many responses view race through a social lens, responses overall tilt toward a biological lens. Limitations thus far include inability to determine underlying intent or seek clarification of ambiguous responses. Recently adopted policy at the AMA has called for eliminating race-based medicine, while select institutions and specialty societies have elected to remove racial multipliers from clinical algorithms (e.g. MDRD eGFR equation). Potential avenues for innovation in preclinical curricula include incorporating holistic competencies regarding race into problem-based learning, focus groups, and curriculum townhalls.


Researching and Remedying Perceptions of Race in Preclinical Medical Education

Background: In recent years there has been increased recognition regarding the importance of social determinants of health (SDOH) on health outcomes. Although SDOH awareness is already incorporated into medical education, there is a lack of comprehensive tools available to assess SDOH understanding, attitudes, and application in medicine (Samouei 2020). By determining the limitations in current medical education, this study aims to address shortcomings in SDOH education to provide optimal care to patients from diverse backgrounds by meeting their individual needs. 

Methods: Participants completed a pre-survey prior to the intervention.​ Participants were then presented with an educational PowerPoint on the social determinants of health. Post-presentation, participants were divided randomly into small groups to discuss 3 different cases incorporating SDOH​. Afterward, they completed a post-survey. 

Results: Participants are statistically significantly more comfortable talking with their future patients after the session on 10 out of 12 topics (the exceptions are gender identity and sexual orientation). These improvements remain significant on 6 topics (i.e., the community they practice, access to care, food insecurity, healthy diet, homelessness, and intimate partner violence) even after multiple comparison corrections using Benferroni procedure. 

Conclusion: The current study provides evidence that residents benefit from the inclusion of SDOH specific education in their curriculum as shown by our quantitative assessment. 

Social Determinants of Health Awareness in Emergency Medicine​

Background
The Caring for Communities of Color Conference (CCCC) is an annual student-led initiative that aims to educate healthcare trainees in health justice principles, expose them to primary care careers, connect them to clinicians in the field, and build a network of providers centering the health needs of communities of color. Developed to address gaps in medical school curricula regarding primary care and health advocacy for minoritized patients, the CCCC provides a unique approach to addressing healthcare disparities: shared learning through engagement with experts in the community.

Methods
Conference Organization: Conference objectives were defined as: research in partnership with communities, increased exposure to primary care careers, and serving communities of color in specialty professions. Fifteen second-year medical students developed virtual workshop content, sought out speakers, participated in social media marketing, designed survey tools to evaluate the impact of workshops, and obtained funding. 

Survey: Pre- and post-conference surveys were designed to evaluate changes in racial justice competency and awareness of primary care professions before and after the conference. Survey questions consisted of Likert scale responses, free-response, and demographic questions. All participants received the pre- and post-surveys via email. Both surveys received IRB exemption. 

Funding: Funding was provided by student organizations at the medical school.

Results
The inaugural conference occurred from August 31-September 2, 2020 with interactive workshops, dialogues with clinicians and patients, and a call-to-action. Speakers included 49 health professionals and over 400 participants nationwide. Participants included students in social work, public health, nursing, and medicine.

The pre-conference survey was administered to 400 individuals with a response rate of 40%. Data demonstrated that 60% of participants felt they did not have the skills necessary to care for communities of color in their line of work (n=160). The post-conference survey data were insufficient (n=15), but qualitative comments demonstrated that participants improved their knowledge and comfort in caring for communities of color. 

Conclusion
The CCCC expands upon the traditional confines of knowledge-sharing regarding health justice work in medical school curricula through collaboration and inter-professionalism amongst healthcare providers, students, and community activists. A 2021 conference has since been held.

Premium content

Enhancing Holistic Review During Residency Selection: Using Natural Language Processing of Applicants’ Experiences to Predict an Interview Invitation

Next from AMA Research Challenge 2022

Going Viral: Expanding a Student-Run COVID-19 Journal Club to Social Media

Similar lecture

The Bystander effect: Impact of rural hospital closures on the health of surrounding institutions

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES