United States

Obstructive sleep apnea (OSA) affects millions of U.S. Veterans and is linked to severe illnesses such as stroke and heart failure. However, understanding how OSA interacts with common comorbidities and the additive risk of OSA in developing severe illnesses is lacking. Large language models (LLMs) are promising tools for clinical natural language processing (NLP). These models are pre-trained on vast amounts of text data and learn to generate coherent and contextually appropriate text by predicting tokens in a sequence. This study explores the quality of LLM embeddings in clustering disease phenotypes related to OSA and its comorbidities such as atrial fibrillation, coronary artery disease, heart failure, hypertension, stroke, and type 2 diabetes mellitus. By identifying sub-cohorts of patients with similar diagnoses, procedures, medications, and adherence to continuous positive airway pressure (CPAP) treatment, we aim to score LLM embeddings by how well pre-trained LLMs discriminate OSA-related phenotypes. Document-level embeddings for discharge reports in the MIMIC-IV database were computed using NERSC&#39;s Perlmutter supercomputer. Twelve clinical LLMs of different sizes and pre-training schemes were evaluated including BioBART, BioBERT, BioGPT, BioMegatron, Gatortron, and RadBERT. To visualize the LLMs embedding, UMAP was used to enable plotting of embedded documents in 2D. Each set was clustered using K-Means with increasing values for K, and the purity of the clusters was measured using the Shannon entropy of cluster assignments. Finally, we test the performance of quantized document-level data for supervised phenotype classification.
Qualitatively, we examined the LLM embeddings along UMAP plots and reviewed K-Means cluster assignment by reading sampled text. Gatrotron_medium, BioGPT_large, and BioBERT_base were the best models at separating OSA from its comorbidities. Quantitatively, the clusters were measured on their Shannon entropy factor in which the lower value correlated to greater purity of clusters and vice versa. Gatortron_medium, Gatortron_base, and BioBART_base consistently scored with low entropy across comorbidities. The developed method selected the Gatortron_medium LLM as the best pre-trained model for OSA-related phenotypes both qualitatively and quantitatively. The method also has potential for assessing bias in LLMs by measuring how embeddings group patients according to social determinants of health such as gender and race. In the context of the Department of Veterans Affairs (VA) healthcare, there is a need for early OSA detection and prevention. Leveraging LLMs for feature discovery and risk factor identification, this research can provide valuable insights for allocating CPAP treatment effectively by grouping patients with similar health histories and OSA outcomes.

2023 AMA Research Challenge

Utilizing Large Language Models for DiseasePhenotyping in Obstructive Sleep Apnea

Obstructive sleep apnea (OSA) affects millions of U.S. Veterans and is linked to severe illnesses such as stroke and heart failure. However, understanding how OSA interacts with common comorbidities and the additive risk of OSA in developing severe illnesses is lacking. Large language models (LLMs) are promising tools for clinical natural language processing (NLP). These models are pre-trained on vast amounts of text data and learn to generate coherent and contextually appropriate text by predicting tokens in a sequence. This study explores the quality of LLM embeddings in clustering disease phenotypes related to OSA and its comorbidities such as atrial fibrillation, coronary artery disease, heart failure, hypertension, stroke, and type 2 diabetes mellitus. By identifying sub-cohorts of patients with similar diagnoses, procedures, medications, and adherence to continuous positive airway pressure (CPAP) treatment, we aim to score LLM embeddings by how well pre-trained LLMs discriminate OSA-related phenotypes. Document-level embeddings for discharge reports in the MIMIC-IV database were computed using NERSC's Perlmutter supercomputer. Twelve clinical LLMs of different sizes and pre-training schemes were evaluated including BioBART, BioBERT, BioGPT, BioMegatron, Gatortron, and RadBERT. To visualize the LLMs embedding, UMAP was used to enable plotting of embedded documents in 2D. Each set was clustered using K-Means with increasing values for K, and the purity of the clusters was measured using the Shannon entropy of cluster assignments. Finally, we test the performance of quantized document-level data for supervised phenotype classification.
Qualitatively, we examined the LLM embeddings along UMAP plots and reviewed K-Means cluster assignment by reading sampled text. Gatrotron_medium, BioGPT_large, and BioBERT_base were the best models at separating OSA from its comorbidities. Quantitatively, the clusters were measured on their Shannon entropy factor in which the lower value correlated to greater purity of clusters and vice versa. Gatortron_medium, Gatortron_base, and BioBART_base consistently scored with low entropy across comorbidities. The developed method selected the Gatortron_medium LLM as the best pre-trained model for OSA-related phenotypes both qualitatively and quantitatively. The method also has potential for assessing bias in LLMs by measuring how embeddings group patients according to social determinants of health such as gender and race. In the context of the Department of Veterans Affairs (VA) healthcare, there is a need for early OSA detection and prevention. Leveraging LLMs for feature discovery and risk factor identification, this research can provide valuable insights for allocating CPAP treatment effectively by grouping patients with similar health histories and OSA outcomes.

poster

Welcome to the **largest national, multi-specialty research** event featuring research from nearly 700 medical students, residents, fellows, and international medical graduates.

Use the buttons below or navigation menu to access the event.
 


[![](https://assets.underline.io/uploads/markdown_image/1/image/3e33a8c754cc49610b30950617d813e7.png)](https://underline.io/events/434/sessions?searchGroup=all)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/551404d429c4e274f9a54cbdd7081918.png)](https://underline.io/events/434/posters?searchGroup=all)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/677d92f566b880cebd36905fc738fc10.png)](https://underline.io/events/434/lounge)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/c7392833c7261c3874c5b43fa2f4df19.png)](https://underline.io/events/434/help-desk)
 

#### Event Details

The 2023 American Medical Association Research Challenge includes a poster symposium hall and a semifinals hall with the top scored research. **Log in at any time throughout the event to:**

* **View posters and presentations** on a variety of topics and specialties
* **Score posters in the semifinals** to help decide the top five finalists (AMA member exclusive opportunity)
* **Connect with participants** and provide feedback to help advance their research
* **Attend education sessions** with physician experts on key research topics

At the conclusion of this event, the AMA will announce the top five finalists who will advance to the AMA Research Challenge finals, present their research to an elite panel of judges, and compete to win a $10,000 grand prize sponsored by Laurel Road. 

### Grand Prize Sponsor

AMA would like to thank Laurel Road for sponsoring a $10,000 grand prize for the winner of the AMA Research Challenge. Laurel Road is not involved in the judging or scoring process to select the winner. The winner will be announced on February 4, 2024 during the AMA Research Challenge finals. Payment of the grand prize will be directly from Laurel Road. See the AMA Research Challenge Official Rules for terms and conditions.

![](https://assets.underline.io/uploads/markdown_image/1/image/611e331d4a0eba0920f240cfc04b4d77.png)

**There is no registration fee for this event! **

Register to explore research in a variety of topics. AMA members get exclusive access to score posters in the semifinals.

**Please use this link [here](https://www.ama-assn.org/about/events/ama-research-challenge) to register for free.**

Registration Is Required

Join the AMA Research Challenge poster symposium and semifinals to explore top research posters, score posters in the semifinals (an AMA member exclusive opportunity) and attend education sessions with experienced physicians to learn tips to advance your research and career. 

Background: Breast cancer is the second most common cancer amongst women in the United States. From 2013 to 2018, the breast cancer death rate fell by 1%; this decline was attributed to earlier screening and awareness. Early diagnosis of breast cancer can reduce morbidity and the intensity of treatment plans (ex: chemotherapies). There are many different screening tests to predict 5 year and lifetime risk of breast cancer. These screening tests and software programs have large variabilities in their risk prediction. Recent publications show a 5% to 19% overdiagnosis of breast cancer in women. In this study we predicted the 5-year risk, 10-year risk, and lifetime risk of breast cancer for mammogram patients using four different breast cancer screening tools: Mammorisk, Tyer Cuzick 2013 (TC 2013), Tyer Cuzick 2017 (TC 2017), and Gail. We compared their variability using t test statistical analysis. The purpose of our study is to determine the precision of these screening tests and begin streamlining the screening process for breast cancer. 

Methods: We collected Mammorisk data from 242 participants at Southwood Imaging (Youngstown, OH) who came in for their annual mammogram. Most women were between 40 to 70 years old. While participants waited to be screened, we asked them to fill out a questionnaire. The questionnaire contained 19 questions about the patient’s personal health, prior breast biopsies, and family history of breast cancer. We put this additional patient information into the additional screening tests Tyer Cuzick 2013, Tyer Cuzick 2017, and Gail. We calculated the 5-year risk and lifetime risk for each patient. We then measured the 5-year risk, 10-year risk, and lifetime risk assessment using t-test statistical analysis (two tailed). 

Results: The 5-year risk did not significantly differ between breast cancer risk models. The 10-year risk did not significantly differ between breast cancer risk models. The results showed statistically significant differences between lifetime risk for breast cancer risk models. Mammorisk and TC2017 lifetime risks differed by 0.544. The difference in percent lifetime risk between the Mammorisk and TC2013 models was -2.666. Mammorisk and Gail lifetime risks varied by 0.104. Even though our lifetime risk data was significant, the effect size was low . So some of our data may be inconclusive.

Conclusion: Our study indicated a lack of significant difference in 5-year and 10-year risk between models, but lifetime risk did significantly differ between breast cancer risk models. This creates variability that is difficult for physicians to interpret. This can affect physicians’ decision making and future patients’ health outcomes. Additional studies with a larger and more diverse population are needed for definitive evidence in variability between breast cancer risk models. Our study is a first step in finding a streamlined way to approach breast cancer risk in patients. In creating a streamlined approach radiologists will be able to care for cancer patients in a more organized manner.

Variability Between Different Breast Cancer risk Models and Effects on Various Parameters

**Scoring is only possible in the semifinals hall**

**Abstract:**



Evaluating Demographic Reporting in Publicly Available Prostate MRI Data Sets

Background:

The medical field is known to be male-predominated and in 2019, 64% of the total number of physicians were males. However, in 2017, females (50.5%) outnumbered males (49.4%) entering medical school for the first time. The gender disparity gap in medicine has been closing due to the increased number of women entering the field. The goal of this analysis is to predict gender differences in the medical profession. 


Methods: 

Data was collected from AAMC Physician Specialty Data Reports’, from 2007-2021. Statistical analysis, including a linear forecast, was used to determine predicted values implementing the equation. 

Results: 

Based on data from the AAMC, in 2007, 2011, 2013, 2015, 2017, 2019, and 2021, when looking at all subspecialties combined, there were 71.7%, 69.6% 67.4%, 66%, 64.8%, 63.7%, and 62.9% active male physicians, compared to 28.3%, 30.4%, 32.6%, 34%, 35.2%, 36.3%, and 37.1% active female physicians, respectively. Using the slopes of each data set (y = -0.6586x + 1393.5 for males and y = 0.6586x - 1293.5 for females), it is predicted that by the year 2040, there will be 50% active female physicians with a 95% CI [49.64-51.01] and 50% active male physicians with a 95% CI [48.99-50.36]. Furthermore, orthopedic surgery is the specialty with the greatest proportion of male physicians (94.1% male in 2021) while pediatrics is the specialty with the greatest proportion of female physicians (65% female in 2021). 


Conclusion:

These results conclude that females are underrepresented in medicine when looking at all specialties combined, however there has been a progressive rise in active female physicians. Moreover, 2017 was the first year females comprised the majority of medical students. It is predicted that the number of active female and male physicians will become equal by the year 2040.

A Predictive Analysis of Gender Disparities in the Medical Profession

Background: There is no standardized sexual assault (SA) or trauma-informed care (TIC) curriculum in medical education, resulting in little to no exposure to these topics for many medical students. The goal of this project is to evaluate medical student perceptions of TIC before and after a weeklong SA/TIC curriculum. We hypothesize that students will be more willing to use TIC strategies and think that it is more important after they have been introduced to instruction on the topic.
Methods: All second-year medical students at MCG participated in a weeklong SA/TIC curriculum in August 2021. Students were tasked with learning about TIC during the Case-Based Learning (CBL) sessions and a standardized patient encounter in Patient-Centered Learning (PCL). Students were given a voluntary survey on their perceptions regarding TIC before (pre-survey) and after (post-survey) this week as part of the curriculum. The survey was adapted from Cannon, et al. 2020 and used 5-point Likert scale (1 = strongly disagree, 5 = strongly agree) responses to 17 questions. Data were coded by the curriculum office and de-identified. After IRB approval, quantitative data were analyzed using descriptive statistics and paired t-tests.
Results: In total, 193 medical students participated in the SA/TIC curriculum. There were 101 pre-surveys and 99 post-week surveys completed for a response rate of 52.3% and 51.3% respectively. Results from the survey demonstrated significant gains in knowledge, skills, and attitudes and significant improvement towards trauma informed care overall on average (+0.360, p<0.001).
Conclusion: Our results demonstrate that having a TIC curriculum significantly improved medical students’ knowledge of and attitude towards providing trauma-informed care. This curriculum can be reproduced at other institutions to introduce future physicians across all specialties to principles of SA/TIC care to improve health outcomes.

Assessing Medical Students’ Perceptions of Trauma-Informed Care Beforeand After a Pilot Sexual Assault and Trauma-Informed Care Curriculum

Trained medical interpreters are crucial to ensure that patients who speak English with limited proficiency (LEP) receive high-quality care
Current medical education lacks formal training that incorporates practice in simulated environments 
We sought to evaluate MS1 and MS2 ability to use interpreters and address knowledge gaps through near-peer training sessions

Analyzing Medical Students’ Preparation and Confidence with Interpreters

Background: Physicians often cite finances as a significant source of burnout and report that the prospect of future earnings played a role in specialty choice. Basic financial literacy is not a required prerequisite for medical school, nor is it mandated as a part of medical school curriculum. Therefore, we wanted to determine medical students’ amount of debt, baseline financial literacy, and financial anxiety and compare this to their financial literacy and financial anxiety following a basic financial education class. 

Methods: Two hundred first semester medical students at the public medical school in Georgia were surveyed immediately before and after a 45-minute lecture and a small group activity on basic finances. The survey included a financial education questionnaire which included financial literacy and attitudes about financial education and the financial anxiety scale which was adapted from the DSM-5. Both scales were graded on a 10-point Likert scale, with 10 being strongly agree. Demographic questions and participant characteristics were included on the pre-survey. 

Results: The presurvey had a 77% response rate and the postsurvey had a 30% response rate. Of participants, 60.4% were between 22-24 years of age and 97.4% did not have dependents. Almost half (43.5%) reported $0 of debt of current debt, and 17.5% reported between $25,000 and $50,000 of debt from medical school. Participants’ financial literacy significantly improved following the basic financial education (p<0.05) in seven of the nine questions asked, except for students’ beliefs that they would be able to pay off their education debt and meet financial goals within a reasonable time. None of the financial anxiety measures significantly changed after the education session (p>0.05). 

Conclusion: Incorporating basic financial education into medical curriculum improves medical student financial literacy but is not sufficient to reduce financial anxiety. Future studies should look at longer term educational courses to determine if financial anxiety can also be mitigated among medical students. Based on physicians citing finances as a significant source of burn out, by including basic financial education into medical school curriculum, we may be able to buffer physician burnout and provide aid in the effort to mitigate the growing physician shortage in America.

Empowering Future Physicians: Enhancing Financial Literacy among Medical Students

Background. Dialogue is recognized as a crucial medical competency that promotes adherence to medical regimens and health outcomes. Pain and substance use disorders (SUD) are frequently stigmatized, and its education in medical schools is currently fragmented across other courses. The LICENSE (Language, Impact, Communication, Engagement, Non-Stigmatizing, Effectiveness) curriculum is a longitudinal course, with the overall goal of teaching medical students how to better communicate with patients with SUDs. It describes the intersection of SUD with Social Determinants of Health (SDH) and treatment adherence, and how to conduct effective clinical interviews with this patient population.

Method. Students interact with this curriculum across the three phases of Stony Brook’s medical curriculum. At each phase, the Substance Abuse Attitude Survey (SAAS), a validated survey assessing attitudes about SUD, is administered. A 60-minute didactic session is then followed by case presentations and reflection questions. 

Results. Currently, this curriculum has been implemented across four years of medical students, with 290 participants for our data collection. Thus far, the qualitative data has shown an increase in awareness of SUD and of potential concerns of SUD. More, the SAAS values are trending toward less stigmatizing answers. For example, in phase 2, the statement “drug addiction is a treatable illness” is more strongly supported relative to phase 1. 

Future Directions. This study hopes to show that increased education on SUD and SDH helps to decrease stigma in these patient populations. We plan to continue this data collection and compare the SAAS data across all the phases of the curriculum to determine if attitudes change over time. We plan to also implement the Phase III portion of the curriculum starting with the graduating class of 2024. This portion will be developed and carried out in the spring of 2024.

Innovative Learning and Teaching About Substance Use/Opioid Use Disorders

Background
Several residency advisor opinion pieces have been published on the transition of the USMLE Step 1 exam from a numerical score to pass/fail since January 2022. However, there have been few assessments of medical student perspectives. The impact of this change on minoritized medical students including those from racial/ethnic backgrounds underrepresented in medicine (URiM; Black/African American, Hispanic/Latinx, American Indian/Alaska Native, and Pacific Islander/Native Hawaiian), LGBTQ+, or first-generation college students is poorly understood. The decision for this scoring transition was multifactorial, including a potential to increase medical student well-being and diversity and inclusion in residency selection. We investigate the impact of the pass/fail Step 1 scoring on wellness, career planning, and resource access from a medical student perspective with comparisons between minoritized and represented students.
Methods
Medical students at a single LCME accredited institution who had completed the USMLE Step 1 exam in 2022 or 2023 and received a pass/fail score were recruited for this study. Participants completed a 27-question anonymous electronic survey and were asked to self-report demographics, sense of wellness, career planning, and access to study resources during their dedicated study period. Statistical analyses with chi-squared tests were conducted using R.
Results
There were 128 eligible study participants; 55 identified as minoritized, including URiM (18.0%), LBGTQ+ (19.5%), or first-generation college student (25.0%). Overall, 66.4% of students felt that the change to pass/fail scoring improved student wellness in their class. However, 28.9% reported that it also affected their interest or confidence in pursuing certain specialties, and 75.8% reported feeling pressured to pursue more research and extracurricular activities. Many students (49.2%) reported “poor” or “terrible” wellness while preparing for the exam, including URiM (69.6% vs. 44.8%, p=0.054) and all minoritized students (58.2% vs. 42.5%, p=0.114) compared to their respective counterparts. Minoritized students (38.2% vs. 19.2%, p=0.029), especially URiM (52.2% vs. 21.9%, p=0.007), were more likely to report poor financial access to exam preparation resources compared to their respective counterparts. 
Conclusion
Although generalizability is limited by sample size at a single institution, a large proportion of medical students continue to experience poor wellness while preparing for the pass/fail Step 1 exam. Minoritized students, especially those URiM, report more financial barriers in accessing resources for Step 1 preparation. The scoring transition has heightened uncertainty regarding how to prepare for residency. It is imperative that medical schools expand career advising efforts and continually update recommendations to ensure adequate preparation for residency applications.

Perspectives on the Pass/Fail USMLE Step 1: Comparisons between Minoritized & Represented Students

Students from historically marginalized backgrounds have fewer role models and less access to resources that might spark their interest in healthcare careers. Medical students who engage 6th-12th grade students in discussion about careers in health will increase future applicant diversity. 


Medical students from the University of Utah School of Medicine visited 18 public middle or high school classes in the Salt Lake City valley. There were 201 participants total who ranged in age from 12 to 18 years old. 16% of participants reported that they had at least one parent who did not graduate high school. 60% of participants reported that they did not have an immediate family member employed in health care. 58% of participants were from families of low income. Presentations on various medical specialties were given to educate the participants on the different careers available in the medical field. Activities such as ultrasonography, blood pressure measuring/interpretation, MOCA exams, and proper stethoscope usage were also conducted to give participants hands-on experience of what clinicians do on a daily basis. Question-and-answer discussions were led at the closure of the presentation to help provide mentorship to participants interested in going into the medical field. A survey was handed out to each participant to compare their level of interest in pursuing a career in medicine prior to and after the session. 


As a result of the sessions, there was a 17.9% increase among participants who were interested in pursuing a career in medicine.


82.9% of participants either agreed or strongly agreed that the sessions were effective in helping them learn about careers as a clinician. Among high school participants, 90.9% reported that the sessions were effective in helping them learn about careers as a clinician. In contrast to middle school participants, only 75% reported that the sessions were effective.


46% of participants identified as Hispanic or Latino, 5% of participants identified as African American, 11% of participants identified as Asian or Pacific Islander, 1% of participants identified as Native American or Alaskan Native, 33% of participants identified as Caucasian, and 4% of participants identified as other.


Direct peer-to-peer mentoring improves student perceptions regarding accessibility and interest of a healthcare career. Conducting community outreach and consistently analyzing the way we talk about health careers to students will populate a strong, diverse applicant pool for admission into health science fields. Diversity will contribute to a rich student/physician experience and equip them with the perspectives and skill sets needed to advance health equity, reduce health disparities, and meet the needs of our changing community.


Increasing Student Interest in Health Careers among Diverse Populations

Background: Food insecurity is an issue impacting the health and well-being of patients across the US. Patients experiencing food insecurity are at higher risk for poor health outcomes than those with food security. These findings support the need for engagement with this pillar of social determinants of health (SDoH) in the medical school curriculum. The preclinical curriculum affords limited opportunities for medical students to engage with SDoH, specifically food insecurity, in an experiential learning model. This pilot project sought to create a learning opportunity that challenged existing perspectives on the experience of being food-insecure while promoting awareness of existing patient resources in Vermont.

Methods: Modeled after the Food Research & Action Center's (FRAC) SNAP Challenge, the Larner College of Medicine (LCOM) Eat on SNAP Challenge invited the medical community to eat on Vermont's SNAP-allotted budget of $6/day for one week. Participants were encouraged to attend educational events, including community resources, service opportunities, strategies for clinical practice, and research. We utilized pre-and post-participation survey data to assess knowledge and understanding of community resources and the experience and impact of food insecurity on VT patient populations. 

Results: Of the pre-event survey respondents (n=24), 52% identified as male, and 62% identified as white. 81% of respondents had never received SNAP benefits in the past. 37% of respondents believed SNAP was "slightly effective," while 33% were unfamiliar with the SNAP program at all. Of the post-event survey respondents (n=11), 78% had
participated in the Challenge, and 88% of post-event respondents answered "possibly to definitely yes" when asked if they would participate in this event next year. Importantly, using a scale from 0 to 100, we asked how strongly people felt that LCOM should add more information to the curriculum on food insecurity and resources, and the responses had a mean of 90 (SD= 11.80, n=7).

Conclusion: This pilot program identified gaps in medical student knowledge and experience with food insecurity, both personally and academically. The program provides an experiential educational opportunity to better engage with patients on food insecurity and effective mitigation strategies within patient care. Our survey results identified medical students' desire to learn more about food insecurity in their preclinical curriculum. Despite the limited sample size of this study, this pilot program exemplifies an opportunity for medical schools to implement programming for their students to engage with SDoH health outside the classroom and within the community.


Premium content

Utilizing Large Language Models for DiseasePhenotyping in Obstructive Sleep Apnea

Next from 2023 AMA Research Challenge

Variability Between Different Breast Cancer risk Models and Effects on Various Parameters

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES