United States

Background: Predicting the discharge destination, otherwise known as discharge disposition, can offer several advantages for the inpatient rehabilitation patient, including treatment and discharge planning. Machine learning (ML) models have demonstrated a strong predictive capacity for discharge disposition, particularly for patients with specific diagnoses such as traumatic brain injuries, or in specific settings such as acute neurologic care, or among specific populations. 



Objective: Explore various ML models’ ability to predict inpatient rehabilitation discharge disposition to home versus non-home using patient encounter data. 



Methods: Patient data included demographics, diagnoses, lab values, length of stay, and insurance status from all inpatient rehabilitation encounters at a large midwestern health system occurring between 2015 and 2022 (N=4919). We compared two commonly used ML approaches: a random forest (RF) classifier and a decision tree (DT) classifier. Two bagging classifiers, which aggregate the results of multiple RF or DT classifiers to improve predictions, were also tested. Because the imbalance in number between home (N=3687) and non-home discharges (N=1232) can impair predictions, balancing techniques were used prior to training the ML classifiers. The ML classifiers were then optimized with grid search and cross validation. Performance of the ML classifiers was determined by the receiver operating characteristic area under the curve (AUC) – a measure of the ability to discriminate the true discharge disposition, accuracy (computed using the Jaccard Index), and macro f1-score (indicative of classification ability across classes). 



Results: 

A bagging classifier using RF showed highest predictive performance in terms of AUC (0.68) and accuracy (0.75), however it struggled to predict non-home discharges (macro f1-score = 0.54). A bagging DT classifier improved prediction of non-home discharges (macro f1-score = 0.59), however AUC (0.65) and accuracy (0.69) were lower. Generally, bagging classifiers combining the predictions from multiple RF or DT classifiers improved predictive performance compared to single RF or DT classifiers. 



Conclusions: As artificial intelligence and ML continue to grow in healthcare applications, this study showcased the effectiveness of ML approaches to predict discharge disposition from inpatient rehabilitation. Our findings complement previous studies in other diagnoses and settings, due to our diversity of available variables and differences in our patient population and treatment paradigms. Of note, utilization of bagging classifiers proved valuable in enhancing the performance of the tested base classifiers. Our results may help guide optimization of ML approaches to predict discharge disposition, in hopes of improving patients&#39; hospital courses and post-hospital planning and treatment.

AMA Research Challenge 2023

Evaluation of Machine Learning Approaches for Predicting Inpatient Rehabilitation Patient Discharge Disposition

### Welcome!
The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. Our Virtual Poster Sessions will take place online Thursday, August 22, 2024.

You are required to register for this event. **Please register [here](https://2024.aclweb.org/registration). **

If you have already registered, please check your inbox for an email from Underline granting you access to ACL 2024 content.

Please register!

ACL 2024

The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. More information will be announced soon.

Background: Predicting the discharge destination, otherwise known as discharge disposition, can offer several advantages for the inpatient rehabilitation patient, including treatment and discharge planning. Machine learning (ML) models have demonstrated a strong predictive capacity for discharge disposition, particularly for patients with specific diagnoses such as traumatic brain injuries, or in specific settings such as acute neurologic care, or among specific populations. 



Objective: Explore various ML models’ ability to predict inpatient rehabilitation discharge disposition to home versus non-home using patient encounter data. 



Methods: Patient data included demographics, diagnoses, lab values, length of stay, and insurance status from all inpatient rehabilitation encounters at a large midwestern health system occurring between 2015 and 2022 (N=4919). We compared two commonly used ML approaches: a random forest (RF) classifier and a decision tree (DT) classifier. Two bagging classifiers, which aggregate the results of multiple RF or DT classifiers to improve predictions, were also tested. Because the imbalance in number between home (N=3687) and non-home discharges (N=1232) can impair predictions, balancing techniques were used prior to training the ML classifiers. The ML classifiers were then optimized with grid search and cross validation. Performance of the ML classifiers was determined by the receiver operating characteristic area under the curve (AUC) – a measure of the ability to discriminate the true discharge disposition, accuracy (computed using the Jaccard Index), and macro f1-score (indicative of classification ability across classes). 



Results: 

A bagging classifier using RF showed highest predictive performance in terms of AUC (0.68) and accuracy (0.75), however it struggled to predict non-home discharges (macro f1-score = 0.54). A bagging DT classifier improved prediction of non-home discharges (macro f1-score = 0.59), however AUC (0.65) and accuracy (0.69) were lower. Generally, bagging classifiers combining the predictions from multiple RF or DT classifiers improved predictive performance compared to single RF or DT classifiers. 



Conclusions: As artificial intelligence and ML continue to grow in healthcare applications, this study showcased the effectiveness of ML approaches to predict discharge disposition from inpatient rehabilitation. Our findings complement previous studies in other diagnoses and settings, due to our diversity of available variables and differences in our patient population and treatment paradigms. Of note, utilization of bagging classifiers proved valuable in enhancing the performance of the tested base classifiers. Our results may help guide optimization of ML approaches to predict discharge disposition, in hopes of improving patients' hospital courses and post-hospital planning and treatment.

poster

Welcome to the **largest national, multi-specialty research** event featuring research from nearly 700 medical students, residents, fellows, and international medical graduates.

Use the buttons below or navigation menu to access the event.
 


[![](https://assets.underline.io/uploads/markdown_image/1/image/3e33a8c754cc49610b30950617d813e7.png)](https://underline.io/events/434/sessions?searchGroup=all)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/551404d429c4e274f9a54cbdd7081918.png)](https://underline.io/events/434/posters?searchGroup=all)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/677d92f566b880cebd36905fc738fc10.png)](https://underline.io/events/434/lounge)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/c7392833c7261c3874c5b43fa2f4df19.png)](https://underline.io/events/434/help-desk)
 

#### Event Details

The 2023 American Medical Association Research Challenge includes a poster symposium hall and a semifinals hall with the top scored research. **Log in at any time throughout the event to:**

* **View posters and presentations** on a variety of topics and specialties
* **Score posters in the semifinals** to help decide the top five finalists (AMA member exclusive opportunity)
* **Connect with participants** and provide feedback to help advance their research
* **Attend education sessions** with physician experts on key research topics

At the conclusion of this event, the AMA will announce the top five finalists who will advance to the AMA Research Challenge finals, present their research to an elite panel of judges, and compete to win a $10,000 grand prize sponsored by Laurel Road. 

### Grand Prize Sponsor

AMA would like to thank Laurel Road for sponsoring a $10,000 grand prize for the winner of the AMA Research Challenge. Laurel Road is not involved in the judging or scoring process to select the winner. The winner will be announced on February 4, 2024 during the AMA Research Challenge finals. Payment of the grand prize will be directly from Laurel Road. See the AMA Research Challenge Official Rules for terms and conditions.

![](https://assets.underline.io/uploads/markdown_image/1/image/611e331d4a0eba0920f240cfc04b4d77.png)

**There is no registration fee for this event! **

Register to explore research in a variety of topics. AMA members get exclusive access to score posters in the semifinals.

**Please use this link [here](https://www.ama-assn.org/about/events/ama-research-challenge) to register for free.**

Registration Is Required

Join the AMA Research Challenge poster symposium and semifinals to explore top research posters, score posters in the semifinals (an AMA member exclusive opportunity) and attend education sessions with experienced physicians to learn tips to advance your research and career. 

Background
Post Traumatic Stress Disorder (PTSD) is a disorder characterized by exposure to some traumatic event leaving the individual with a variety of symptoms that can result in debilitating disability. Treatment has historically been centered around Cognitive Processing Therapy (CPT) and literature has shown a significant risk of suicide in those left untreated. Thus, the importance of appropriate healthcare utilization is important for addressing morbidity and mortality of PTSD. Determining appropriate utilization is not well recorded in the literature though some studies show increased utilization with increased symptom severity. The aim of
this study is to replicate these findings and determine if there exists any range of symptom severity that over or under-utilizes the healthcare system.

Methods
Data was collected by Rosen et al. 2012 through Project VALOR, a longitudinal registry of PTSD in combat exposed veterans of Afghanistan and Iraqi military theaters. PTSD checklist (PCL) scores and data regarding number of healthcare utilizations for PTSD treatment year to year were recorded over a five-year period. Utilization of healthcare services for PTSD were stratified by this study into categories and compared against PCL scores using an ordinal logistical regression model to predict probability of being placed in a higher utilization category. Quadratic analysis was conducted on the PCL scores to determine linear relationship of findings.

Results
Ordinal logistical regression of mean centered PCL value was 1.042 (p=0.000). Quadratic effect of ordinal logistical regression of mean centered PCL2 was determined to be 0.99 (p=0.061).

Conclusion
Self-reported PCL scores are positively predictive of healthcare utilization. For every 1 increase in PCL score, there is a predicted 4.29% increased chance of being placed into the next highest utilization category (p=0.000). In addition, the relationship between PCL score and healthcare utilization was determined to be log-linear due to quadratic effect being found not significant 0.99 (p=0.061). This indicates no PCL score is correlated with increased probability of increased or decreased healthcare utilization. This likely represents healthcare services are appropriately being administered based on symptom severity and no group is over or under-utilizing resources. Ultimately, this study replicates previous literature that increased severity is associated with increased healthcare utilization while also quantifying the predictive value of increasing PCL scores. Currently the healthcare system is correctly identifying and capturing individuals who would in theory benefit from increased healthcare utilization. Additional research into capture strength would be beneficial in determining impact of hospital system capture rate of at risk patients.

Increasing PTSD Symptom Severity Predicts Increasing Healthcare Utilization Among OEF/OIF Veterans.

Obstructive sleep apnea (OSA) affects millions of U.S. Veterans and is linked to severe illnesses such as stroke and heart failure. However, understanding how OSA interacts with common comorbidities and the additive risk of OSA in developing severe illnesses is lacking. Large language models (LLMs) are promising tools for clinical natural language processing (NLP). These models are pre-trained on vast amounts of text data and learn to generate coherent and contextually appropriate text by predicting tokens in a sequence. This study explores the quality of LLM embeddings in clustering disease phenotypes related to OSA and its comorbidities such as atrial fibrillation, coronary artery disease, heart failure, hypertension, stroke, and type 2 diabetes mellitus. By identifying sub-cohorts of patients with similar diagnoses, procedures, medications, and adherence to continuous positive airway pressure (CPAP) treatment, we aim to score LLM embeddings by how well pre-trained LLMs discriminate OSA-related phenotypes. Document-level embeddings for discharge reports in the MIMIC-IV database were computed using NERSC's Perlmutter supercomputer. Twelve clinical LLMs of different sizes and pre-training schemes were evaluated including BioBART, BioBERT, BioGPT, BioMegatron, Gatortron, and RadBERT. To visualize the LLMs embedding, UMAP was used to enable plotting of embedded documents in 2D. Each set was clustered using K-Means with increasing values for K, and the purity of the clusters was measured using the Shannon entropy of cluster assignments. Finally, we test the performance of quantized document-level data for supervised phenotype classification.
Qualitatively, we examined the LLM embeddings along UMAP plots and reviewed K-Means cluster assignment by reading sampled text. Gatrotron_medium, BioGPT_large, and BioBERT_base were the best models at separating OSA from its comorbidities. Quantitatively, the clusters were measured on their Shannon entropy factor in which the lower value correlated to greater purity of clusters and vice versa. Gatortron_medium, Gatortron_base, and BioBART_base consistently scored with low entropy across comorbidities. The developed method selected the Gatortron_medium LLM as the best pre-trained model for OSA-related phenotypes both qualitatively and quantitatively. The method also has potential for assessing bias in LLMs by measuring how embeddings group patients according to social determinants of health such as gender and race. In the context of the Department of Veterans Affairs (VA) healthcare, there is a need for early OSA detection and prevention. Leveraging LLMs for feature discovery and risk factor identification, this research can provide valuable insights for allocating CPAP treatment effectively by grouping patients with similar health histories and OSA outcomes.

Utilizing Large Language Models for DiseasePhenotyping in Obstructive Sleep Apnea

Background: Breast cancer is the second most common cancer amongst women in the United States. From 2013 to 2018, the breast cancer death rate fell by 1%; this decline was attributed to earlier screening and awareness. Early diagnosis of breast cancer can reduce morbidity and the intensity of treatment plans (ex: chemotherapies). There are many different screening tests to predict 5 year and lifetime risk of breast cancer. These screening tests and software programs have large variabilities in their risk prediction. Recent publications show a 5% to 19% overdiagnosis of breast cancer in women. In this study we predicted the 5-year risk, 10-year risk, and lifetime risk of breast cancer for mammogram patients using four different breast cancer screening tools: Mammorisk, Tyer Cuzick 2013 (TC 2013), Tyer Cuzick 2017 (TC 2017), and Gail. We compared their variability using t test statistical analysis. The purpose of our study is to determine the precision of these screening tests and begin streamlining the screening process for breast cancer. 

Methods: We collected Mammorisk data from 242 participants at Southwood Imaging (Youngstown, OH) who came in for their annual mammogram. Most women were between 40 to 70 years old. While participants waited to be screened, we asked them to fill out a questionnaire. The questionnaire contained 19 questions about the patient’s personal health, prior breast biopsies, and family history of breast cancer. We put this additional patient information into the additional screening tests Tyer Cuzick 2013, Tyer Cuzick 2017, and Gail. We calculated the 5-year risk and lifetime risk for each patient. We then measured the 5-year risk, 10-year risk, and lifetime risk assessment using t-test statistical analysis (two tailed). 

Results: The 5-year risk did not significantly differ between breast cancer risk models. The 10-year risk did not significantly differ between breast cancer risk models. The results showed statistically significant differences between lifetime risk for breast cancer risk models. Mammorisk and TC2017 lifetime risks differed by 0.544. The difference in percent lifetime risk between the Mammorisk and TC2013 models was -2.666. Mammorisk and Gail lifetime risks varied by 0.104. Even though our lifetime risk data was significant, the effect size was low . So some of our data may be inconclusive.

Conclusion: Our study indicated a lack of significant difference in 5-year and 10-year risk between models, but lifetime risk did significantly differ between breast cancer risk models. This creates variability that is difficult for physicians to interpret. This can affect physicians’ decision making and future patients’ health outcomes. Additional studies with a larger and more diverse population are needed for definitive evidence in variability between breast cancer risk models. Our study is a first step in finding a streamlined way to approach breast cancer risk in patients. In creating a streamlined approach radiologists will be able to care for cancer patients in a more organized manner.

Variability Between Different Breast Cancer risk Models and Effects on Various Parameters

**Scoring is only possible in the semifinals hall**

**Abstract:**



Evaluating Demographic Reporting in Publicly Available Prostate MRI Data Sets

Background:

The medical field is known to be male-predominated and in 2019, 64% of the total number of physicians were males. However, in 2017, females (50.5%) outnumbered males (49.4%) entering medical school for the first time. The gender disparity gap in medicine has been closing due to the increased number of women entering the field. The goal of this analysis is to predict gender differences in the medical profession. 


Methods: 

Data was collected from AAMC Physician Specialty Data Reports’, from 2007-2021. Statistical analysis, including a linear forecast, was used to determine predicted values implementing the equation. 

Results: 

Based on data from the AAMC, in 2007, 2011, 2013, 2015, 2017, 2019, and 2021, when looking at all subspecialties combined, there were 71.7%, 69.6% 67.4%, 66%, 64.8%, 63.7%, and 62.9% active male physicians, compared to 28.3%, 30.4%, 32.6%, 34%, 35.2%, 36.3%, and 37.1% active female physicians, respectively. Using the slopes of each data set (y = -0.6586x + 1393.5 for males and y = 0.6586x - 1293.5 for females), it is predicted that by the year 2040, there will be 50% active female physicians with a 95% CI [49.64-51.01] and 50% active male physicians with a 95% CI [48.99-50.36]. Furthermore, orthopedic surgery is the specialty with the greatest proportion of male physicians (94.1% male in 2021) while pediatrics is the specialty with the greatest proportion of female physicians (65% female in 2021). 


Conclusion:

These results conclude that females are underrepresented in medicine when looking at all specialties combined, however there has been a progressive rise in active female physicians. Moreover, 2017 was the first year females comprised the majority of medical students. It is predicted that the number of active female and male physicians will become equal by the year 2040.

A Predictive Analysis of Gender Disparities in the Medical Profession

Background: There is no standardized sexual assault (SA) or trauma-informed care (TIC) curriculum in medical education, resulting in little to no exposure to these topics for many medical students. The goal of this project is to evaluate medical student perceptions of TIC before and after a weeklong SA/TIC curriculum. We hypothesize that students will be more willing to use TIC strategies and think that it is more important after they have been introduced to instruction on the topic.
Methods: All second-year medical students at MCG participated in a weeklong SA/TIC curriculum in August 2021. Students were tasked with learning about TIC during the Case-Based Learning (CBL) sessions and a standardized patient encounter in Patient-Centered Learning (PCL). Students were given a voluntary survey on their perceptions regarding TIC before (pre-survey) and after (post-survey) this week as part of the curriculum. The survey was adapted from Cannon, et al. 2020 and used 5-point Likert scale (1 = strongly disagree, 5 = strongly agree) responses to 17 questions. Data were coded by the curriculum office and de-identified. After IRB approval, quantitative data were analyzed using descriptive statistics and paired t-tests.
Results: In total, 193 medical students participated in the SA/TIC curriculum. There were 101 pre-surveys and 99 post-week surveys completed for a response rate of 52.3% and 51.3% respectively. Results from the survey demonstrated significant gains in knowledge, skills, and attitudes and significant improvement towards trauma informed care overall on average (+0.360, p<0.001).
Conclusion: Our results demonstrate that having a TIC curriculum significantly improved medical students’ knowledge of and attitude towards providing trauma-informed care. This curriculum can be reproduced at other institutions to introduce future physicians across all specialties to principles of SA/TIC care to improve health outcomes.

Assessing Medical Students’ Perceptions of Trauma-Informed Care Beforeand After a Pilot Sexual Assault and Trauma-Informed Care Curriculum

Trained medical interpreters are crucial to ensure that patients who speak English with limited proficiency (LEP) receive high-quality care
Current medical education lacks formal training that incorporates practice in simulated environments 
We sought to evaluate MS1 and MS2 ability to use interpreters and address knowledge gaps through near-peer training sessions

Analyzing Medical Students’ Preparation and Confidence with Interpreters

Background: Physicians often cite finances as a significant source of burnout and report that the prospect of future earnings played a role in specialty choice. Basic financial literacy is not a required prerequisite for medical school, nor is it mandated as a part of medical school curriculum. Therefore, we wanted to determine medical students’ amount of debt, baseline financial literacy, and financial anxiety and compare this to their financial literacy and financial anxiety following a basic financial education class. 

Methods: Two hundred first semester medical students at the public medical school in Georgia were surveyed immediately before and after a 45-minute lecture and a small group activity on basic finances. The survey included a financial education questionnaire which included financial literacy and attitudes about financial education and the financial anxiety scale which was adapted from the DSM-5. Both scales were graded on a 10-point Likert scale, with 10 being strongly agree. Demographic questions and participant characteristics were included on the pre-survey. 

Results: The presurvey had a 77% response rate and the postsurvey had a 30% response rate. Of participants, 60.4% were between 22-24 years of age and 97.4% did not have dependents. Almost half (43.5%) reported $0 of debt of current debt, and 17.5% reported between $25,000 and $50,000 of debt from medical school. Participants’ financial literacy significantly improved following the basic financial education (p<0.05) in seven of the nine questions asked, except for students’ beliefs that they would be able to pay off their education debt and meet financial goals within a reasonable time. None of the financial anxiety measures significantly changed after the education session (p>0.05). 

Conclusion: Incorporating basic financial education into medical curriculum improves medical student financial literacy but is not sufficient to reduce financial anxiety. Future studies should look at longer term educational courses to determine if financial anxiety can also be mitigated among medical students. Based on physicians citing finances as a significant source of burn out, by including basic financial education into medical school curriculum, we may be able to buffer physician burnout and provide aid in the effort to mitigate the growing physician shortage in America.

Empowering Future Physicians: Enhancing Financial Literacy among Medical Students

Background. Dialogue is recognized as a crucial medical competency that promotes adherence to medical regimens and health outcomes. Pain and substance use disorders (SUD) are frequently stigmatized, and its education in medical schools is currently fragmented across other courses. The LICENSE (Language, Impact, Communication, Engagement, Non-Stigmatizing, Effectiveness) curriculum is a longitudinal course, with the overall goal of teaching medical students how to better communicate with patients with SUDs. It describes the intersection of SUD with Social Determinants of Health (SDH) and treatment adherence, and how to conduct effective clinical interviews with this patient population.

Method. Students interact with this curriculum across the three phases of Stony Brook’s medical curriculum. At each phase, the Substance Abuse Attitude Survey (SAAS), a validated survey assessing attitudes about SUD, is administered. A 60-minute didactic session is then followed by case presentations and reflection questions. 

Results. Currently, this curriculum has been implemented across four years of medical students, with 290 participants for our data collection. Thus far, the qualitative data has shown an increase in awareness of SUD and of potential concerns of SUD. More, the SAAS values are trending toward less stigmatizing answers. For example, in phase 2, the statement “drug addiction is a treatable illness” is more strongly supported relative to phase 1. 

Future Directions. This study hopes to show that increased education on SUD and SDH helps to decrease stigma in these patient populations. We plan to continue this data collection and compare the SAAS data across all the phases of the curriculum to determine if attitudes change over time. We plan to also implement the Phase III portion of the curriculum starting with the graduating class of 2024. This portion will be developed and carried out in the spring of 2024.

Innovative Learning and Teaching About Substance Use/Opioid Use Disorders

Background
Several residency advisor opinion pieces have been published on the transition of the USMLE Step 1 exam from a numerical score to pass/fail since January 2022. However, there have been few assessments of medical student perspectives. The impact of this change on minoritized medical students including those from racial/ethnic backgrounds underrepresented in medicine (URiM; Black/African American, Hispanic/Latinx, American Indian/Alaska Native, and Pacific Islander/Native Hawaiian), LGBTQ+, or first-generation college students is poorly understood. The decision for this scoring transition was multifactorial, including a potential to increase medical student well-being and diversity and inclusion in residency selection. We investigate the impact of the pass/fail Step 1 scoring on wellness, career planning, and resource access from a medical student perspective with comparisons between minoritized and represented students.
Methods
Medical students at a single LCME accredited institution who had completed the USMLE Step 1 exam in 2022 or 2023 and received a pass/fail score were recruited for this study. Participants completed a 27-question anonymous electronic survey and were asked to self-report demographics, sense of wellness, career planning, and access to study resources during their dedicated study period. Statistical analyses with chi-squared tests were conducted using R.
Results
There were 128 eligible study participants; 55 identified as minoritized, including URiM (18.0%), LBGTQ+ (19.5%), or first-generation college student (25.0%). Overall, 66.4% of students felt that the change to pass/fail scoring improved student wellness in their class. However, 28.9% reported that it also affected their interest or confidence in pursuing certain specialties, and 75.8% reported feeling pressured to pursue more research and extracurricular activities. Many students (49.2%) reported “poor” or “terrible” wellness while preparing for the exam, including URiM (69.6% vs. 44.8%, p=0.054) and all minoritized students (58.2% vs. 42.5%, p=0.114) compared to their respective counterparts. Minoritized students (38.2% vs. 19.2%, p=0.029), especially URiM (52.2% vs. 21.9%, p=0.007), were more likely to report poor financial access to exam preparation resources compared to their respective counterparts. 
Conclusion
Although generalizability is limited by sample size at a single institution, a large proportion of medical students continue to experience poor wellness while preparing for the pass/fail Step 1 exam. Minoritized students, especially those URiM, report more financial barriers in accessing resources for Step 1 preparation. The scoring transition has heightened uncertainty regarding how to prepare for residency. It is imperative that medical schools expand career advising efforts and continually update recommendations to ensure adequate preparation for residency applications.

Premium content

Next from AMA Research Challenge 2023

Increasing PTSD Symptom Severity Predicts Increasing Healthcare Utilization Among OEF/OIF Veterans.

Similar lecture

UF-HOBI at "Discharge Me!": A Hybrid Solution for Discharge Summary Generation Through Prompt-based Tuning of GatorTronGPT Models

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES