United States

Background: ChatGPT is an artificial intelligence large language model that can comprehend and generate human-like text. Previous studies have evaluated the success of its earlier models on USMLE Step 1 questions, but have not examined differences in performance by subjects or disciplines. The current study utilizes the newer model of GPT-4 to assess performance by subject and disciplines and explore its implications for medical education.
Methods: Questions were obtained from the AMBOSS Step 1 question bank and inputted into ChatGPT to analyze accuracy. Questions from 18 systems and 12 disciplines were randomly selected and differences of accuracy between the various domains were assessed. ChatGPT accuracy was also compared to the accuracy of students towards the individual questions.
Results: A total of 1,300 questions were analyzed. ChatGPT-4 achieved an overall accuracy of 86% across all questions. There were no significant differences between the various systems or disciplines. ChatGPT accuracy increased as the student accuracy increased for each question (p&lt;.001).
Conclusion: ChatGPT-4 demonstrated high accuracy towards USMLE Step 1 questions, validating the proficiency of the software. The 86% accuracy observed is notably higher than percentages noted on previous studies assessing Step 1 questions that used earlier models of ChatGPT and exceeds the 60% passing threshold with a large margin. The lack of a difference in accuracy by systems and disciplines further demonstrates its reliability in being adopted as a supplemental learning platform for medical students. In addition to ChatGPT’s multi-step processing and ability to provide detailed and logical explanations to questions, we demonstrate an example of how it may be further utilized to enhance understanding of medical topics through applying prompts. In leveraging this technology, we stand on the brink of significant breakthroughs in our ability to enhance learning outcomes, especially in fields as critical and demanding as medicine.

AMA Research Challenge 2023

ChatGPT-4 performance on USMLE Step 1 questions and its implications for medical education: A comparative study across systems and disciplines

Background: ChatGPT is an artificial intelligence large language model that can comprehend and generate human-like text. Previous studies have evaluated the success of its earlier models on USMLE Step 1 questions, but have not examined differences in performance by subjects or disciplines. The current study utilizes the newer model of GPT-4 to assess performance by subject and disciplines and explore its implications for medical education.
Methods: Questions were obtained from the AMBOSS Step 1 question bank and inputted into ChatGPT to analyze accuracy. Questions from 18 systems and 12 disciplines were randomly selected and differences of accuracy between the various domains were assessed. ChatGPT accuracy was also compared to the accuracy of students towards the individual questions.
Results: A total of 1,300 questions were analyzed. ChatGPT-4 achieved an overall accuracy of 86% across all questions. There were no significant differences between the various systems or disciplines. ChatGPT accuracy increased as the student accuracy increased for each question (p<.001).
Conclusion: ChatGPT-4 demonstrated high accuracy towards USMLE Step 1 questions, validating the proficiency of the software. The 86% accuracy observed is notably higher than percentages noted on previous studies assessing Step 1 questions that used earlier models of ChatGPT and exceeds the 60% passing threshold with a large margin. The lack of a difference in accuracy by systems and disciplines further demonstrates its reliability in being adopted as a supplemental learning platform for medical students. In addition to ChatGPT’s multi-step processing and ability to provide detailed and logical explanations to questions, we demonstrate an example of how it may be further utilized to enhance understanding of medical topics through applying prompts. In leveraging this technology, we stand on the brink of significant breakthroughs in our ability to enhance learning outcomes, especially in fields as critical and demanding as medicine.

poster

Welcome to the **largest national, multi-specialty research** event featuring research from nearly 700 medical students, residents, fellows, and international medical graduates.

Use the buttons below or navigation menu to access the event.
 


[![](https://assets.underline.io/uploads/markdown_image/1/image/3e33a8c754cc49610b30950617d813e7.png)](https://underline.io/events/434/sessions?searchGroup=all)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/551404d429c4e274f9a54cbdd7081918.png)](https://underline.io/events/434/posters?searchGroup=all)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/677d92f566b880cebd36905fc738fc10.png)](https://underline.io/events/434/lounge)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/c7392833c7261c3874c5b43fa2f4df19.png)](https://underline.io/events/434/help-desk)
 

#### Event Details

The 2023 American Medical Association Research Challenge includes a poster symposium hall and a semifinals hall with the top scored research. **Log in at any time throughout the event to:**

* **View posters and presentations** on a variety of topics and specialties
* **Score posters in the semifinals** to help decide the top five finalists (AMA member exclusive opportunity)
* **Connect with participants** and provide feedback to help advance their research
* **Attend education sessions** with physician experts on key research topics

At the conclusion of this event, the AMA will announce the top five finalists who will advance to the AMA Research Challenge finals, present their research to an elite panel of judges, and compete to win a $10,000 grand prize sponsored by Laurel Road. 

### Grand Prize Sponsor

AMA would like to thank Laurel Road for sponsoring a $10,000 grand prize for the winner of the AMA Research Challenge. Laurel Road is not involved in the judging or scoring process to select the winner. The winner will be announced on February 4, 2024 during the AMA Research Challenge finals. Payment of the grand prize will be directly from Laurel Road. See the AMA Research Challenge Official Rules for terms and conditions.

![](https://assets.underline.io/uploads/markdown_image/1/image/611e331d4a0eba0920f240cfc04b4d77.png)

**There is no registration fee for this event! **

Register to explore research in a variety of topics. AMA members get exclusive access to score posters in the semifinals.

**Please use this link [here](https://www.ama-assn.org/about/events/ama-research-challenge) to register for free.**

Registration Is Required

Join the AMA Research Challenge poster symposium and semifinals to explore top research posters, score posters in the semifinals (an AMA member exclusive opportunity) and attend education sessions with experienced physicians to learn tips to advance your research and career. 

Background: The COVID-19 pandemic exposed the gap in the effective communication of health information between the medical community and general public. For example, vaccine hesitancy demonstrated the need to engage patients in challenging conversations, explain complex scientific research and advancements, and appropriately acknowledge areas of medical uncertainty. To meet this need, medical educators play an important role in the training of rising physicians. Specifically, we can employ innovative training models such as medical improv which utilizes improvisational theatre principles to enhance trainees’ communication skills. To this end, we examined the use of medical improv as a tool for teaching clinical communication skills to UC Davis medical students ahead of their first clinical clerkships.
Methods: We piloted a four-part medical improv workshop for second-year medical students in January of 2022 to teach core improv communication skills such as active listening, quick thinking, and collaboration. After initially assessing the workshop impact through pre-and-post workshop surveys, we conducted one year of follow up by distributing digital surveys after 4 of the 7 clinical clerkships. Surveys were comprised of Likert scale and open-response questions to assess how, if at all, medical students employed improv communication skills during clerkships. Data was analyzed using mixed methods. Of note, enrollment was limited due to the spread of the Omicron variant in Winter of 2022.
Results: Of the 10 enrolled students, 8 (80%) completed the workshop, and 3-7 respondents submitted each follow-up survey. Participants implemented improv skills an average of 3 times during a day of clinic activity. Thematic analysis of the narrative responses revealed that students most commonly implemented their improv skills when 1) actively listening to patients, 2) answering questions posed by patient’s families, and 3) having difficult conversations. Students described how active listening was critical to establishing connections with patients and understanding their perspectives. One student remarked, “actively listening to the patient was much more important than trying to pre-form a response to what I imagine they are going to say.”
Conclusion: After one-year of follow up, this pilot study shows that medical improv can meaningfully impact students’ clinical communication skills by teaching the difference between responding and reacting in conversations with patients and their families. Through honing active listening skills, we can better equip future physicians to minimize the communication gap between them and their patients.

Responding vs. Reacting: How medical improv can enhance clinical communication skills in rising physicians

Background
Adoption has life-long health implications for nearly 8 million adopted people and their families in the United States. Most U.S. adoptees have limited family medical history (LFMH) and adoptees are also 3.7 times more likely to attempt suicide than their non-adopted siblings. Genetic testing kits promising health information to consumers have become increasingly available in recent years, amplifying the need to address genetic testing for adult adopted patients with LFMH. Numerous patient narratives by adult adoptees with LFMH illustrate encounters with clinicians that trigger medical anxiety, shame, guilt, and distrust. However, adult adoptees with LFMH are nearly absent from medical literature and training and little work has been done examining the perspectives, approach, and training of PCPs caring for this population.

Methods
This exploratory, qualitative study is the first to explore primary care physicians’ (PCPs) training and approaches to adult adoptees with LFMH. PCPs in Minnesota and Rhode Island specializing in family medicine, medicine, and medicine-pediatrics were recruited via convenience and reputational sampling. Twenty-three semi-structured hour-long interviews were conducted utilizing hypothetical clinical vignettes. Interview transcripts were coded according to the template organizing method, and then interpreted via immersion crystallization.

Results
Findings from this study suggest that PCPs typically receive little education, training, or resources throughout medical school, residency, and continuing medical education about adopted patients with LFMH. PCPs reported limited knowledge and discomfort in managing adoption issues and mental health concerns of adopted patients. Although previous research has observed an increased likelihood of attempted suicide among adopted people, half of PCPs were unaware of local mental health professionals or community organizations that specialize in adoption. Most PCPs expressed uncertainty around best practices for adult adoptees with LFMH and genetic testing, especially recreational direct-to-consumer kits. PCPs varied widely on whether they believed genetic testing beneficial or harmful for adoptees with LFMH. Lastly, many PCPs reported obtaining family medical history in ways that may cause miscommunication and risk being interpreted as devaluing non-biological families. Nearly all PCPs endorsed additional training about adult adoptees with LFMH as beneficial, particularly for mental health issues and genetic testing.

Conclusion
Given that these issues remain largely unaddressed by medical literature, curricula, and professional physician societies, future research should expand education and training opportunities for medical professionals to improve health care for adult adoptees with limited family medical history.

Exploring Primary Care Physicians’ Approach to Adult Adopted Patients with Limited Family Medical History: A Qualitative Study

Background: The financial climate for medical students has grown precarious, with increasing educational debt and unfavorable loan terms affecting students’ career decisions and overall well-being. This situation is exacerbated by a critical lack of financial literacy, which has been shown to vary among graduate students from different backgrounds. Thus, many students enter the professional sphere ill-equipped to manage their finances. High debt levels may also sway residency specialty choices, decrease interest in primary care specialties, and contribute to burnout, adversely impacting professional performance and personal happiness. Conversely, financial literacy significantly improves personal well-being and career satisfaction, so there is an urgent need to incorporate financial training into early medical education. The aim of this study is to identify gaps in students’ financial literacy and their associations with demographic findings.

Hypothesis: We hypothesize that demographic factors will display a strong correlation with self-assessed financial literacy but not with objective literacy. We also believe that students will both select a specialty and a location based on their financial circumstances. 

Methods: An anonymous web-based questionnaire was designed using Qualtrics and sent to medical students of all four years at one medical school. It explored various topics to evaluate demographic data and students' financial behaviors, knowledge, and perceptions. Objective financial literacy was measured using the “Big 3”, a validated financial literacy questionnaire.

Results: A total of 218 students responded to the questionnaire. No significant differences in financial literacy and habits were found across different years of medical school (p>0.05). Notably, immigrants both scored lower in financial literacy (p<0.01) and considered themselves to be less financially literate than their non-immigrant counterparts (p<0.01). However, students from underrepresented backgrounds had no significant difference in financial literacy, but considered themselves to be less financially literate, than their counterparts (p<0.01). Additionally, higher parental education mildly correlated with improved financial literacy (p<0.05). Students' finances significantly impacted their desired future practice location (p<0.01) but not specialty choice (p>0.05). Lastly, there were no significant associations among student debt, debt-drive behaviors, and demographic data (p>0.05).

Conclusion: The study demonstrates distinct disparities in financial literacy and student debt perspectives among medical students, particularly influenced by factors such as immigration and underrepresented status. There were also interesting correlations between parental education levels and financial literacy. However, many expected associations, like that between immigrant status, or underrepresented status and student debt, were insignificant, challenging some preconceptions. Notably, student finances significantly impacted the location choice for practice, independent of immigration or underrepresentation status, further emphasizing the multi-faceted nature of these economic and demographic factors in medical students' lives.

Financial Literacy Among Medical Students:
Debt, Demographics, and Career

**Scoring is only possible in the semifinals hall**

**Abstract:**



Medical School’s Hidden Curriculum: The Financial Literacy Gap Among Future Physicians

Medical students have reported higher rates of stress and depression compared to their age-matched peers, which worsen while studying in isolation for high stakes examinations such as the United States Medical Licensing Examination (USMLE) Step 1. Medical schools should therefore implement and evaluate additional support offerings to improve the student experience during this time. We implemented a support framework, titled Step-Up, to directly address isolation for students studying for Step 1 in the winter of 2021-2022. This included multiple group, individual, and email interventions. Students were asked to self-report wellness and connectivity before and during Step 1 studying, levels of satisfaction and usefulness of the resources offered. As expected, self-reports of wellness and connectivity worsened during Step 1 study. Many interventions had low uptake. Students reported highest satisfaction with and usefulness of weekly emails and near-peer tutors. Our data show that near-peer tutoring and regular support emails are successful interventions to help mitigate the worsening wellness and increased isolation medical students experience during Step 1 studying. It also examines multiple other interventions that educators may wish to study further, or not include due to lack of uptake or benefit.

Step-Up to Address Step 1 Anxiety and Improve Wellness

Background
Research is a cornerstone of medical education, and demonstration of aptitude in this area is necessary to fulfill the general competencies identified by the ACGME. As such, research opportunities for medical students should be available, accessible, and fulfilling. However, a variety of reasons exist that hinder the ability of medical students to achieve their desired and essential experiences in clinical and/or laboratory research. These impediments can lead to student dissatisfaction and less competitive residency applications. 

Methods
We conducted a cross-sectional survey of University of Massachusetts (UMass) Chan Medical School students to better understand the current state of student research. Current and past research participation was collected, including the number of projects and weekly commitment. A Likert scale was used to query students’ perceptions regarding importance of participating in research, ease of accessing and applying to research opportunities, and barriers to research opportunities. Student satisfaction with access to research opportunities was ranked from 1-10 (least-most satisfied). This survey was approved by the UMass Chan Medical School IRB (STUDY00000459). 

Results
Overall, 108 students (16.7% of all UMass medical students) completed the survey, of which 63 (58.3%) were pre-clinical students. Most students (88.9%) responded that scholarly activity was ‘slightly important’ or ‘very important’, but only 61.1% of students reported involvement in a scholarly project. 51% of students responded that research opportunities are ‘somewhat’ or ‘very difficult’ to access. Mean satisfaction with access to research was 4.4 +/- 2.6. The three most common selected barriers to research activity were difficulty finding research project listings (64.8%), not having enough time available (45.4%), and trouble contacting investigators (42.6%). 

Conclusion
Students recognize the importance of scholarly activity during medical education in molding their professional and academic identity, and in improving their residency competitiveness. However, gaining access to opportunities can be difficult. Despite UMass Chan Medical School being amongst the top 25% of institutions in terms of research output per student, we identified a significant opportunity to improve student satisfaction in medical education through research. While it would be difficult to address some barriers, such as available time for medical students, improving accessibility of opportunities and communication represent targets for countermeasures. We recently secured funding to establish resident liaisons within clinical departments and a Drupal server to create a research listing database called 'UMatch'.

Facilitating Medical Students in Pursuit of Scholarly Activity: A Quality Initiative Using A3 Problem Solving to Improve Student Resources

Stop the Bleed Training’s Effects on Preclinical Medical Students’ Confidence and Willingness to Intervene in a Hemorrhage Situation

Background- Drawing is a form of active learning that enhances knowledge and increases retention. In human anatomy education, drawing allows for the artist to use their creations to form clinical correlations and expand their clinical knowledge. Previous studies demonstrate that incorporating art in anatomy sessions improves visual learning, engagement, and complex content understanding. Multiple exposures to art led to further comprehension of the material, demonstrated by the comparison of pre and post session assessments. However, to this day, cadaver-based learning is well known as the gold standard in anatomy education. This study intended to determine whether instructional based drawing increases knowledge and long-term retention of anatomy material, compared to a standard cadaveric-based anatomy lab session.
Methods- Participants were given an initial demographic assessment. Four sessions were held to simulate medical anatomy laboratory coursework. Topics included arterial flow, venous drainage, the nervous system, and skeletal myology. Participants were second-year medical students. All participants first attended an instructor-led station in which pre-selected structures were identified on a prosected cadaver. The control group then participated in an exercise in which certain structures in the cadavers were pinned and identified, and the study group participated in an instructor-led drawing exercise. At each session, participants completed a pre-assessment (ten questions) and immediate post-session assessment (five questions). Six-weeks later, participants completed a second post session assessment (five questions) to assess long term retention. These assessments were curated from standardized and validated questions.
Results- There were 15 participants, with most participating in more than one session. Data was pooled across the four sessions with a total of n=34 including repeated participation. There were no statistically significant differences between the control and study group for the pre-, immediate post-, and 6-week post-assessments. The control and study groups both significantly improved from the pre-assessment to immediate post-assessment (p<0.001, p<0.001) and to the six-week post-assessment (p<0.001, p=0.003).
Conclusion- Utilizing art as a cadaver-based study strategy was found to be as effective as the gold standard cadaver-based teaching. Incorporating art into medical anatomy curricula can further enhance understanding, knowledge, and retention.

Do art and anatomy be-lung together? Using drawing to increase anatomical 
knowledge and retention for medical students.

Background: The objective of this research is to identify current best practices for United States medical schools looking to implement evidence-based Cultural Competence Curricula (CCC). The Liaison Committee on Medical Education requires medical students to demonstrate cultural competence upon graduation to ensure future physicians are prepared to provide quality care to diverse groups. CCC is commonly defined in a medical context as the knowledge, skills, and attitudes conducive for cross-cultural communication. However, there’s an abundance of methods for planning, implementation, and assessment of CCC that make determining a best model difficult. 

Methods: We conducted a scoping literature review to determine the current state of CCC in U.S. medical schools. We collected data on the details of the curricula described in 77 papers, which was used to find patterns related to curriculum structure, evaluation, and study design.

Results: Our results illustrate the prevalence of structure styles conducive for high-order learning, including longitudinality, integration, incorporation into clinical training, and experiential learning. The most common method for evaluating student learning was student self-evaluation with few programs performing reevaluations or utilizing patients as evaluators. Of knowledge, attitudes and skills, skills were least evaluated.

Conclusion: Collected data reveals gaps in CCC regarding evaluation and execution while also revealing learning models that suggest positive efficacy. Characteristics of schools with positive evaluations include maximized quantity of structural components, higher quality of evaluation, and connection with the local community. It seems that longitudinal, non-passive learning in integrated courses evaluated via experimental and quasi-experimental study designs produce more reports of success. Overall, in an increasingly cross-cultural world, it is imperative that we support educators in ensuring that students become effective and benevolent healthcare providers for diverse groups.

What Works? A Scoping Review of Implementation and Evaluation Frameworks for Cultural Competence in U.S. Medical Education

Background
Research is recommended in medical training and residency, as per the Accreditation Council for Graduate Medical Education (ACGME). This not only encourages academic interests among residents but also helps them adopt evidence-based approaches while treating patients. Early research training has been linked to continued scholarly work and informed career choices. However, many medical students and residents focus more on non-hypothesis-driven research, likely due to inadequate technical support and low interest. We have developed an OpenAI GPT3-based web application called AutoAnalyze to address this issue. 
Methods
The program was created using Visual Studio Code, and the code was written with Python, along with Streamlit and a variety of visualization packages. For ad hoc visualization, additional programs are invoked de novo by user input. Application programming interfaces (APIs) for OpenAI ChatGPT3.5 were integrated for activation depending on specific user feature choices. Custom prompts and APIs enabled highly focused functionality, with coding details available on 
GitHub repository under the MIT licence: https://github.com/DrDavidL/auto_analyze.Streamlit.io hosts the application on a public site: https://data-auto-analysis.streamlit.app/.
Results
AutoAnalyze simplifies the technical aspects of hypothesis-driven research, making it easy to conduct research using pre-coded options or via natural language inputs. With AutoAnalyze, users can perform data exploration, missing data imputation, and data visualization. Additionally, it generates publication-worthy images like bar charts, pie charts, scatterplots, and principal component analysis with just a few clicks. AutoAnalyze also provides teaching pearls on visualization techniques and summarizes relevant statistics in table format. AutoAnalyze can perform machine learning analytic methods like logistic regression, decision trees, random 
forests, gradient boosting machines (GBMs), and support vector machines (SVM) to perform predictive analysis in an easy-to-comprehend user-friendly format. It does this without requiring the user to type in any code or understand a computer programming language. Furthermore, the chatbot teacher feature enables users to probe data or learn data science using regular English words and sentences.
Conclusion
Overall, AutoAnalyze represents an efficient solution to address the inadequate technical support for medical research challenges medical students and residents face. It has the potential to increase their research interest in academic medicine.


Premium content

ChatGPT-4 performance on USMLE Step 1 questions and its implications for medical education: A comparative study across systems and disciplines

Next from AMA Research Challenge 2023

Responding vs. Reacting: How medical improv can enhance clinical communication skills in rising physicians

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES