A crucial challenge for generative large language models (LLMs) is diversity: when a user's prompt is under-specified, models may follow implicit assumptions while generating a response, which may result in homogenization of the responses, as well as certain demographic groups being under-represented or even erased from the generated responses. In this paper, we formalize the problem diversity of representation in LLM generations. We present evaluation datasets and propose metrics to measure diversity in generated responses along people and culture axes. We find that LLMs understand the notion of diversity, and that they can reason and critique their own responses for that goal. This finding motivated a new prompting technique called collective-critique and self-voting (CCSV) to self-improve people diversity of LLMs by tapping into its diversity reasoning capabilities, without relying on handcrafted examples or prompt tuning. Extensive empirical experiments with both human and automated evaluations show that our proposed approach is effective at improving people and culture diversity, and outperforms all baseline methods by a large margin.

Improving Diversity of Demographic Representation in Large Language Models via Collective-Critiques and Self-Voting

Most of the existing compositional generalization datasets are synthetically-generated, resulting in a lack of natural language variation. While there have been recent attempts to introduce non-synthetic datasets for compositional generalization, they suffer from either limited data scale or a lack of diversity in the forms of combinations. To better investigate compositional generalization with more linguistic phenomena and compositional diversity, we propose the DIsh NamE Recognition (DiNeR) task and create a large realistic Chinese dataset. Given a recipe instruction, models are required to recognize the dish name composed of diverse combinations of food, actions, and flavors. Our dataset consists of 3,811 dishes and 228,114 recipes, and involves plenty of linguistic phenomena such as anaphora, omission and ambiguity. We provide two strong baselines based on T5 and large language models (LLMs). This work contributes a challenging task, baseline methods to tackle the task, and insights into compositional generalization in the context of dish name recognition.

DiNeR: A Large Realistic Dataset for Evaluating Compositional Generalization

Rich and diverse knowledge bases (KB) are foundational building blocks for online knowledge sharing communities such as StackOverflow and Quora, and applications such as conversational assistants (aka chatbots). A popular format for knowledge bases is question-answer pairs (or FAQs), where questions are designed to accurately match a multitude of queries. In this paper, we address the problem of automatic creation of such Q\&A-based knowledge bases from domain-specific, long-form textual content (e.g., web articles). Specifically, we consider the problem of question generation, which is the task of generating questions given a paragraph of text as input, with a goal to achieve both diversity and fidelity of the generated questions. Towards this goal we propose PROTEGE, a diverse question generation framework which consists of (1) a novel encoder-decoder based Large Language Model (LLM) architecture which can take a variety of prompts and generate a diverse set of candidate questions, and (2) a hill-climbing algorithm that maximizes a sub-modular objective function to balance diversity with fidelity. Through our experiments on three popular public Q\&A datasets, we demonstrate that PROTEGE improves diversity by +16% and fidelity by +8% over diverse beam search and prompt-based baselines.

PROTEGE: Prompt-based Diverse Question Generation from Web Articles

Given a textual passage and an answer, humans are able to ask questions with various expressions, but this ability is still challenging for most question generation (QG) systems. Existing solutions mainly focus on the internal knowledge within the given passage or the semantic word space for diverse content planning. These methods, however, have not considered the potential of external knowledge for expression diversity. To bridge this gap, we propose RAST, a framework for Retrieval-Augmented Style Transfer, where the objective is to utilize the style of diverse templates for question generation. For training RAST, we develop a novel Reinforcement Learning (RL) based approach that maximizes a weighted combination of diversity reward and consistency reward. Here, the consistency reward is computed by a Question-Answering (QA) model, whereas the diversity reward measures how much the final output mimics the retrieved template. Experimental results show that our method outperforms previous diversity-driven baselines on diversity while being comparable in terms of consistency scores. Our code is available at \url{https://github.com/gouqi666/RAST}.

Diversify Question Generation with Retrieval-Augmented Style Transfer | VIDEO

Background
Pack-year history is an invaluable clinical tool for measuring tobacco-associated disease risk and determining eligibility for lung cancer screening with low-dose computed tomography. Patient-reported measures of smoking history have demonstrated strong test-retest reliability across various survey methods, but the variation between research and clinical settings suggests that non-standardized smoking history interview questions in community practice may not always reveal the true extent of a patient’s tobacco exposure. To date, no study has compared patient responses to differently phrased questions assessing the same measure of smoking history. We hypothesize there is a significant discordance between patient-reported number of years smoked and calculated elapsed time between age at initiation and age at cessation (or present age if current smoker).

Methods
This was a two-month prospective survey-based study conducted at the Western Michigan University Homer Stryker M.D. School of Medicine's Internal Medicine resident clinic in Kalamazoo, Michigan, including patients with a documented smoking history in their electronic medical records. Through verbal interviews, we collected data on number of years smoked, packs per day, and age of smoking initiation/cessation. Primary outcome variables were total years smoked and calculated pack-years. Secondary outcome variables were gender, age, and age of smoking initiation.

Results
There were 124 respondents (36% male, 63% female, 1% non-binary) with an average age of 55 and average smoking history of 1.2 packs per day. Of them, 46% were current and 54% were former smokers. Using a paired sample one-tail t-test, we compared patient-reported years smoked (M=22.06, SD=12.44) to calculated years based on age of initiation/cessation (M=24.03, SD=13.53), which showed a significant mean difference of 1.97 years (95% CI 1.16- 2.78, t=4.05, p<0.001). This discordance translated to a 21.29% increase in pack-years (from 24.32 to 29.50 pack-years) and a mean difference of 5.21 pack-years (95% CI 1.04-9.38, t=2.07, p=0.02).

Conclusion
Our study underscores the importance of critically evaluating the ways in which we gather a patient’s smoking history. When compared to pack-years calculated using age of smoking initiation/cessation, our sample population tended to underestimate their exposure by over 5 pack-years—a clinically significant difference for someone at high risk who might not otherwise receive screening imaging. Often a pivotal moment in adolescence, participants recalled their age of smoking initiation with greater precision compared to their estimations of total exposure. These results call for urgent reevaluation of the tobacco history interview, its notation in electronic medical records, and the methodology of taking a substance abuse history in general.

Rethinking the Smoking History Interview: A Prospective Survey-based Study

Background:
Since the release of the COVID-19 vaccines, vaccine safety and hesitancy have been significant topics of contention. While studies have described COVID-19 vaccine receptiveness in the general population, its uptake in inflammatory bowel disease (IBD) patients is not well-characterized. This study aimed to investigate COVID-19 immunization rates and barriers to immunization in IBD patients.

Methods:
A retrospective study investigated 545 adult patients with Crohn’s disease (CD) or ulcerative colitis (UC) seen at an IBD subspecialty clinic at a university-based tertiary care center between January 1, 2020 and October 31, 2021. For each patient, data collected includes: demographics, past medical history, use of immunosuppressive medications, and COVID-19 and influenza immunization status. COVID-19 immunization rates in the general U.S. population were obtained from data published by the Center for Disease Control and Prevention (CDC).

Results:
As of October 31, 2021, 87.7% of IBD patients had received at least one dose of the COVID-19 vaccine, 84.4% had received the full vaccine series, and 21.8% had received the booster and/or additional dose. COVID-19 immunization rates in IBD patients were significantly higher than in the general population (p < 0.001). No significant difference was found in COVID-19 immunization rates when stratified by IBD type (CD vs UC), sex, or concurrent immunosuppressive therapy. Patients who received the COVID-19 booster/additional dose were older (mean age = 54.0 ± 17.9) than those who did not (mean age = 50.0 ± 16.5; p = 0.02). Asthmatic patients had a significantly lower rate of receiving one dose (79.5% vs 89.2%, p = 0.02) or the full series (74.7% vs 86.1%, p = 0.01) of the COVID-19 vaccine compared to non-asthmatics. Tobacco users had a significantly lower rate of receiving the full COVID-19 vaccine series (73.9% vs 85.9%, p = 0.02). Patients previously immunized for influenza had a significantly higher rate of receiving the COVID-19 booster/additional dose than those who had not (31.5% vs 17.6%, p < 0.001). History of hypertension, hyperlipidemia, heart failure, diabetes, malignancy, ischemic heart disease, pulmonary circulation disorder, or COPD did not significantly affect COVID-19 immunization rates.

Conclusion:
Our study reveals that IBD patients are more receptive to the COVID-19 vaccine compared to the general U.S. population. Immunosuppressive therapy did not affect IBD patients’ uptake of the COVID-19 vaccine, although younger age, asthma, and tobacco use were associated with decreased uptake. Particular attention is required when counseling IBD patients to identify barriers to vaccination and increase COVID-19 immunization rates.

Uptake of COVID-19 Immunization in the Inflammatory Bowel Disease Population

Introduction: 
Current AAOS guidelines recommend that children younger than thirty-six months with a femur fracture be evaluated for non-accidental trauma. However, the likelihood of abuse in walking-age children presenting with these fractures has been found to be less than 10%. These guidelines could subject up to 63% of cases to unnecessary evaluation by Child Protective Services. The purpose of this study was to identify how patient age and femur fracture morphology impact the likelihood that providers assess a fracture as non-accidental and pursue further investigation of possible child abuse. 

Methods:
This study evaluated a questionnaire completed by 75 clinicians from multiple specialties. There were 5 subgroup populations (6, 12, 18, 24, 35 months old) combined with diaphyseal and metaphyseal femur fracture types. For each age group, the same radiograph was presented and described as an oblique diaphyseal fracture in one scenario and spiral diaphyseal fracture in another. Similarly, some cases were described as a corner fracture in one scenario and classic metaphyseal lesion in another. This design assessed if fracture nomenclature, in addition to patient age, impacted clinician suspicion of abuse. Participants assessed case presentations and reported their perceived likelihood the case represented non-accidental trauma utilizing a Likert scale from 1 to 5 corresponding to a percent likelihood. They also reported the percent likelihood they would pursue further investigation. 

Results:
75 questionnaires were analyzed. Mean likelihood of abuse rating in all fracture types for patients 18-35 months was 3.8, corresponding to 61-80% perceived likelihood of abuse. Mean likelihood for further investigation of child abuse in patients 18-35 months of age was 3.7, corresponding to 61-80% clinician likelihood to further investigate. Fracture nomenclature did not significantly impact clinician ratings. In the linear mixed model, the strongest predictors of perceived high likelihood of abuse were respondent specialty, with emergency medicine (p<0.001) and advanced practice providers (p=0.006) having the highest likelihood ratings. 

Conclusions: 
Clinicians are trained to have a high index of suspicion for child abuse in pediatric patients. However, this study demonstrates a high assessment of chance for abuse and low threshold for pursuing further investigation for femoral shaft fractures in walking-age children which contradicts literature that reports actual rates of non-accidental trauma to be lower than 10%. This bias may exhaust hospital and community resources in evaluating toddler femur fractures for abuse and cause unnecessary stress for patients and families.

Common Misperceptions of Child Abuse in Toddler Femur Fractures

Background
Links between cancer and microbes date back four millennia (Sepich-Poore et al. 2021. Science). Recently, we showed that microbial DNA is detectable in tumor tissues and patient blood from dozens of human cancer types (Poore et al. 2020. Nature; Narunsky-Haziza et al. 2022. Cell), suggesting that cell-free microbial DNA (cf-mbDNA) could augment human-centric liquid biopsies of cancer, which currently lack sensitivity at early-stage disease. Thus, we set out to build and evaluate the first multi-omic, multi-species (human+microbe) blood tests for early-stage lung cancer detection with two intended use populations: screening-eligible individuals without known disease (i.e., a “rule-in” test with high specificity), and lung nodule-bearing individuals (i.e., a “rule-out” test with high sensitivity).

Methods
We created a cancer-specific, microbial reference database using ¬de novo metagenome assembly on non-human DNA from 5187 whole-genome sequenced blood and tumor tissue samples. This microbial database was bioinformatically evaluated across 18,782 samples and ten international cohorts, with predictions made using machine learning. Diagnostic performance was measured with and without the addition of human analytes to test for synergy. Source tracking was performed among paired lung tumors, pre-operative plasma, and post-operative plasma to identify what proportion of cf-mbDNA was tumor derived. After tuning the tests using discovery cohort samples, we applied them to validation cohorts comprising (i) screening-eligible individuals without known disease and (ii) lung nodule-bearing individuals.

Results
Microbial DNA discriminated blood samples from 24 cancer types simultaneously with an average pairwise area under the receiver operating characteristic (AUROC) curve of 95.5%. A median of 81.4% of cf-mbDNA was directly attributable to intratumoral microbial DNA via source tracking, indicating its predominant tumor origin. The addition of human cell-free tumor DNA (ctDNA) fragments and two plasma-derived proteins (CEA, osteopontin (OPN)) to cf-mbDNA significantly and synergistically improved diagnostic performance (Delong’s test: p<0.001). Application of the joint host+microbial test to screening-eligible, stage I lung cancers versus healthy smokers showed a sensitivity of 68.2% at 96.2% specificity. Then, collectively across two independent validation cohorts comprising lung nodule-positive individuals, the test significantly outperformed PET-CT and clinical risk scores, with an overall specificity of 49.5% at 95.7% sensitivity.

Conclusion
A multi-omic, multi-species test diagnoses early-stage lung cancer better than existing methods. In January 2023, the FDA provided Breakthrough Designation status for this approach to determine malignancy likelihood of lung nodules for patients who refuse or lack access to PET-CT testing.

Early-stage detection of lung cancer using a multi-omic, multi-species test

Background: Life adversities, such as neighborhood disadvantage and lower household incomes, are associated with well-established impairments on language development, which predict academic difficulties throughout school. These language delays are observable by age 2, but it remains unclear if adversities experienced in utero affect neurological underpinnings of language development before delays emerge. Of interest are 3 white matter tracts implicated in early language skills: the corpus callosum (CC), inferior fronto-occipital fasciculus (IFOF), and uncinate (Unc).

Methods: A longitudinal cohort of 160 term-born (>36 weeks gestation) neonates were recruited from the eLABE study at Washington University, which oversampled St. Louis families exposed to poverty. Structural equation modeling was used to create 2 latent constructs of prenatal adversity. The first, “social disadvantage”, aggregates neighborhood disadvantage, family income, insurance status, maternal education, and diet. The second, “maternal psychosocial stress”, models maternal depression, stress, life adversities, and perceived discrimination. At birth, diffusion MRI data were collected, and parameters for white matter tract development were created using probabilistic tractography in FSL. In-person follow-up at age 2 evaluated child language skills. To assess the independent and interactive effects of prenatal adversities and neonatal white matter microstructure on language skills, stepwise linear regression models were conducted after adjustment for child sex, gestational age, and postmenstrual age at scan. 


Results: Prenatal social disadvantage (p=.00) and maternal psychosocial stress (p=.00) predicted significantly lower language scores by age 2. However, in a joint model, only the effects due to social disadvantage remained significant. Lower CC fractional anisotropy predicted significantly lower language scores (p=.02) independently of social disadvantage (p=.00). IFOF and Unc were not predictive of language ( p > .05). To examine the relationship between CC maturation and social disadvantage, a moderation analysis was conducted after dichotomizing families’ income into above and below 200% the federal poverty line (FPL), as most federal assistance programs use this value as a threshold for benefit eligibility. The interaction term was not significant (p= .11), but upon plotting results by income group, the slope for infants above 200% FPL was somewhat stronger. 

Conclusion: This study suggests that prenatal adversity, particularly social disadvantage, and immature CC myelination are strong predictors of language delays by age 2. These data can be used as empirical evidence to advocate for increased funding of federal assistance programs that benefit pregnant people (ex., WIC) and downstream fetal brain development. By continuing to explore the relationship between prenatal adversities on fetal brain development and emerging language skills, future studies will continue to identify specific neurological markers of early language development to allow clinicians to intervene before delays emerge.

The Effects of Prenatal Adversity and Neonatal White Matter Microstructure on Child Language Skills at Age 2

Background: Clinical Practice Guidelines (CPGs) aim to offer guidance to physcians to improve patient outcomes. The Reporting Items for Practice Guidelines in Healthcare (RIGHT) Working Group created a set of rules for CPGs to aid guideline presentation and communication. CPGs in Plastic and Reconstructive surgery have not yet been evaluated by CPG metrics, such as RIGHT. This investigation aims to assess and recognize areas for improvement in currently available guidelines. 
Methods: Three reviewers extracted and reviewed all 5 guidelines listed on the American Society of Plastic Surgeons (ASPS) website in March 2023 and scored them according to the RIGHT checklist. RIGHT criteria in the form of 22 distinct statements were broken down into 35 items, and guidelines were assigned a “1” if they met the criteria and a “0” otherwise. Interrater agreeability through Fleiss’s kappa scores calculated for each guideline. 
Results: Among the 5 PRS guidelines checked against the RIGHT criteria, Fleiss’s kappa interrater agreeability ranged from 0.762 to 0.962, indicating strong agreement among independent raters. However, 4 (22.9%) RIGHT items were not met by any ASPS guidelines. There was a mean adherence of 21 RIGHT items. Guidelines were most lacking in categories concerning reporting of quality and review processes. 
Conclusion: Overall, many RIGHT framework recommendations were met fully by ASPS guidelines, but there is room for improvement in disclosure of the guideline development process. By evaluating fidelity to the RIGHT checklist, specific and actionable opportunities for improvement to guideline language and reporting can be implemented after identification.

Evaluation of Plastic Surgery Clinical Practice Guidelines Using the RIGHT Checklist

Introduction

Adverse outcomes and complications relating to SARS-CoV-2 (COVID-19) infection may be influenced by a number of factors, including pre-existing comorbidities and past medical history. Despite traumatic brain injury (TBI) being a public health crisis worldwide, the relationship between the prevalence of a recent TBI and adverse outcomes following COVID-19 infection remains poorly understood. This study aims to identify whether patients with a recent TBI face increased risk of mortality and adverse outcomes associated with COVID-19, as compared to patients without a recent TBI.

Methods

We queried the TriNetX United States Collaborative Network and identified patients with a recent TBI within 1 year of a hospital encounter for COVID-19. A propensity-match analysis was performed using baseline demographics and past medical history to compare patients with a recent TBI and COVID-19 to a cohort of patients with COVID-19 without recent TBI. We collected outcome measures within 1 month of COVID-19 diagnosis relating to overall mortality, intensive care unit (ICU) admission, and in-hospital complications. A Kaplan-Meier survival (KMS) analysis was performed to determine overall survival differences.

Results
A total of 13,970 patients were identified in each cohort. The mean age was 59 years old, and over 51% of patients were male. Compared to patients without a recent TBI, patients with a recent TBI had increased risk of hospitalization (OR 3.7 [95%-CI 3.5-4.0]; p<0.001), intubation (OR 2.4 [95%-CI 1.9-3.0]; p<0.001), ICU admission (OR 3.7 [95%-CI 3.3-4.0]; p<0.001), respiratory failure (OR 1.5 [95%-CI 1.4-1.6]; p<0.001), hepatic injury (OR 1.7 [95%-CI 1.3-2.3]; p=0.001), and overall mortality (OR 2.5 [95%-CI 2.2-2.8]; p<0.001) within 1 month of COVID-19 diagnosis. Recent TBI patients had reduced risk of cardiac injury (OR 0.8 [95%-CI 0.7-0.9]; p<0.001). KMS analysis demonstrates that patients with prior TBI had increased mortality within 1 month of COVID-19 diagnosis (HR 2.5 [95%-CI 2.2-2.9]; p<0.001).

Conclusion 

Patients with recent TBI up to 12 months prior to a separate hospital encounter for COVID-19 diagnosis have increased risk of hospitalization, intubation, ICU admission, respiratory failure, hepatic injury, and overall mortality within 1 month of COVID-19 diagnosis. We hope that these results improve medical decision-making and management of patients with a recent TBI and COVID-19. 


The Impact of a Traumatic Brain Injury Within 1 Year of COVID-19 Infection: A Propensity-Match Analysis

Glioblastoma Multiforme (GBM) is one of the most common and aggressive high-grade primary brain tumors that arises from the glial cells of the central nervous system and occurs most often in the frontal and temporal lobes. Despite current treatment options, long-term prognosis of GBM remains low with no significant improvement rates. Chimeric antigen receptor (CAR)-expressing T cells target the B cell marker CD19 and have shown increased efficacy in B cell lymphomas, and lymphoblastic leukemias suggesting benefits in GBM. This study aims to quantify the persistence of CAR-T cells in the context of GBM tumor volume following neurosurgical debulking procedure. A radiology software known as the mint Lesion was utilized to provide Tumor Response Assessment by Criteria (TRAC) reports that consist of volumetric measurements of the tumor lesions (target enhancing lesions, non-target enhancing lesions, and non-target non-enhancing lesions). Fluorescence-activated cell sorting (FACS) analysis takes a sample mixture of cells and sorts them into different populations based on specific biomarkers and characteristics. B7-H3 CAR-T cell persistence was increased in patients with a larger post-surgical tumor volume (3.04% Subject 001 with 2438 mm² volume and 2.38% Subject 003 with 224 mm² tumor volume). However, the number of CAR-T cells per 1 mm² was significantly reduced in patients with a larger post-surgical tumor volume (0.0012 cells/mm² Subject 001 and 0.010 cells/mm² Subject 003). This study shows that although there is an overall increased CAR-T cell persistence for a larger tumor volume, the number of CAR-T cells per 1 mm² is significantly smaller for larger tumor volumes, while illustrating that a larger post-surgical tumor bulk volume is correlated with an increased CAR-T cell persistence within 1-week post-infusion. These findings provide novel information and complement other correlative studies as part of the Phase 1 interventional clinical trial for glioblastoma patients.

Underline digital video library

Improving Diversity of Demographic Representation in Large Language Models via Collective-Critiques and Self-Voting

DiNeR: A Large Realistic Dataset for Evaluating Compositional Generalization

PROTEGE: Prompt-based Diverse Question Generation from Web Articles

Diversify Question Generation with Retrieval-Augmented Style Transfer | VIDEO

Rethinking the Smoking History Interview: A Prospective Survey-based Study

Uptake of COVID-19 Immunization in the Inflammatory Bowel Disease Population

Common Misperceptions of Child Abuse in Toddler Femur Fractures

Early-stage detection of lung cancer using a multi-omic, multi-species test

The Effects of Prenatal Adversity and Neonatal White Matter Microstructure on Child Language Skills at Age 2

Evaluation of Plastic Surgery Clinical Practice Guidelines Using the RIGHT Checklist

The Impact of a Traumatic Brain Injury Within 1 Year of COVID-19 Infection: A Propensity-Match Analysis

Glioblastoma Multiforme Tumor Volume and Persistence of Chimeric Antigen Receptor T Cells Following Neurosurgical Debulking

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES