Detecting hallucinations in natural language processing (NLP) is a critical undertaking that demands a deep understanding of both the semantic and pragmatic aspects of languages. Cognitive approaches that leverage users’ behavioural signals, such as gaze, have demonstrated effectiveness in addressing NLP tasks with similar linguistic complexities. However, their potential in the context of hallucination detection remains largely unexplored. In this paper, we propose a novel cognitive approach for hallucination detection that leverages gaze signals from humans. We first collect and introduce an eye tracking corpus (IITB-HGC: IITB-Hallucination Gaze corpus) consisting of 500 instances, annotated by five annotators for hallucination detection. Our analysis reveals that humans selectively attend to relevant parts of the text based on distributional similarity, similar to the attention bias phenomenon in psychology. We identify two attention strategies employed by humans: global attention, which focuses on the most informative sentence, and local attention, which focuses on important words within a sentence. Leveraging these insights, we propose a novel cognitive framework for hallucination detection that incorporates these attention biases. Experimental evaluations on the FactCC dataset demonstrate the efficacy of our approach, obtaining a balanced accuracy of 87.1%. Our study highlights the potential of gaze-based approaches in addressing the task of hallucination detection and sheds light on the cognitive processes employed by humans in identifying inconsistencies.

Eyes Show the Way: Modelling Gaze Behaviour for Hallucination Detection

In behavioural testing, system functionalities underrepresented in the standard evaluation setting (with a held-out test set) are validated through controlled input-output pairs. Optimising performance on the behavioural tests during training (behavioural learning) would improve coverage of phenomena not sufficiently represented in the i.i.d. data and could lead to seemingly more robust models. However, there is the risk that the model narrowly captures spurious correlations from the behavioural test suite, leading to overestimation and misrepresentation of model performanceone of the original pitfalls of traditional evaluation. In this work, we introduce BeLUGA, an analysis method for evaluating behavioural learning considering generalisation across dimensions of different granularity levels. We optimise behaviour-specific loss functions and evaluate models on several partitions of the behavioural test suite controlled to leave out specific phenomena. An aggregate score measures generalisation to unseen functionalities (or overfitting). We use BeLUGA to examine three representative NLP tasks (sentiment analysis, paraphrase identification and reading comprehension) and compare the impact of a diverse set of regularisation and domain generalisation methods on generalisation performance.

Cross-functional Analysis of Generalisation in Behavioural Learning | VIDEO

Large language models (LLMs) show powerful reasoning abilities on various text-based tasks. However, their reasoning capability on structured data such as tables has not been systematically explored. In this work, we first establish a comprehensive taxonomy of reasoning and operation types for tabular data analysis. Then, we construct a complex reasoning QA dataset over tabular data, named CRT-QA dataset (Complex Reasoning QA over Tabular data), with the following unique features: (1) it is the first Table QA dataset with multi-step operation and informal reasoning; (2) it contains fine-grained annotations on questions' directness, composition types of sub-questions, and human reasoning paths which can be used to conduct a thorough investigation on LLMs' reasoning ability; (3) it contains a collection of unanswerable and indeterminate questions that commonly arise in real-world situations. We further introduce an efficient and effective tool-augmented method, named ARC (Auto-exemplar-guided Reasoning with Code), to use external tools such as Pandas to solve table reasoning tasks without handcrafted demonstrations. The experiment results show that CRT-QA presents a strong challenge for baseline methods and ARC achieves the best result.

CRT-QA: A Dataset of Complex Reasoning Question Answering over Tabular Data

Research in psychopathology has shown that, at an aggregate level, the patterns of emotional change over time---emotion dynamics---are indicators of one's mental health. One's patterns of emotion change have traditionally been determined through self-reports of emotions; however, there are known issues with accuracy, bias, and convenience. Recent approaches to determining emotion dynamics from one's everyday utterances, addresses many of these concerns, but it is not yet known whether these measures of utterance emotion dynamics (UED) correlate with mental health diagnoses. Here, for the first time, we study the relationship between tweet emotion dynamics and mental health disorders. We find that each of the UED metrics studied varied by the user's self-disclosed diagnosis. For example: average valence was significantly higher (i.e., more positive text) in the control group compared to users with ADHD, MDD, and PTSD. Valence variability was significantly lower in the control group compared to ADHD, depression, bipolar disorder, MDD, PTSD, and OCD but not PPD. Rise and recovery rates of valence also exhibited significant differences from the control. This work provides important early evidence for how linguistic cues pertaining to emotion dynamics can play a crucial role as biosocial markers for mental illnesses and aid in the understanding, diagnosis, and management of mental health disorders.

Language and Mental Health: Measures of Emotion Dynamics from Text as Linguistic Biosocial Markers

Subword tokenization has become the de-facto standard for tokenization although comparative evaluations of their quality across languages are scarce. Existing evaluation studies focus on the effect of a tokenization algorithm on the performance in downstream tasks, or on engineering criteria such as the compression rate. We present a new evaluation paradigm that focuses on the cognitive plausibility of subword tokenization. We analyze the correlation of the tokenizer output with the reading time and accuracy of human responses on a lexical decision task. We compare three tokenization algorithms across several languages and vocabulary sizes. Our results indicate that the Unigram algorithm yields less cognitively plausible tokenization behavior and a worse coverage of derivational morphemes, in contrast with prior work.

Analyzing Cognitive Plausibility of Subword Tokenization

As short-form funny videos on social networks are gaining popularity, it becomes demanding for AI models to understand them for better communication with humans. Unfortunately, previous video humor datasets target specific domains such as speeches or sitcoms, and mostly focus on verbal cues.
We curate a user-generated dataset of 10K multimodal funny videos from YouTube, called ExFunTube. Using a video filtering pipeline with GPT-3.5, we verify both verbal and visual elements contributing to humor. After filtering, we annotate each video with timestamps and text explanations for funny moments. Our ExFunTube is unique over existing datasets in that our videos cover a wide range of domains with various types of humor that necessitate a multimodal understanding of the content.
Also, we develop a zero-shot video-to-text prompting to maximize video humor understanding of large language models (LLMs). With three different evaluation methods using automatic scores, rationale quality experiments, and human evaluations, we show that our prompting significantly improves LLMs' ability for humor explanation.

Can Language Models Laugh at YouTube Short-form Videos?

Background: 
With more than half of U.S. adults reporting alcohol consumption the last 30 days and nearly 6% reporting heavy drinking, alcohol imposes a significant socioeconomic burden. While there are a host of current medications approved to treat alcoholism, they have limited effectiveness and lack support from both providers and patients. Nevertheless, new medications are slow in coming. Semaglutide, a medication originally developed for obese type 2 diabetics, may offer a distinctly new approach. 
Methods: 
A survey to assess the impact of semaglutide on alcohol consumption and desire was designed and distributed in semaglutide-user Facebook and Reddit groups and frequencies and proportions were calculated.
Results: 
Of those who had reported drinking before starting semaglutide, 21% of respondents (N= 121) ceased drinking altogether, and 88·4% of respondents (N=538) reported a reduction in their desire to consume alcohol after commencing their medication. Respondents also shared further personal accounts of decreased occurrences of compulsive or "addictive" behaviors beyond a reduction in the desire to consume alcohol.
Conclusions: 
Considering the growing number of reports detailing experiences such as these, combined with our survey results indicating a reduction in both frequency and quantity of alcohol consumption, we strongly recommend conducting scientifically rigorous investigations and clinical trials to explore the potential of semaglutide for addressing conditions associated with alcohol use. 


Semaglutide: A Promising Treatment for Alcohol Use Disorder

**Scoring is only possible in the semifinals hall**

**Abstract:**



Analyzing the Quality of Reporting of AI Studies for Hip Fracture Detection

Background:
Among the genes related to familial Parkinson’s disease (PD), the D620N VPS35 mutation is associated with an increase in -synuclein (αS) aggregates and neuronal death. Reducing the αS levels in neurons is a potential strategy for reducing αS accumulation and consequent dopaminergic neuronal damage. Recent studies suggest that VPS35 regulates αS degradation by impacting the endosomal-lysosomal system (ELS), macroautophagy (MA), chaperone-mediated autophagy (CMA), and the ubiquitin-proteasome system (UPS). Our aim is to investigate the role of VPS35 in increasing αS clearance and to test a novel VPS35 stabilizer, guanidine hydrazone (2a). 

Methods:
We overexpressed exogenous mutant A53T αS by lentiviral infection or transient transfection in SH-SY5Y cells, co-treated this group with 2a (or saline control) at 20 uM for 3, 4 or 5 days, and analyzed the levels of αS, VPS35, and αS degradation pathway markers by western blot. We evaluated lysosomal function through the Cathepsin-D (CTSD) activity assay. We then looked at the effect of 2a on αS levels after selective blockage of αS degradation pathways. Macroautophagy was blocked after 1 day with bafilomycin A1 (BfA1) at 50 nM and UPS was inhibited after 8 hours with MG-132 at 10 uM. 

Results:
2a increased VPS35 protein levels (~75%) and retromer function, with an increased level of the VPS35 cargo VMAT2 (~50%). This correlates with a ~ 50% reduction of Phospho-Ser129 αS with transient transfection and total αS with lentiviral infection, in the soluble fraction. Furthermore, we found an increase in ELS markers (LAMP1, Rab5), CMA markers (LAMP2a, Hsc70), macroautophagy markers (LC3b), UPS markers (PSMB5), highlighting the effect of 2a on boosting these pathways. Interestingly, CTSD activity was decreased after 2a treatment, which could be explained by a CTSD “burnout” due to 2a-mediated ELS boosting. After blocking macroautophagy with BfA1 and UPS with MG-132, the western blot data showed ~75 % reduction of Phospho-Ser129 αS in the infected group treated with 2a plus BfA1 or MG-132 compared to the infected cells treated only with BfA1 or MG-132. Thus, 2a can decrease αS levels even when macroautophagy or UPS are inhibited by BfA1 or MG-132 respectively. This result could suggest that other degradation pathways predominate in mediating the 2a-induced αS clearance, but it is also possible that compensatory upregulation occurs in the non-blocked pathways. 

Conclusion:
These findings highlight Vps35 as a novel pharmacological target for PD treatment and the role ELS and UPS play in αS degradation and, potentially, neuroprotection in PD patients.

The endosomal-lysosomal and ubiquitin-proteasome systems control alpha-synuclein degradation in PD

Background: Every year, over 3 million new concussions are diagnosed in the United States. The recovery from these concussions is not yet standardized as the post concussive symptoms (PCS) patients face are influenced by a myriad of medical, social, and psychological problems. Although recent studies have noted a correlation between premorbid psychological conditions and worsened PCS, there is not enough literature that studies a possible relationship between adverse childhood experiences (ACEs) and PCS. ACEs, traumatic events that occur to individuals during their childhood, have been shown to be associated with numerous negative health outcomes. This study aims to identify if higher ACE scores are correlated with worse PCS, or if patients with high ACEs develop a resiliency that is protective against PCS. 



Methods: This is a cross-sectional study of patients recruited from a concussion clinic at Saint Louis University Hospital. The inclusion criteria for this study: (1) patients between ages of 18-80, (2) diagnosed with a concussion or mild traumatic brain injury by their physician, (3) presenting as a new patient or a follow up for concussion care, (4) possess ability to speak and read English. Eligible participants, after completing appropriate consent and HIIPA authorization forms, complete assessments evaluating their ACEs and current PCS. Surveys for this study include: the Adverse Childhood Experiences questionnaire (ACEs), Adult Resilience Measure (ARM), Neuropsychiatric Symptom Inventory (NSI), Patient Healthcare Questionnaire (PHQ-9) and Generalized Anxiety Disorder-7 (GAD-7). The data from these assessments is compiled into a HIIPA-compliant spreadsheet. From there, statistical analyses are done to identify possible correlations between ACEs, PCS, ARM scores, depression, and anxiety. 



Results: Values from 66 patients were collected and Pearson Correlation values were subsequently calculated. The analyses yielded the following correlations: NSI and ACEs had a mild positive correlation of 0.19, NSI and ARM had a moderate negative correlation of -0.46, and ACE & ARM had a mild negative correlation of -0.13. Scatterplots with trendlines of these correlations were also created. 



Conclusion: Our results so far indicate a mild positive correlation between PCS and ACE and a moderate negative correlation between PCS and resiliency. There is not a strong correlation between ACE and resiliency.

Influence of Adverse Childhood Experiences on Postconcussive Symptoms

Brain network organization varies between individuals, and individual differences in intrinsic brain networks have been linked to differences in cognitive ability. Individual differences in brain connectivity may be especially evident in brain networks that support executive functions such as cognitive control and attention such as the frontoparietal and salience networks. The frontoparietal network is theorized to play a role in cognitive control, which refers to the selection of thoughts and behaviors in context of task demands. Specifically, the frontoparietal network is necessary to flexibly coordinate behavior in a goal-driven manner. Meanwhile, the salience network has been hypothesized to modulate the relationship between the default mode network, which contributes to self-oriented cognition, and the frontoparietal network.
Prior work suggests that the maturation of executive functions during development parallels the integration of the salience networks with the frontoparietal network. Additionally, increased frontoparietal and salience intranetwork connectivity has been associated with development from periadolescence to adulthood. However, associations between frontoparietal or salience intranetwork connectivity and executive functions abilities have not been tested in periadolescent children. Here we tested whether intranetwork connectivity of frontoparietal and salience networks was correlated with executive function measures in periadolescent children. Participants in the NIA-funded Polygenic Risk of Alzheimer’s disease in Nebraska Kids (PRANK) study (N = 94) completed the NIH toolbox cognition battery as well as a functional MRI (fMRI) scan including resting-state data collection. Intranetwork connectivity was measured using the Human Connectome Project’s connectome workbench software. Results of the data analysis indicated a positive correlation between scores on fluid cognition measures and frontoparietal intranetwork connectivity, r(73) = .29, p = .01. A positive correlation was also observed with flanker task scores and frontoparietal intranetwork connectivity, r(73) = .25, p = .03. Finally, we observed a positive correlation with salience intranetwork connectivity and age-adjusted dimensional card sorting task scores, r(91) = .22, p = .03. Our preliminary findings show an association of executive function measures with intranetwork connectivity of two key intrinsic functional networks, and they suggest that frontoparietal and salience intranetwork connectivity may reflect increased efficiency of cognitive control during development.

Associations between salience and frontoparietal intranetwork connectivity and executive functions tasks in periadolescent children.

Background
• To assess knowledge regarding MDI and its technique amongst healthcare
personnel and to understand its importance.
• To spread awareness regarding the same via educational tools
• To compare the understanding of the technique before and after the use of
educational tools.
Methods
A study group of 213 participants consisting of residents from the
departments of internal medicine (48), pediatrics (36), respiratory medicine
(8), community medicine (11) and staff nurses (110) of all departments were
selected as these acquire most of the cases related to the usage MDI. They
were provided with educational tools. A comparison was made in their
knowledge about the technique of MDI before and after the use of education
tools using a pre-validated questionnaire and the results were obtained in
terms of the mean scores.
Results
The mean score attained by residents of the internal medicine department
was 6.5/16 (standard deviation (S.D)-2.44) in the pre-test and 12.82/16 (S.D-
2.61) in the post-test with an improvement of 6.32 marks (t-value-10.00) and
a p-value of 0.0001. The residents in the pediatrics department had a mean
score of 5.76/16 (S.D-2.36) in the pre-test and 12.36/16(S.D-3.47) in the post
test with an improvement of 6.60 (t-value-10.22) and a p-value of 0.0001. The residents in respiratory medicine had a mean score of 6.33/16(S.D-2.87) in the pre-test and 13.66/16(S.D-2.42) in the post-test with an improvement of 7.33 marks(t-value-4.82) and a p-value of 0.005. The residents in the community medicine department had a mean score of 3.90/16 (S.D-1.37) in the pre-test and 10.09/16 (S.D-3.08) in the post-test with an improvement of 6.18 marks (t-value-6.43) and a p-value of 0.0001. The nurses had a mean score of 4.81/16 (S.D-2.43) in the pre-test and 11.75/16 (S.D- 2.94) in the post-test with an improvement of 6.93 marks (t-value 18.33) and a p value of 0.0001.
Conclusion
Residence of all departments performed significantly better after audiovisual educational tools. Maximum improvement was seen amongst the nurses. Most of the healthcare personnel had better knowledge regarding the usage of the MDI technique after the education tools. In developing countries like India where the number of asthma patients is so high, formal and systematic training should be provided to the healthcare personnel who are directly involved in prescribing and instructing the use of MDI technique to patients to improve patient outcomes by increasing adherence and to decrease healthcare burden by decreasing the number of exacerbations.

Underline digital video library

Eyes Show the Way: Modelling Gaze Behaviour for Hallucination Detection

Cross-functional Analysis of Generalisation in Behavioural Learning | VIDEO

CRT-QA: A Dataset of Complex Reasoning Question Answering over Tabular Data

Language and Mental Health: Measures of Emotion Dynamics from Text as Linguistic Biosocial Markers

Analyzing Cognitive Plausibility of Subword Tokenization

Can Language Models Laugh at YouTube Short-form Videos?

Semaglutide: A Promising Treatment for Alcohol Use Disorder

Analyzing the Quality of Reporting of AI Studies for Hip Fracture Detection

The endosomal-lysosomal and ubiquitin-proteasome systems control alpha-synuclein degradation in PD

Influence of Adverse Childhood Experiences on Postconcussive Symptoms

Associations between salience and frontoparietal intranetwork connectivity and executive functions tasks in periadolescent children.

IMPACT OF AUDIO-VISUAL AIDS IN IMPROVING THE KNOWLEDGE OF THE METERED DOSE INHALER (MDI) TECHNIQUE AMONG HEALTHCARE PERSONNEL

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES