United States

The advent of large language models (LLMs) has sparked significant interest in using natural language for preference learning. However, existing methods often suffer from high computational burdens, taxing human supervision, and lack of interpretability. To address these issues, we introduce MAPLE, a framework for large language model-guided Bayesian active preference learning. MAPLE leverages LLMs to model the distribution over preference functions, conditioning it on both natural language feedback and conventional preference learning feedback, such as pairwise trajectory rankings. MAPLE also employs active learning to systematically reduce uncertainty in this distribution and incorporates a language-conditioned active query selection mechanism to identify informative and easy-to-answer queries, thus reducing human burden. We evaluate MAPLE&#39;s sample efficiency and preference inference quality across two benchmarks, including a real-world vehicle route planning benchmark using OpenStreetMap data. Our results demonstrate that MAPLE accelerates the learning process and effectively improves humans&#39; ability to answer queries.

AAAI 2025

MAPLE: A Framework for Active Preference Learning Guided by Large Language Models

The advent of large language models (LLMs) has sparked significant interest in using natural language for preference learning. However, existing methods often suffer from high computational burdens, taxing human supervision, and lack of interpretability. To address these issues, we introduce MAPLE, a framework for large language model-guided Bayesian active preference learning. MAPLE leverages LLMs to model the distribution over preference functions, conditioning it on both natural language feedback and conventional preference learning feedback, such as pairwise trajectory rankings. MAPLE also employs active learning to systematically reduce uncertainty in this distribution and incorporates a language-conditioned active query selection mechanism to identify informative and easy-to-answer queries, thus reducing human burden. We evaluate MAPLE's sample efficiency and preference inference quality across two benchmarks, including a real-world vehicle route planning benchmark using OpenStreetMap data. Our results demonstrate that MAPLE accelerates the learning process and effectively improves humans' ability to answer queries.

poster

We are pleased to announce the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25), which will be held in Philadelphia, Pennsylvania at the Pennsylvania Convention Center from February 25 to March 4, 2025.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

### [Invited Speakers](https://aaai.org/conference/aaai/aaai-25/aaai-25-invited-speakers/)

Register [here](https://aaai.org/conference/aaai/aaai-25/registration/)

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.



Significant efforts have been made to analyze the political stance or bias in news articles, especially as political polarization intensifies over the years. Recent advancements in machine learning have enabled researchers to develop various bias prediction models, which typically learn features not only from the text of the news articles but also from external knowledge. However, when training these models, the political bias label assigned to a news article is often based solely on the news source that published it. This approach can be problematic, as a news outlet with a particular political stance might publish an article that reflects a different political perspective.

To address this issue, we first find out distinct text patterns that pertain to a particular news source (or a publisher), which are barely meaningful to predict the political bias of its news article. Then, we conduct comprehensive experiments to investigate (i) whether the prior models trained to predict the bias can also predict its source and (ii) whether the prior models change prediction results if a distinct pattern of a source with a different political stance is inserted to a news article. 
Our experimental results reveal that all the prior models are prone to predict the source even if they are trained to predict the bias.
We finally suggest a new deep learning model for political bias prediction that avoids learning the source-indicative patterns.

Political Bias Prediction Models Focus on Source Cues, Not Semantics

The success of the reward model in distinguishing between responses with subtle safety differences depends critically on the high-quality preference dataset, which should capture the fine-grained nuances of harmful and harmless responses. This motivates the need to develop the datasets involving preference margins, which accurately quantify how harmless one response is compared to another. In this paper, we take the first step to propose an effective and cost-efficient framework to promote the margin-enhanced preference dataset development. Our framework, Legend, Leverages rEpresentation enGineering to annotate preferENce Datasets. It constructs the specific direction within the LLM's embedding space that represents safety. By leveraging this safety direction, Legend can then leverage the semantic distances of paired responses along this direction to annotate margins automatically. We experimentally demonstrate our effectiveness in both reward modeling and harmless alignment for LLMs. Legend also stands out for its efficiency, requiring only the inference time rather than additional training. This efficiency allows for easier implementation and scalability, making Legend particularly valuable for practical applications in aligning LLMs with safe conversations.

LEGEND: Leveraging Representation Engineering to Annotate Safety Margin for Preference Datasets

Human preference alignment is critical in building powerful and reliable large language models (LLMs). However, current methods either ignore the multi-dimensionality of human preferences (e.g. helpfulness and harmlessness) or struggle with the complexity of managing multiple reward models. To address these issues, we propose Sequential Preference Optimization (SPO), a method that sequentially fine-tunes LLMs to align with multiple dimensions of human preferences. SPO avoids explicit reward modeling, directly optimizing the models to align with nuanced human preferences. We theoretically derive closed-form optimal SPO policy and loss function. Gradient analysis is conducted to show how SPO manages to fine-tune the LLMs while maintaining alignment on previously optimized dimensions. Empirical results on LLMs of different size and multiple evaluation datasets demonstrate that SPO successfully aligns LLMs across multiple dimensions of human preferences and significantly outperforms the baselines.

Sequential Preference Optimization: Multi-Dimensional Preference Alignment with Implicit Reward Modeling

Accurate standard plane acquisition in fetal ultrasound (US)
videos is crucial for fetal growth assessment, anomaly detec-
tion, and adherence to clinical guidelines. However, manu-
ally selecting standard frames is time-consuming and prone
to intra- and inter-sonographer variability. Existing methods
primarily rely on image-based approaches that capture stan-
dard frames and then classify the input frames across dif-
ferent anatomies. This ignores the dynamic nature of video
acquisition and its interpretation. To address these chal-
lenges, we introduce Multi-Tier Class-Aware Token Trans-
former (MCAT); a visual query-based video clip localiza-
tion (VQ-VCL) method to assist sonographers by enabling
them to capture a quick ultrasound sweep. By then pro-
viding a visual query of the anatomy they wish to ana-
lyze, MCAT returns the video clip containing the standard
frames for that anatomy, facilitating thorough screening for
potential anomalies. We evaluate MCAT on two ultrasound
video datasets and a natural image VQ-VCL dataset based
on Ego4D. Our model outperforms state-of-the-art methods
by 10% and 13% mtIoU on the ultrasound datasets and by
5.35% mtIoU on the Ego4D dataset, using 96% fewer tokens.
MCAT’s efficiency and accuracy have significant potential
implications for public health, especially in low- and middle-
income countries (LMICs), where it may enhance prenatal
care by streamlining standard plane acquisition, simplifying
ultrasound-based screening and diagnosis andallowing sono-
graphers to examine more patients. The code will be available
at xxx.github.com and in supplementary material.

MCAT: Visual Query-Based Localization of Standard Anatomical Clips in Fetal Ultrasound Videos Using Multi-Tier Class-Aware Token Transformer

Local governments around the world are making consequential decisions on behalf of their constituents, and these constituents are responding with requests, advice, and assessments of their officials at public meetings. So many small meetings cannot be covered by traditional newsrooms at scale. We propose PublicSpeak, a probabilistic framework which can utilize meeting structure, domain knowledge, and linguistic information to discover public remarks in local government meetings. We then use our approach to inspect the issues raised by constituents in 7 cities across the United States. We evaluate our approach on a novel dataset of local government meetings and find that PublicSpeak improves over state-of-the-art by 10\% on average,  with gains of up to 40\%.

PUBLICSPEAK: Hearing the Public with a Probabilistic Framework

Large language models (LLMs) offer a valuable technology for various applications in healthcare. However, their tendency to hallucinate and the existing reliance on proprietary systems pose challenges in environments concerning critical decision-making and strict data privacy regulations, such as healthcare, where the trust in such systems is paramount. Through combining the strengths and discounting the weaknesses of humans and AI, the field of Human-AI Collaboration (HAIC) presents one front for tackling these challenges and hence improving trust. This paper presents a novel HAIC $\textit{guided deferral}$ system that can simultaneously parse medical reports and defer uncertain predictions with intelligent guidance to humans. We develop methodology which builds efficient, effective and open-source LLMs for this purpose, for the real-world deployment in healthcare. We conduct a pilot study which showcases the effectiveness of our proposed system in practice. Additionally, we highlight drawbacks of standard calibration metrics in imbalanced data scenarios commonly found in healthcare, and suggest a simple yet effective solution: the Imbalanced Expected Calibration Error ($\mathrm{ECE_{Imb}}$). We release our code for practitioners wishing to replicate our system.

Trustworthy and Practical AI for Healthcare: A Guided Deferral System with Large Language Models

Large language models (LLMs) and small language models (SLMs) are being adopted at remarkable speed, although their safety still remains a serious concern. With the advent of multilingual S/LLMs, the question now becomes a matter of scale: can we expand multilingual safety evaluations of these models with the same velocity at which they are deployed? To this end, we introduce RTP-LX, a human-transcreated and human-annotated corpus of toxic prompts and outputs in 28 languages. RTP-LX follows participatory design practices, and a portion of the corpus is especially designed to detect culturally-specific toxic language. We evaluate 10 S/LLMs on their ability to detect toxic content in a culturally-sensitive, multilingual scenario. We find that, although they typically score acceptably in terms of accuracy, they have low agreement with human judges when scoring holistically the toxicity of a prompt; and have difficulty discerning harm in context-dependent scenarios, particularly with subtle-yet-harmful content (e.g. microaggressions, bias). We release this dataset to contribute to further reduce harmful uses of these models and improve their safe deployment.

RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios?

Tropical cyclone (TC) intensity forecasting is crucial for early disaster warning and emergency decision-making. Numerous researchers have explored deep-learning methods to address computational and post-processing issues in operational forecasting. Regrettably, they exhibit subpar long-term forecasting capabilities. We use two strategies to enhance long-term forecasting. (1) By enhancing the matching between TC intensity and spatial information, we can improve long-term forecasting performance. (2) Incorporating physical knowledge and physical constraints can help mitigate the accumulation of forecasting errors. To achieve the above strategies, we propose the VQLTI framework. VQLTI transfers the TC intensity information to a discrete latent space while retaining the spatial information differences, using large-scale spatial meteorological data as conditions. Furthermore, we leverage the forecast from the weather prediction model FengWu to provide additional physical knowledge for VQLTI. Additionally, we calculate the potential intensity (PI) to impose physical constraints on the latent variables. In the global long-term TC intensity forecasting, VQLTI achieves state-of-the-art results for the 24h to 120h, with the MSW (Maximum Sustained Wind) forecast error reduced by 35.65%-42.51% compared to ECMWF-IFS. The code implementation is available at https://anonymous.4open.science/r/VQLTI-CEF2.

VQLTI: Long-Term Tropical Cyclone Intensity Forecasting with Physical Constraints

The soaring drug and substance use crisis in the United States has claimed more than half a million lives in the past decade and remains a major public health threat. The ability to predict drug overdose deaths at the county level can help local communities develop action plans in response to emerging changes. Applying off-the-shelf machine learning algorithms for prediction can be challenging due to the heterogeneous risk profiles of the counties and suppressed data in common publicly available data sources. To fill these gaps, we develop a cluster-aware supervised learning (CASL) framework to enhance the prediction of county-level drug overdose deaths. This CASL model simultaneously clusters counties into groups based on geographical and socioeconomic characteristics and minimizes the loss function that accounts for suppressed values and cluster-specific regularization. Our computational study uses real-world data from 2010 to 2021, focusing on the ten states most severely impacted by the drug overdose crisis. The results demonstrate that our proposed CASL framework significantly outperforms state-of-the-art methods by achieving a superior balance in prediction accuracy for both unsuppressed and suppressed observations. The proposed model also uncovers different clusters of counties, capturing the underlying heterogeneity in the patterns of overdose mortality among counties of various characteristics.

A Spatio-temporal Cluster-aware Supervised Learning Framework for Predicting County-level Drug Overdose Deaths

Understanding internal joint loading is critical for diagnosing gait-related diseases such as knee osteoarthritis; however, current methods of measuring joint risk factors with force plates and 3D motion capture systems are time-consuming, expensive, and restricted to controlled lab settings, limiting their applicability to real-world contexts. Thus, in this paper, we aim to enable large-scale, cost-effective diagnosis of joint-related diseases via three key contributions: the development and deployment of novel instrumented insoles, the curation of a large multimodal biomechanics dataset, VidSole, and the evaluation of a baseline deep learning pipeline to predict internal joint loading factors. Our VidSole dataset combines the forces and moments measured by the insoles with RGB video from two viewpoints, 3D body motion capture, and force plate data for over 2,6000 trials of 52 participants performing four fundamental activities of daily living (sit-to-stand, stand-to-sit, walking, and running). We feed the insole data and kinematic parameters extractable from video (ie. pose, knee angle) into a deep learning pipeline, consisting of ensembled Gated Recurrent Unit (GRU) and Long Short Term Memory (LSTM) models, to first classify between the activities of daily living (99.16% accuracy) then estimate knee adduction moment (KAM). We validate the model’s KAM estimation by demonstrating that our mean absolute error (MAE) falls within the acceptable range of < (0.5% to 2.1%)$\times$body weight$\times$height, the current threshold for accurately detecting knee osteoarthritis with KAM. Thus, we believe our instrumented insoles, VidSole dataset, and deep learning pipeline are useful for accurately quantifying joint kinetic measurements and can assist in preventing joint-related diseases.

MAPLE: A Framework for Active Preference Learning Guided by Large Language Models

Downloads

Next from AAAI 2025

Political Bias Prediction Models Focus on Source Cues, Not Semantics

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES