United States

Deep reinforcement learning (DRL) has gained significant attention in autonomous systems, yet its black-box nature and lack of explainability hinder user trust in safety-critical domains such as autonomous driving. Existing experience replay approaches enhance sample efficiency but often fail to capture the internal causality of training data, leading to a convoluted training process that is difficult for humans to explain. In this work, we introduce Experience Replay with Causal Inference (ERCI), an explainable approach that integrates time series representation and causal inference to offer human-aligned explanations for DRL. Specifically, ERCI 1) introduces a novel multivariate time series representation to extract explainable Time Series Causal Factors (TSCF) from experimental data and 2) leverages internal causality in TSCFs with causal inference as a crucial standard for experience replay in DRL training. We evaluate ERCI using multiple baseline algorithms across diverse environments. Results show that ERCI provides human-aligned explanations and further improves sample efficiency through enhanced explainability. Notably, ERCI outperforms other state-of-the-art approaches by 15% in average performance, highlighting its effectiveness and generalizability.

AAAI 2025

ERCI: An Explainable Experience Replay Approach with Causal Inference for Deep Reinforcement Learning

technical paper

We are pleased to announce the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25), which will be held in Philadelphia, Pennsylvania at the Pennsylvania Convention Center from February 25 to March 4, 2025.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

### [Invited Speakers](https://aaai.org/conference/aaai/aaai-25/aaai-25-invited-speakers/)

Register [here](https://aaai.org/conference/aaai/aaai-25/registration/)

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.



This study evaluates the usage of OpenAI’s ChatGPT Large Language Model (LLM) as a tool for constructing multiple choice questions for assessing student academic performance through quizzes and exams. We randomly deploy questions constructed with and without use of the LLM tool and gauge the ability of the students to correctly answer, as well as their ability to correctly perceive the difference between human-authored and LLM-authored questions. In determining whether the questions written with the aid of ChatGPT were consistent with the instructor's questions and source text, we computed representative vectors of both the human and ChatGPT questions using SBERT and compared cosine similarity to the course textbook. A non-significant Mann-Whitney U test ($\textit{z}$ = 1.018, $\textit{p}$ = .309) suggests that students were unable to perceive whether questions were written with or without the aid of ChatGPT. However, student scores on LLM-authored questions were almost 9\% lower ($\textit{z}$ = 2.702, $\textit{p}$ $<$ .01). This result may indicate that either the AI questions were more difficult or that the students were more familiar with the instructor’s style of questions. Overall, the study suggests that while there is potential for using LLM tools to aid in the construction of assessments, care must be taken to ensure that the questions are fair, well-composed, and relevant to the course material.

LLMs in the Classroom: Outcomes and Perceptions of Questions Written with the Aid of AI

This paper addresses the problem of preference learning, which aims to align robot behaviors through learning user-specific preferences (e.g., “good pull-over location”) from visual demonstrations. Despite its similarity to learning factual concepts (e.g., “red cube”), preference learning is a fundamentally harder problem due to its subjective nature and the paucity of person-specific training data. We address this problem using a novel framework called Synapse, which is a neuro-symbolic approach designed to efficiently learn preferential concepts from limited data. Synapse represents preferences as neuro-symbolic programs -- facilitating inspection of individual parts for alignment -- in a domain-specific language (DSL) that operates over images and leverages a novel combination of visual parsing, large language models, and program synthesis to learn programs representing individual preferences. We perform extensive evaluations on various preferential concepts as well as a small user case study demonstrating the ability of Synapse to align well with dissimilar user preferences. Our method significantly outperforms baselines, especially in out-of-distribution generalization. We show the importance of the design choices in the framework through multiple ablation studies.

SYNAPSE: SYmbolic Neural-Aided Preference Synthesis Engine

Researchers, policymakers, and developers of artificial intelligence (AI) are actively collaborating to establish trustworthy AI standards that align with broader societal values, particularly in the context of large language models (LLMs). However, the critical discourse on bridging the vast knowledge gap between experts who shape and implement standards for LLMs and end-users whose values are at stake remains largely unaddressed. Taking a ‘bottom-up’ approach and using a mixed-method approach, we first conducted interviews (N = 12) to engage with end-users’ perceptions of normative standards in the context of LLMs. We thereby identified 68 criteria that end-users’ consider when evaluating value congruence. Second, we conducted an online survey (N = 379) to further investigate how end-users prioritise these standards and the identified criteria in conversational LLM-based applications. Findings reveal a clear preference for governance, monitoring, and transparent information provision over other criteria and highlight the necessity of non-technical complements to technical solutions for bridging the knowledge gap. We discuss end-user information requirements and outline actionable steps to effectively communicate trustworthy AI standards.

Bridging the Knowledge Gap: Understanding User Expectations for Trustworthy LLM Standards

Despite significant progress has been made in image deraining, most existing methods are limited to handling only a single type of rain degradation or a specific pattern of rain. However, real-world rain scenarios tend to contain diverse rainy patterns due to variations in the rainfall process and lighting conditions. To address this dilemma and advance this field, we introduce a new task: Universal Rainy Image Restoration (URIR), which aims to handle multiple types of rain degradation on a single model. To benchmark this task, we construct a high-quality dataset called URIR-8K, which contains four patterns: rain streak, raindrop, rain accumulation and nighttime rain. Building upon this dataset, we present a comprehensive study on existing approaches by evaluating their universal deraining capabilities and their effect on downstream object detection task. In addition, we design a multi-scale vision Mamba as a baseline model, leveraging the benefits of multi-scale learning for its robustness to diverse rain appearances. Unlike existing methods that use fixed-scale scanning for feature extraction, we employ a multi-scale 2D scanning technique to better help image restoration in the richer scale space. Extensive experimental analysis demonstrates the potential of our proposed task and the effectiveness of the baseline model. The code and dataset will be available.

Towards Universal Rainy Image Restoration: Benchmark and Baseline

Backdoor attacks have posed a serious threat in machine learning models, wherein adversaries can poison training samples with maliciously crafted triggers to compromise the victim model. Advanced backdoor attack methods have focused on selectively poisoning more vulnerable training samples, achieving a higher attack success rate (ASR). However, we found that when the manipulation strength of the trigger is constrained to a very small value for imperceptible attacks, they suffer from extremely uneven class-wise ASR due to the unequal selection of instances per class. To solve this issue, we propose a novel backdoor attack method based on \underline{I}nfluence-based \underline{F}air \underline{S}election (IFS), including two objectives: 1) selecting samples that significantly contribute to ASR and 2) ensuring class balance during the selection process. Specifically, we adapt Influence Functions, a classic technique in robust statistics, to evaluate the influence of trigger-embedded training samples on ASR. In this case, training samples contributing to reducing the backdoored test risk could possess higher influence scores. Further, a group-based pruning strategy is designed to avoid calculating the influence on ASR for all training samples, thereby significantly reducing the computational cost. Then, based on the influence score, we design an adaptive thresholding scheme to dynamically select samples with higher influence while maintaining class balance. Extensive experiments on four datasets verify the effectiveness of IFS compared with advanced methods.

Influence-Based Fair Selection for Sample-Discriminative Backdoor Attack

Abstract
Reinforcement learning (RL), particularly its combination with deep neural networks referred to as
deep RL (DRL), has shown tremendous promise across a wide range of applications, suggesting its po-
tential for enabling the development of sophisticated robotic behaviors. Robotics problems, however,
pose fundamental difficulties for the application of RL, stemming from the complexity and cost of
interacting with the physical world. These challenges notwithstanding, recent advances have enabled
DRL to succeed at some real-world robotic tasks. However, state-of-the-art DRL solutions’ maturity
varies significantly across robotic applications. In this talk, I will review the current progress of DRL
in real-world robotic applications based on our recent survey paper (with Tang, Abbatematteo, Hu,
Chandra, and Martı́n-Martı́n), with a particular focus on evaluating the real-world successes achieved
with DRL in realizing several key robotic competencies, including locomotion, navigation, stationary
manipulation, mobile manipulation, human-robot interaction, and multi-robot interaction. The anal-
ysis aims to identify the key factors underlying those exciting successes, reveal underexplored areas,
and provide an overall characterization of the status of DRL in robotics. I will also highlight several
important avenues for future work, emphasizing the need for stable and sample-efficient real-world
RL paradigms, holistic approaches for discovering and integrating various competencies to tackle
complex long-horizon, open-world tasks, and principled development and evaluation procedures. The
talk is designed to offer insights for RL practitioners and roboticists toward harnessing RL’s power
to create generally capable real-world robotic systems.

Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes

Scientific discovery is a complex cognitive process that has driven human knowledge and technological progress for centuries. While artificial intelligence (AI) has made significant advances in automating aspects of scientific reasoning, simulation, and experimentation, we still lack integrated AI systems capable of performing autonomous long-term scientific research and discovery. This paper examines the current state of AI for scientific discovery, highlighting recent progress in large language models and other AI techniques applied to scientific tasks. We then outline key challenges and promising research directions toward developing more comprehensive AI systems for scientific discovery, including the need for science-focused AI agents, improved benchmarks and evaluation metrics, multimodal scientific representations, and unified frameworks combining reasoning, theorem proving, and data-driven modeling. Addressing these challenges could lead to transformative AI tools to accelerate progress across disciplines towards scientific discovery.

Towards Scientific Discovery with Generative AI: Progress, Opportunities, and Challenges

This paper surveys Machine Learning approaches to build predictive models that know what they don't know. The consequential action of this knowledge can consist of abstaining from providing an output (rejection), deferring to another model (dynamic model selection), deferring to a human expert (learning to defer), or informing the user (uncertainty estimation). We formally state the problems each approach solves and point to key references. We discuss open issues that deserve investigation from the scientific community.

Things Machine Learning Models Know That They Don’t Know

The future of Artificial Intelligence demands a paradigm shift towards multisensory perception—to systems that can digest ongoing multisensory observations, that can discover structure in unlabeled raw sensory data, and that can intelligently fuse useful information from different sensory modalities for decision making. While we humans perceive the world by looking, listening, touching, smelling, and tasting, traditional form of machine intelligence mostly focuses on a single sensory modality, particularly vision. Therefore, my research, which I call multisensory machine intelligence, aims to empower machines to emulate and enhance human capabilities in seeing, hearing, and feeling, ultimately enabling them to comprehensively perceive, understand, and interact with the multisensory world. In my AAAI-25 new faculty highlight talk, I will present my research that studies two important aspects of the multisensory world: 1) multisensory objects, and 2) multisensory space. In both aspects, I will talk about how we design systems to reliably capture multisensory data from real-world objects and space, how we effectively model them with differentiable simulation algorithms that build a unified multisensory representation to virtualize real objects, and how we explore creative cross-modal/multi-modal applications with sight, sound, and touch in vision, graphics, and robotics. In the end, I will briefly conclude with my future plans.

Multisensory Machine Intelligence

In many human-robot collaboration and multi-agent tasks, it is vital to model the partners and estimate their objectives to efficiently collaborate/interact with them. While learning from demonstrations is the most common approach for this, it is very data-hungry, which we cannot afford in many settings including robotics, and demonstrations are unreliable in a surprisingly large number of domains, including those we think humans perform reasonably well, e.g., driving. In this talk, I will start with introducing comparison-based feedback and explain why it does not suffer from most of the problems that demonstrations have, but is still data-hungry. To address this problem, I will propose comparative language based feedback and active learning techniques, which will result in (1) a new type of human feedback, and (2) an active querying algorithm that optimizes the information the AI agent will elicit from the human. I will conclude the talk by discussing what other types of human feedback exist, e.g., interventions or hand gestures, and how we can incorporate them into the existing learning algorithms.

ERCI: An Explainable Experience Replay Approach with Causal Inference for Deep Reinforcement Learning

Downloads

Next from AAAI 2025

LLMs in the Classroom: Outcomes and Perceptions of Questions Written with the Aid of AI

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

ERCI: An Explainable Experience Replay Approach with Causal Inference for Deep Reinforcement Learning

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAAI 2025

LLMs in the Classroom: Outcomes and Perceptions of Questions Written with the Aid of AI

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads