United States

This study investigates the creative potential of Large Language Models (LLMs) in mathematical reasoning, an area previously under-explored. We propose a novel framework and benchmark, incorporating problems from middle school to Olympic-level competitions, to evaluate LLMs&#39; ability to generate novel solutions, employ multi-stage methods, and provide insightful reasoning. Our experiments reveal that while LLMs excel in standard mathematical tasks, their creative problem-solving abilities vary significantly. Notably, the Gemini-1.5-Pro model excelled in producing novel solutions across all tested LLMs. This research pioneers a new direction in assessing AI creativity, highlighting both the strengths and limitations of LLMs in mathematical innovation, and paves the way for future advancements in AI-driven mathematical discovery.

AAAI 2025

Assessing the Creativity of LLMs in Proposing Novel Solutions to Mathematical Problems

snlp

language models

This study investigates the creative potential of Large Language Models (LLMs) in mathematical reasoning, an area previously under-explored. We propose a novel framework and benchmark, incorporating problems from middle school to Olympic-level competitions, to evaluate LLMs' ability to generate novel solutions, employ multi-stage methods, and provide insightful reasoning. Our experiments reveal that while LLMs excel in standard mathematical tasks, their creative problem-solving abilities vary significantly. Notably, the Gemini-1.5-Pro model excelled in producing novel solutions across all tested LLMs. This research pioneers a new direction in assessing AI creativity, highlighting both the strengths and limitations of LLMs in mathematical innovation, and paves the way for future advancements in AI-driven mathematical discovery.

technical paper

We are pleased to announce the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25), which will be held in Philadelphia, Pennsylvania at the Pennsylvania Convention Center from February 25 to March 4, 2025.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

### [Invited Speakers](https://aaai.org/conference/aaai/aaai-25/aaai-25-invited-speakers/)

Register [here](https://aaai.org/conference/aaai/aaai-25/registration/)

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.



Considering the difficulty of interpreting generative model output, there is significant current research focused on determining meaningful evaluation metrics. Several recent approaches utilize "precision" and "recall," borrowed from the classification domain, to individually quantify the output fidelity (realism) and output diversity (representation of the real data variation), respectively. With the increase in metric proposals, there is a need for a unifying perspective, allowing for easier comparison and clearer explanation of their benefits and drawbacks. To this end, we unify a class of kth-nearest-neighbors (kNN)-based metrics under an information-theoretic lens using approaches from kNN density estimation. Additionally, we propose a tri-dimensional metric composed of Precision Cross-Entropy (PCE), Recall Cross-Entropy (RCE), and Recall Entropy (RE), which separately measure fidelity and two distinct aspects of diversity, inter- and intra-class. Our domain-agnostic metric, derived from the information-theoretic concepts of entropy and cross-entropy, can be dissected for both sample- and mode-level analysis. Our detailed experimental results demonstrate the sensitivity of our metric components to their respective qualities and reveal undesirable behaviors of other metrics.

A Unifying Information-theoretic Perspective on Evaluating Generative Models

This paper studies a generalized variant of the Colonel Blotto game, referred to as the Colonel Blotto game with costs. Unlike the classic Colonel Blotto game, which imposes the \textit{use-it-or-lose-it} budget assumption, the Colonel Blotto game with costs captures the strategic importance of costs related both to obtaining resources and assigning them across battlefields. We show that every instance of the Colonel Blotto game with costs is strategically equivalent to an instance of the zero-sum Colonel Blotto game with one additional battlefield. This enables the computation of Nash equilibria of the Colonel Blotto game with costs in polynomial time with respect to the game parameters: the number of battlefields plus the number of resources available to the players.

Equilibria of the Colonel Blotto Games with Costs

Knowledge editing aims to update outdated or incorrect knowledge in large language models (LLMs). However, current knowledge editing methods have limited scalability for lifelong editing. This study explores the fundamental reason why knowledge editing fails in lifelong editing. We begin with the closed-form solution derived from linear associative memory, which underpins state-of-the-art knowledge editing methods. We extend the solution from single editing to lifelong editing, and through rigorous mathematical derivation, identify an interference term in the final solution, suggesting that editing knowledge may impact irrelevant knowledge. Further analysis of the interference term reveals a close relationship with superposition between knowledge representations. When knowledge superposition does not exist in language models, the interference term vanishes, allowing for lossless knowledge editing. Experiments across numerous language models reveal that knowledge superposition is universal, exhibiting high kurtosis, zero mean, and heavy-tailed distributions with clear scaling laws. Ultimately, by combining theory and experiments, we demonstrate that knowledge superposition is the fundamental reason for the failure of lifelong editing. Moreover, this is the first study to investigate knowledge editing from the perspective of superposition and provides a comprehensive observation of superposition across numerous real-world language models.

Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models

We consider correlated equilibria in an adversarial environment, where an adversary can compromise the public signal used by the players for choosing their strategies, while players aim at detecting a potential attack as soon as possible to avoid loss of utility. We model the interaction between the adversary and the players as a zero-sum game and we derive the maxmin strategies for both the defender and the attacker using the framework of quickest change detection. We define a class of adversarial strategies that achieve the optimal trade-off between the impact and the detectability of the attack for the adversary and show that a generalized CUSUM scheme is asymptotically optimal for their detection. Our numerical results on the Sioux-Falls benchmark traffic routing game show that the proposed detection scheme can effectively limit the utility loss by a potential adversary.

Quickest Detection of Adversarial Attacks against Correlated Equilibria

The perception system for autonomous driving generally requires to handle multiple diverse sub-tasks. However, current algorithms typically tackle individual sub-tasks separately, which leads to low efficiency when aiming at obtaining full-perception results. Some multi-task learning methods try to unify multiple tasks with one model, but do not solve the conflicts in multi-task learning. In this paper, we propose M3Net, a novel multimodal and multi-task network that simultaneously tackles detection, segmentation, and 3D occu-
pancy prediction for autonomous driving and achieve superior performance than single task model. M3Net takes multimodal data as input and integrates multiple tasks via query-token interactions. To enhance the integration of multi-modal features
for multi-task learning, we first propose the Task-Adaptive Feature Integration (TAFI) module, which enables single-modality features to predict channel-wise attention weights for their high-performing tasks, respectively. Based on integrated features, we develop task-specific query initialization strategies to accommodate the needs of detection/segmentation and 3D occupancy prediction. Leveraging the properly initialized queries, a shared decoder transforms queries and BEV features in a layer-wise manner, facilitating multi-task learning. Furthermore, we propose a Task-oriented Channel Scaling (TCS) module in the decoder to mitigate conflicts between optimizing for different tasks. Additionally, our proposed multi-task querying and TCS module support both Transformer-based decoder and Mamba-based decoder, demonstrating its flexibility to different architectures. M3Net achieves state-of-the-art multi-task learning performance on the nuScenes benchmarks.

M3Net: Multimodal Multi-task Learning for 3D Detection, Segmentation, and Occupancy Prediction in Autonomous Driving

Despite the widespread success of pattern database (PDB) heuristics in classical planning, to date there has been no application of PDBs to planning with numeric variables. In this paper we attempt to close this gap. We address optimal numeric planning involving conditions characterized by linear expressions and actions that modify numeric variables by constant quantities. Building upon prior research, we present an adaptation of PDB heuristics to numeric planning, introducing several approaches to deal with the unbounded nature of numeric variable projections. This approach aims to restrict the initially infinite projections, thereby bounding the number of states and ultimately constraining the resulting PDBs. We show that the PDB heuristics obtained with our approach can provide strong guidance for the search.

PDBs Go Numeric: Pattern-Database Heuristics for Simple Numeric Planning

Imitating how humans move their gaze in a visual scene is a vital research problem for both visual understanding and psychology, kindling crucial applications such as building alive virtual characters. Previous studies aim to predict gaze trajectories when humans are free-viewing an image, searching for required targets, or looking for clues to answer questions in an image. While these tasks focus on visual-centric scenarios, humans move their gaze also along with audio signal inputs in more common scenarios. To fill this gap, we introduce a new task that predicts human gaze trajectories in a visual scene with synchronized audio inputs and provide a new dataset containing 20k gaze points from 8 subjects. To effectively integrate audio information and simulate the dynamic process of human gaze motion, we propose a novel learning framework called EyEar (Eye moving while Ear listening) based on physics-informed dynamics, which considers three key factors to predict gazes: eye inherent motion tendency, vision salient attraction, and audio semantic attraction. We also propose a probability density score to overcome the high individual variability of gaze trajectories, thereby improving the stabilization of optimization and the reliability of the evaluation. Experimental results show that EyEar outperforms all the baselines in the context of all evaluation metrics, thanks to the proposed components in the learning model.

EyEar: Learning Audio Synchronized Human Gaze Trajectory Based on Physics-Informed Dynamics

Object detection in Unmanned Aerial Vehicle (UAV) images has emerged as a focal area of research, which presents two significant challenges: i) objects are typically small and dense within vast images; ii) computational resource constraints render most models unsuitable for real-time deployment. Current real-time object detectors are not optimized for UAV images, and complex methods designed for small object detection often lack real-time capabilities. To address these challenges, we propose a novel detector, RemDet (Reparameter efficient multiplication Detector). Our contributions are as follows: 1) Rethinking the challenges of existing detectors for small and dense UAV images, and proposing information loss as a design guideline for efficient models. 2) We introduce the ChannelC2f module to enhance small object detection performance, demonstrating that high-dimensional representations can effectively mitigate information loss. 3) We design the GatedFFN module to provide not only strong performance but also low latency, effectively addressing the challenges of real-time detection. Our research reveals that GatedFFN, through the use of multiplication, is more cost-effective than feed-forward networks for high-dimensional representation. 4) We propose the CED module, which combines the advantages of ViT and CNN downsampling to effectively reduce information loss. It specifically enhances context information for small and dense objects. Extensive experiments on large UAV datasets, Visdrone and UAVDT, validate the real-time efficiency and superior performance of our methods. On the challenging UAV dataset VisDrone, our methods not only provided state-of-the-art results, improving detection by more than 3.4%, but also achieve 110 FPS on a single 4090.

RemDet: Rethinking Efficient Model Design for UAV Object Detection

With the rise of Artificial Intelligence (AI) systems in society, our children have routine interactions with these technologies. It has become increasingly important for them to understand how these technologies are trained, what their limitations are and how they work. To introduce children to AI and Machine Learning (ML) concepts, recent efforts introduce tools that integrate ML concepts with physical computing and robotics. However, some of these tools cannot be easily integrated into building projects and the high price of robotics kits can be a limiting factor to many schools. We address these limitations by offering a low-cost hardware and software toolkit that we call the Smart Motor to introduce supervised machine learning to elementary school students. Our Smart Motor uses the nearest neighbor algorithm and utilizes visualizations to highlight the underlying decision-making of the model. We conducted a one week long study using Smart Motors with 9- to 12- year old students and measured their learning through observation, questioning and examining what they built. We found that students were able to integrate the Smart Motors into their building projects but some students struggled with understanding how the underlying model functioned. In this paper we discuss these findings and insights for future directions for the Smart Motor.

Smart Motor: A Low-Cost Hardware and Software Toolkit for Introducing Supervised Machine Learning to Elementary School Students

In the fast-growing field of K–12 AI education, there is an
urgent need for accessible, hands-on tools that introduce AI
concepts and workflows to novice learners. In recent years,
a variety of AI education tools have been introduced, ranging from coding environments to physical kits and robots. To
provide an alternative to existing AI education tools, this paper presents a low-cost robotics kit (<50€) designed to teach
modern ML concepts through a no-code approach. The kit is
grounded in maker pedagogy and designed for easy customizability to different materials commonly found in classrooms,
like cardboard, wood, metal, and plastic builder kits without
the need for specialized tools. For programming the robot’s
actions, the kit features an all-in-one development studio that
is compatible with most phone, laptop, and tablet platforms
and can operate with or without an Internet connection, making it applicable to a wide range of educational contexts, including ICT4D.

Premium content

Next from AAAI 2025

A Unifying Information-theoretic Perspective on Evaluating Generative Models

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES