Singapore

Large language models dominate current discussions on agentic AI, but their cost and complexity often limit enterprise adoption. Our research explores how small language models can be equipped with function calling, reasoning, and planning capabilities to perform effectively in enterprise workflows. Agentic AI enables systems to reason over complex states, call tools, and plan multi-step actions. I will present methods for aligning small models with these tasks, leveraging synthetic data for adaptation, and evaluating their reliability in real-world automation settings. The goal is to show how efficient models can deliver trustworthy, enterprise-grade agentic AI solutions. 
While our design principles are broadly applicable, I will focus on IT automation—drawing on IT Bench from IBM, BFCL v4, and other agentic benchmarks. IT automation is critical because enterprises depend on reliable, scalable, and secure IT services, and maintaining this reliability requires continuous monitoring, rapid incident response, and efficient remediation.

AAAI 2026

Small Language Models for Enterprise Agentic Workflows

gen ai

agents

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.<br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

invited talk

Educational question generation (EQG) is a crucial
component of intelligent educational systems, significantly
aiding self-assessment, active learning, and personalized
education. While EQG systems have emerged, existing
datasets typically rely on predefined, carefully edited
texts, failing to represent real-world classroom content,
including lecture speech with a set of complementary
slides. To bridge this gap, we collect a dataset of
educational questions based on lectures from real-world
classrooms. On this realistic dataset, we find that current
methods for EQG struggle with accurately generating
questions from educational videos, particularly in aligning
with specific timestamps and target answers. Common
challenges include selecting informative contexts from
extensive transcripts and ensuring generated questions
meaningfully incorporate the target answer. To address the
challenges, we introduce a novel framework utilizing large
language models (LLMs) for dynamically selecting and
rewriting contexts based on target timestamps and answers
in lecture videos. First, our framework selects contexts
from both lecture transcripts and video keyframes based on
answer relevance and temporal proximity. Then, we integrate
the contexts selected from both modalities and rewrite them
into answer-containing knowledge statements, to enhance the
logical connection between the contexts and the desired
answer. This approach significantly improves the quality
and relevance of the generated questions.

Context Selection and Rewriting for Video-based Educational Question Generation

Educational assessment requires understanding student
problem-solving processes, not just final answers. Current
AI-driven analytics focus on static outcomes, missing
valuable insights from temporal dynamics. We present
Explain-from-Stroke, a practical framework that captures
invisible learning processes by integrating handwriting
dynamics with vision-language models. Our approach extracts
temporal features—writing speed, pauses, and
revisions—providing supplementary context for generating
meaningful insights into hidden aspects of student
reasoning. Deployed with real classroom data from a
Japanese secondary school, our system demonstrates 18.2\%
improvement in cognitive depth analysis over static
approaches. This work provides educators with accessible
process-oriented analysis that reveals invisible learning
processes using standard tablet technology.

Explain-from-Stroke: Capturing Invisible Learning Processes Through Handwriting Dynamics Analysis

As generative AI rapidly enters higher education, its
cognitive, motivational, and social impacts across various
disciplines remain underexplored. This qualitative study
investigates the impact of disciplinary epistemologies and
individual digital literacy on AI-assisted academic English
reading among Chinese EFL undergraduates. Guided by
Cognitive Load Theory and Self-Determination Theory, we
studied 45 students at a Hong Kong university across
hard/pure and soft/applied fields. Open-ended
questionnaires and 32 interviews were analyzed via hybrid
thematic analysis, with intercoder agreement above 85% and
member-checking. Participants in soft/applied fields more
often reported AI reducing extraneous load and supporting
deeper semantic elaboration, whereas reports in hard/pure
fields more frequently described surface-level support
(e.g., glossing terminology). Excessive reliance was
associated with cognitive offloading and an illusory sense
of mastery, shaped by digital literacy and metacognitive
awareness. Socially, AI sometimes displaced routine
exchanges but, when integrated into group contexts,
facilitated higher-order collaboration. The study
elaborates applications of CLT and SDT by showing how
disciplinary and individual factors shape AI’s cognitive
and motivational roles. Practically, it proposes
discipline-sensitive design principles, metacognitive
prompts, clear usage boundaries, and interaction-focused
affordances, pointing to deployable interventions (e.g.,
self-check tasks, staged AI fade-out scaffolds, reflective
prompts). Ethical approval and consent were obtained.

Generative AI as a Cognitive Co-Participant: Disciplinary Modulation of EFL Academic Reading Load and Motivation

In computer-supported collaborative learning environments,
analyzing student dialogue is essential for understanding
collaborative problem-solving behaviors and supporting
effective learning. Prior work often treats all dialogue
interactions uniformly, failing to capture how specific
dialogue interaction differentially impact learning
experiences and outcomes. To address this limitation, we
introduce a dialogue-based learning analytics framework
that integrates weighted temporal clustering of dialogue
with large language model-based interpretation. Our
framework identifies student interaction patterns most
predictive of group learning gains and uses these insights
to enable early prediction of learning outcomes and
generate pedagogically meaningful interpretation. We
evaluate our framework on collaborative dialogue from
middle school students engaged in a collaborative
game-based learning environment. Our results show that our
framework achieves 83.1\% accuracy in learning outcome
prediction. In addition, expert evaluations and case
studies demonstrate that the identified weighted dialogue
patterns reflect key collaborative problem-solving
behaviors recognized as important in collaborative
learning. By surfacing high-impact interaction patterns and
enabling prioritized interpretation generation, our
framework provides a promising approach for accurately
analyzing students’ collaborative dialogue.

A Dialogue-Based Learning Analytics Framework for Collaborative Game-Based Learning

Automated scoring of written constructed responses
typically relies on separate models per task, straining
computational resources, storage, and maintenance in
real-world education settings. We propose
\textsc{UniMoE-Guided}, a knowledge-distilled multi-task
Mixture-of-Experts (MoE) approach that transfers expertise
from multiple task-specific large models (teachers) into a
single compact, deployable model (student). The student
combines (i) a shared encoder for cross-task
representations, (ii) a gated MoE block that balances
shared and task-specific processing, and (iii) lightweight
task heads. Trained with both ground-truth labels and
teacher guidance, the student matches strong task-specific
models while being far more efficient to train, store, and
deploy. Beyond efficiency, the MoE layer improves transfer
and generalization: experts develop reusable skills that
boost cross-task performance and enable rapid adaptation to
new tasks with minimal additions and tuning. On nine
NGSS-aligned science-reasoning tasks (seven for
training/evaluation and two held out for adaptation),
\textsc{UniMoE-Guided} attains performance comparable to
per-task models while using ~6x less storage than
maintaining separate students, and 87x less than the
20B-parameter teacher. The method offers a practical path
toward scalable, reliable, and resource-efficient automated
scoring for classroom and large-scale assessment systems.

Generalizable and Efficient Automated Scoring with a Knowledge-Distilled Multi-Task Mixture-of-Experts

Theories of embodied learning emphasize that learning
processes are grounded in bodily actions and interactions
with the environment, suggesting that movements play a
fundamental role in problem solving, decision making, and
learning. This perspective holds particular relevance for
making-based learning settings, where patterns of movement
and spatial engagement can reveal strategic expertise.
Prior research has examined distinctions between students
who learned and did not learn, but manual coding of actions
presents scalability and real-time application challenges.
To address this gap, we develop a computer vision–based
analysis pipeline for automated detection and
characterization of hand movements during complex assembly
tasks. We apply this approach to video data of students
engaged in the assembly of a differential gearbox,
quantifying metrics such as amount and speed of movement.
Results indicate that learners show fewer right-hand
movements than novices and exhibit reduced movement speed,
with a progressive decline in speed as the task unfolds.
Non-learners, by contrast, display more uneven hand
movement speed. These findings highlight measurable
differences in actions of learners and non-learners, and
therefore have implications for learning support.
Specifically, the ability to computationally distinguish
movement profiles can inform the design of adaptive
learning interventions, providing real-time performance
assessment and targeted feedback for making-based learning.

Thinking Through the Hands: An Exploratory Study of Hand Movements to Assess Students Problem-Solving in Mechanistic Reasoning Tasks

In recent years, AI agents centered on generative AI have been rapidly gaining traction. The expansion of these AI agents goes beyond mere content generation - by automating task planning, external tool use, data collection, and result integration, they are extending AI utilization into areas of decision-making and creative work that were once the domain of humans. As a result, a more flexible and autonomous form of human–AI collaboration is emerging across diverse business and social contexts. 
At the same time, despite the growing scope of AI agent applications, the standards for evaluating their validity and reliability are not well established yet. In enterprise environments in particular, AI agents make decisions and take actions while collaborating with multiple business systems and human users, making evaluation especially challenging. There is a growing need for quantitative assessments from multiple perspectives - including agent collaboration, task success rate, and the agent’s ability to improve through self-learning and memory. 
This presentation introduces the current status and challenges of applying AI agents in enterprise settings, using Fujitsu Kozuchi AI Agent as an example. In particular, it discusses the practical difficulties of developing “domain-specific agents” revealed through real-world applications such as field work support and data analysis tasks, and outlines benchmark approaches that enable their quantitative evaluation.

Agentic AI Benchmark in Enterprise

Benchmarks are essential for quantitatively tracking progress in AI. As AI agents become increasingly capable, researchers and practitioners have introduced agentic benchmarks to evaluate agents on complex, real-world tasks. These benchmarks typically measure agent capabilities by evaluating task outcomes via specific reward designs. However, we show that many agentic benchmarks have issues in task setup or reward design. For example, SWE-bench Verified uses insufficient test cases, while TAU-bench counts empty responses as successful. Such issues can lead to under- or overestimation of agents' performance by up to 100% in relative terms. To make agentic evaluation rigorous, we introduce the Agentic Benchmark Checklist (ABC), a set of guidelines that we synthesized from our benchmark-building experience, a survey of best practices, and previously reported issues. When applied to CVE-Bench, a benchmark with a particularly complex evaluation design, ABC reduces the performance overestimation by 33%.

Establishing Best Practices for Building Rigorous Agentic Benchmarks

Amazon's catalog contains billions of ASINs across diverse product categories and global marketplaces, creating a large repository of structured and unstructured product data. Traditional approaches to product intelligence extraction used task-specific modeling of catalog attributes, often missing the deeper multi-modal semantic relationships within this ecosystem. Multi-modal large language models now enable reasoning about and querying product attributes at higher levels of abstraction.
This talk focuses on an agentic application developed by Amazon's Climate Pledge Friendly team for sustainability criteria validation at enterprise scale. We examine our system architecture, key learnings from production deployment, evaluation strategies, and Human-in-the-Loop workflows. Additionally, we introduce a visual prompting methodology for multi-modal LLMs that emerged from this research, accelerating the development of visual inference applications for retail product intelligence.

Agents in Retail

Membership inference attacks (MIAs) test whether a data point was part of a model's training set, posing serious privacy risks. Existing methods often depend on shadow models or heavy query access, which limits their practicality. We propose GP-MIA, an efficient and interpretable approach based on Gaussian process (GP) meta-modeling. Using post-hoc metrics such as accuracy, entropy, dataset statistics, and optional sensitivity features (e.g. gradients, NTK measures) from a single trained model, GP-MIA trains a GP classifier to distinguish members from non-members while providing calibrated uncertainty estimates. Experiments on synthetic data, real-world fraud detection data, CIFAR-10, and WikiText-2 show that GP-MIA achieves high accuracy and generalizability, offering a practical alternative to existing MIAs.

Downloads

Next from AAAI 2026

Context Selection and Rewriting for Video-based Educational Question Generation

Similar lecture

Enabling Reliable Enterprise Agentic AI: A Case Study on Domain-Specific Embeddings and Benchmarking in Offshore Energy

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES