Singapore

The rapid expansion of AI research has intensified the Reviewer Gap, threatening the sustainability of the peer-review system and fueling a cycle of low-quality evaluations. This position paper critiques existing LLM approaches that attempt to automatically generate reviews and argues for a paradigm shift that positions LLMs as tools for assisting and educating human reviewers. We define the core principles of high-quality peer review and propose two complementary systems built on these foundations: (i) an LLM-assisted mentoring system that develops reviewers’ long-term competencies and (ii) an LLM-assisted feedback system that helps reviewers refine the quality of their actual reviews. This human-centered approach aims to strengthen reviewer expertise, address the structural roots of the Reviewer Gap, and support the creation of a more sustainable scholarly ecosystem.

AAAI 2026

Position on LLM-Assisted Peer Review: Addressing Reviewer Gap through Mentoring and Feedback

workshop paper

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.<br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

While AI can accelerate text-intensive research, unsupervised autonomous agents risk compounding errors and cannot be held accountable for the integrity of their findings. Accordingly, we propose a human-supervised, multi-agent model for Human-AI collaboration in science that reconciles the power of Large Language Models (LLMs) with the core principles of transparency and accountability. In our vision, a human researcher supervises a team of specialized AI agents that execute discrete tasks within a rigorously defined scientific methodology. Each step in the process is governed by human expert validation, in order to prevent error propagation and maintain conceptual integrity. We instantiate this model in an end-to-end framework for systematic taxonomy generation by operationalizing the Nickerson et al. (2013) methodology. We provide a proof-of-concept (PoC) implementation and a qualitative case study that validates our framework by re-deriving a recently published taxonomy. Our analysis documents the framework's fidelity to the original method and illustrates how human-gated oversight is crucial for ensuring the scientific rigor of the final taxonomy. To support reproducibility, we release our prompts, agent roles, and workflow design as a generalizable blueprint for structuring human-AI teams. This work argues that method-bounded human-in-command (HIC) supervision offers a principled and practical path to developing reliable and accountable AI research assistants.

Human-in-Command Governance for Multi-Agent Scientific Workflows

Comparing neural network representations is essential for understanding and validating models in scientific applications. Existing methods, however, often provide a limited view. We propose the Triangle of Similarity, a framework that combines three complementary perspectives: static representational similarity (CKA/Procrustes), functional similarity (Linear Mode Connectivity or Predictive Similarity), and sparsity similarity (robustness under pruning). Analyzing a range of CNNs, Vision Transformers, and Vision-Language Models, our initial findings suggest that: (1) architectural family is a primary determinant of representational similarity, forming distinct clusters; (2) CKA self-similarity and task accuracy are strongly correlated during pruning, though accuracy often degrades more sharply; and (3) for some model pairs, pruning appears to regularize representations, exposing a papershared computational core. This framework offers a more holistic approach for assessing whether models have converged on similar internal mechanisms, providing a useful tool for model selection and analysis in scientific research.

The Triangle of Similarity: A Multi-Faceted Framework for Comparing Neural Network Representations

The discovery of next-generation photoinitiators for two-photon polymerization (TPP) is hindered by the absence of large, open datasets containing the quantum-chemical and photophysical properties required to model photodissociation and excited-state behavior. Existing molecular datasets typically provide only basic physicochemical descriptors and therefore cannot support data-driven screening or AI-assisted design of photoinitiators. To address this gap, we introduce QuantumChem-200K, a large-scale dataset of over 200,000 organic molecules annotated with 11 mechanistically relevant properties, including two-photon absorption (TPA) cross sections, TPA spectral ranges, singlet–triplet intersystem crossing (ISC) energies, toxicity and synthetic accessibility scores, hydrophilicity, solubility, boiling point, molecular weight, and aromaticity. These values are computed using a hybrid workflow that integrates density function theory (DFT), semi- empirical excited-state methods, atomistic quantum solvers, and neural-network predictors. Using QuantumChem-200K, we fine-tune the open-source Qwen-2.5-32B large language model to create a chemistry AI assistant capable of forward property prediction from SMILES. Benchmarking on 3000 unseen molecules from VQM24 and ZINC20 demonstrates that domain-specific fine-tuning significantly improves accuracy over GPT-4o, Llama-3.1-70B, and the base Qwen2.5-32B model, particularly for TPA and ISC predictions central to photoinitiator design. QuantumChem-200K and the corresponding AI assistant together provide the first scalable plat- form for high-throughput, LLM-driven photoinitiator screening and accelerated discovery of photo-sensitive materials.

QuantumChem-200K: A Large-Scale Open Organic Molecular Dataset for Quantum-Chemistry Property Screening and Benchmarking

Cell cycle phase identification is essential for understanding cellular proliferation, disease progression, and treatment response in single-cell genomics. While single-cell RNA sequencing (scRNA-seq) enables diverse cell cycle classification strategies, single-cell ATAC sequencing (scATAC-seq) offers complementary epigenetic insights but lacks well-annotated datasets for supervised learning. We present a cross-modal transfer learning framework that leverages paired multiome data to transfer cell cycle knowledge from gene expression to chromatin accessibility domains. Through systematic evaluation of ten single-cell foundation models spanning 10M to 400M parameters (Geneformer, scGPT, scFoundation, UCE, TEDDY), we identify Geneformer-316M as optimal for cross-modal transfer with domain-adversarial neural networks. Our framework integrates class-conditional domain adaptation with diversity-aware training using KL, Jensen–Shannon, CORAL, and MMD divergence to prevent target-domain collapse and stabilize cross-modal learning. Across four divergence measures and eight peak encoder architectures, our models achieve strong cross-dataset performance on SUP-B15 scRNA-seq, and with MMD we obtain robust cross-modal transfer to scATAC-seq, reaching competitive accuracy on REH and generalizing to SUP-B15 despite extreme peak sparsity.

Cross-Modal Cell Cycle Phase Classification via Dual-Encoder Transfer Learning with Foundation Model Embeddings

Causal inference is central to scientific and policy decision-making, yet a major challenge in observational studies is separating true treatment effects from confounding, especially when confounders are high-dimensional or text- and image-based. Despite methodological progress, researchers still struggle with two persistent questions: *which confounders to adjust for* and *which estimators to use*. Here, we test whether large language models (LLMs) can autonomously design and refine causal estimators. We introduce a causal-agent framework with two modes: a zero-shot LLM that proposes estimators from structured prompts, and a self-evolving agent that iteratively improves them using evaluation feedback. Across two benchmarks for heterogeneous treatment effect estimation, the self-evolving agent matched or exceeded human-expert performance, achieving lower RMSE and near-nominal confidence interval coverage, and substantially outperforming zero-shot LLM solutions. Examination of the agent’s updates showed that it progressively learned appropriate specifications: early iterations added regularization and pre-outcome adjustment; mid-stage refinements introduced interactions, weighting, and clustered inference; and later steps converged toward augmented methods that balanced accuracy and calibration. Our results suggest that LLM-based agents can autonomously explore and optimize causal estimators, narrowing the gap between automated and expert-driven analysis. Our next steps are to evaluate generalization to additional causal tasks and to assess whether providing agents with access to advanced causal inference packages can further enhance performance.

Autonomous LLM Agents for Causal Estimation and Model Refinement

Modern scientific AI models excel at pattern prediction but rarely verify physical consistency or refine their hypotheses. This limitation is acute in climate downscaling, where fine-scale fields must respect conservation structure and spatiotemporal dynamics. Physics-Informed Neural Networks (PINNs) embed a partial differential equation (PDE) structure through residual penalties and work well when the governing equations are complete. However, in realistic climate systems with partially known or parameterized operators, enforcing an imperfect PDE can introduce bias and destabilize optimization. Therefore, we introduce a \emph{physics-aware autonomous AI assistant} that treats predictions as hypotheses to be validated and corrected in a closed loop. Rather than relying on a fixed PDE residual, the assistant uses \emph{physical validators}: soft modular diagnostics whose gradients supply adaptive curvature to the learning dynamics. These validators induce strong monotonicity in directions of physical violation and create a dissipative Lyapunov structure, yielding stable, contractive refinement without requiring a fully specified governing equation. Experiments based on our approach on a synthetic controlled 1D diffusion-advection benchmark generated using a stable Forward-Time–Central-Space solver show consistent gains over an architecture-matched PINN baseline, lower bias and variance, smoother error heat maps, slower horizon-wise error growth, improved local curvature recovery, and self-regulated validator weights that decay from $\lambda\approx 5$ to near zero. These results demonstrate that validator-driven refinement is a flexible and stable foundation for interpretable, autonomous scientific AI systems.

Autonomous AI Assistants for Scientific Research: Physics-Aware Foundations and Climate Downscaling Case Study

Despite the increasing application of Artificial Intelligence (AI) in scientific research, the field lacks a unified theoretical framework to systematically guide and understand this transformation, leading to a disconnect between practice and theory. To bridge this gap, this paper proposes a unified, multi-level framework for AI-empowered research. The framework first deconstructs the AI-driven research ecosystem into a three-stage process (pre-paper, corpus-centric, post-paper) and proposes the three evolving human-AI collaboration model (tool, assistant, and participant) that underpin these stages. Besides, this work introduces the Al-empowered research thinking paradigms, which elaborates how AI influences and empowers the core cognitive capacities of researchers (3C: critical, creative, and computational thinking). The framework's validity is substantiated through a two-pronged approach: a theoretical validation of the full lifecycle via the AlphaFold case, and an empirical validation of the framework's core human-AI collaboration models through our Paper2Presentation case study. This work offers a comprehensive roadmap for understanding and building the next generation of human-AI collaborative research platforms.

A Unified Theoretical Framework for AI-Empowered Research: Unifying Ecosystems, Processes, Collaboration and Thinking Paradigms

Many students and early-career researchers lack access to expert research mentorship. We study whether an AI mentor can help undergraduates progress from idea to paper. We built MENTOR, a tool-augmented, stage-aware assistant that integrates literature search, curated guidelines, methodology checks, and memory. We evaluate MENTOR across six writing stages using two LLM-as-a-judge protocols: (i) expert-persona pairwise preferences and (ii) student-persona rubric scores, plus short multi-turn tutoring tasks and evidence/compliance checks. Across 90 single-turn comparisons, LLM judges preferred MENTOR to Claude Sonnet 4.5 in 74.4% (95% CI [64.3, 82.5]) and to GPT-5 in 58.3% (95% CI [47.7, 68.3]). MENTOR's student-persona scores (clarity, actionability, constraint-fit) are consistently higher than both baselines across stages. In multi-turn sessions (five scenarios/agent), MENTOR achieved slightly higher final quality than GPT-5 under our rubric.

Evaluation of LLM Agents for Research Mentorship

Large Language Models (LLMs) are increasingly embedded in academic writing practices. Although numerous studies have explored how researchers employ these tools for scientific writing, their concrete implementation, limitations, and design challenges within the literature review process remain underexplored. In this paper, we report a user study with researchers across multiple disciplines to characterize current practices, benefits, and \textit{pain points} in using LLMs to investigate related work. We identified three recurring gaps: (i) lack of trust in outputs, (ii) persistent verification burden, and (iii) requiring multiple tools. This motivates our proposal of six design goals and a high-level framework that operationalizes them through improved related papers visualization, verification at every step, and human-feedback alignment with generation-guided explanations. Overall, by grounding our work in the practical, day-to-day needs of researchers, we designed a framework that addresses these limitations and models real-world LLM-assisted writing, advancing trust through verifiable actions and fostering practical collaboration between researchers and AI systems.

From Verification Burden to Trusted Collaboration: Design Goals for LLM-Assisted Literature Reviews

Consider a system of multiple physical agents tasked with collaboratively collecting a set of spatially distributed goals while avoiding collisions with the environment and with each other. 
This type of problem, which combines Multi-Agent Path Finding (MAPF) with task allocation, is known as Multi-Agent Combinatorial Path Finding (MCPF). 
Conflict-Based Steiner Search (CBSS) is an optimal algorithm for MCPF, which assumes that each agent has a fixed goal destination. It selects allocations that yield a solution minimizing the sum of costs (SOC), which we denote as CBSS$_{SOC}$. 
However, this objective is problematic in domains such as search and rescue, where timely service of all goals is more critical than minimizing SOC. 
We therefore propose CBSS$_{SST}$, which minimizes the Sum of Service Times (SST) across all goals using a novel mixed-integer linear programming allocation, thereby generalizing MCPF to settings without requiring fixed goal destinations.
Since CBSS assumes perfect execution, we extend it with robust planning to handle stochastic execution delays. 
We propose two variants of CBSS$_{SST}$: Robust CBSS$_{SST}$ with Strict Verifier, which guarantees the desired robustness, and Robust CBSS$_{SST}$ with Anytime Verifier, which addresses planning-time constraints by returning the most robust solution verified within the available time. 
Our experiments on MCPF benchmarks show that Robust CBSS with anytime verifier solves substantially more instances than when using a strict verified within the time limit, while reducing replanning effort and preserving robustness. 
These results demonstrate that RCbss with an anytime verifier provides an effective and practical approach to MCPF under uncertainty.

Premium content

Next from AAAI 2026

Human-in-Command Governance for Multi-Agent Scientific Workflows

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES