Singapore

We present CAIA, a benchmark exposing a critical blind spot in AI evaluation: the inability of state-of-the-art models to operate in adversarial, high-stakes environments where misinformation is weaponized and errors are irreversible. While existing benchmarks measure task completion in controlled settings, real-world deployment demands resilience against active deception. Using cryptocurrency markets as a natural laboratory, where $30 billion was lost to exploits in 2024, we evaluate 17 leading models on 178 time-anchored tasks requiring agents to distinguish truth from manipulation, navigate fragmented information landscapes, and make irreversible financial decisions under adversarial pressure. Our results reveal a fundamental capability gap: without tools, even frontier models achieve only 12-28% accuracy on tasks junior analysts routinely handle. Tool augmentation improves performance but plateaus at 67.4% (GPT-5) versus 80% human baseline, despite unlimited access to professional resources. Most critically, we uncover a systematic tool selection catastrophe: models preferentially choose unreliable web search (55.5% of invocations) over authoritative blockchain data, falling for SEO-optimized misinformation and social media manipulation. This behavior persists even when correct answers are directly accessible through specialized tools, suggesting foundational limitations rather than knowledge gaps. The implications extend beyond cryptocurrency to any domain where adversaries actively exploit AI weaknesses, e.g. cybersecurity, content moderation, etc. Our finding that Pass@k metrics mask dangerous trial-and-error behavior challenges fundamental assumptions about autonomous deployment. We release CAIA with contamination controls and continuous updates, establishing adversarial robustness as a necessary condition for trustworthy AI autonomy. The benchmark reveals that current models, despite impressive reasoning scores, remain fundamentally unprepared for environments where intelligence must survive active opposition.

AAAI 2026

When Hallucination Costs Millions: Benchmarking AI Agents in High-Stakes Adversarial Financial Markets

evaluation and analysis

adversarial learning & robustness

agents

workshop paper

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.<br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

Large Language Models (LLMs) have shown remarkable capabilities in document processing, but their inability to provide visual grounding without OCR dependencies poses significant challenges in business-critical applications. Current solutions either require model fine-tuning or rely on external OCR services, introducing additional costs, latency, and limitations in handling derived information. This paper presents ViG-LLM, a novel framework that enables closed-box LLMs to generate localization information through a multi-agent system combining U-Net-based layout deconstruction with viewport identification tasks. Evaluated on the FATURA and CORD dataset, our framework achieves perfect accuracy over spatial reasoning tuned LLM like Amazon Nova Pro, while demonstrating superior template-specific consistency. The framework maintains robust performance across LLM architectures while reducing operational costs by 60% compared to Textract-based solutions. In real-world document processing applications, the framework helps retain the high reasoning capabilities of the system in document information extraction tasks while improving explainability, reliability and human interaction for information verification. Through human-in-the-loop learning and closed-box prompt alignment techniques, ViG-LLM provides a robust, adaptable solution for visual grounding tasks in document processing workflows.

ViG-LLM: Enhancing Visual Grounding Capabilities in Closed-Box LLMs for Document Information Extraction without OCR Dependencies

Making use of swarm methods in financial market modeling of liquidity, and techniques from financial analysis in swarm analysis, holds the potential to advance both research areas. In swarm research, the use of game theory methods holds the promise of explaining observed phenomena of collective utility adherence with rational self-interested swarm participants. In financial markets, a better understanding of how independent financial agents may self-organize for the betterment and stability of the marketplace would be a boon for market design researchers. This paper unifies Liquidity Games, where trader payoffs depend on aggregate liquidity within a trade, with Rational Swarms, where decentralized agents use difference rewards to align self-interested learning with global objectives. We offer a theoretical frameworks where we define a swarm of traders whose collective objective is market liquidity provision while maintaining agent independence. Using difference rewards within a Markov team games framework, we show that individual liquidity-maximizing behaviors contribute to overall market liquidity without requiring coordination or collusion. This Financial Swarm model provides a framework for modeling rational, independent agents where they achieve both individual profitability and collective market efficiency in bilateral asset markets.

Multiagent Reinforcement Learning for Liquidity in Bond Markets

Terminal-based coding agents have emerged as powerful reasoning systems, achieving strong performance through iterative code execution and self-verification loops. But can these same capabilities, designed for software engineering, transfer effectively to financial reasoning tasks that demand numerical precision over unstructured data? We present a systematic evaluation of three state-of-the-art coding agents (Codex CLI, Claude Code, Gemini CLI) on four diverse financial QA benchmarks spanning numerical reasoning (FinQA, TAT-QA), document grounding (DocFinQA) and mathematical reasoning (DocMath-Eval). Under oracle context evaluation that isolates reasoning from retrieval, agentic execution consistently outperforms direct inference, with improvements ranging from +2.4% on short tabular tasks to +9.77% on long document reasoning. Within-model comparisons reveal that execution loops provide selective advantages on specific task types, particularly those requiring complex multi-step calculations. Our findings demonstrate that general-purpose coding agents can serve as effective financial reasoning systems without domain-specific fine-tuning, although challenges remain in handling extended symbolic contexts.

Code Execution Improves Financial Reasoning: Benchmarking Modern Agentic Systems on Financial QA

This paper introduces a novel Multi-Agent framework tailored for quantitative tasks routinely performed by portfolio managers and researchers within the asset management industry. Though machine learning and artificial intelligence have been used widely within quantitative finance, there is a scarcity of AI frameworks capable of autonomously performing complex tasks and quantitative analysis especially on structured data. Our framework addresses this gap and facilitates mathematical modeling and data analytics by dynamically generating executable code. The framework comprises of an innovative multi-agent architecture which includes specialized components for reflection, summarization, and financial expertise which coordinate to enhance problem solving abilities. We empirically evaluate the framework on a comprehensive dataset comprising of portfolio management-specific tasks, addressing a critical gap in current research. Our findings demonstrate that the proposed Multi-Agent framework vastly outperforms Single-Agent frameworks, demonstrating its practical utility across various task categories. By using dynamic code generation with the agent’s multi-step reasoning capabilities, we broaden the range of tasks that can be successfully addressed.

A Multi-Agent Framework for Portfolio Management Analytics in Quantitative Finance

Credit risk assessment is essential in the financial sector, but has traditionally depended on costly feature-based models that often fail to utilize all available information in raw credit records. This paper introduces LendNova, the first practical automated end-to-end pipeline for credit risk assessment, designed to utilize all available information in raw credit records by leveraging advanced NLP techniques and language models. LendNova transforms risk modeling by operating directly on raw, jargon-heavy credit bureau text using a language model that learns task-relevant representations without manual feature engineering. By automatically capturing patterns and risk signals embedded in the text, it replaces manual preprocessing steps, reducing costs and improving scalability. Evaluation on real-world data further demonstrates its strong potential in accurate and efficient risk assessment. LendNova establishes a baseline for intelligent credit risk agents, demonstrating the feasibility of language models in this domain. It lays the groundwork for future research toward foundation systems that enable more accurate, adaptable, and automated financial decision-making.

LendNova: Towards Automated Credit Risk Assessment with Language Models

We introduce a generic human-in-the-loop multi-agent examiner framework and demonstrate its application for evaluating small language models (SLMs) in the financial domain. While large language models have shown strong performance in finance, their computational cost motivates the exploration of SLMs as a viable alternative. However, assessing SLMs across a range of financial tasks remains challenging due to the specificity and scarcity of existing benchmarks. To address this gap, we extend the LLM-as-examiner paradigm into a modular system that decomposes evaluation into task-specific agents and estimates the performance of the SLMs relative to large strong models. We apply the proposed approach to estimate the performance of models from the PHI-4 and SMOLLM families on three problems. Extensive experiments show that our method generates more varied and complex evaluation questions than standard benchmarks, revealing nuanced insights into model strengths and weaknesses. The results highlight the value of scalable, agent-assisted evaluation for enhancing the understanding and deployment of small language models in financial domain, while also identifying certain limitations and challenges of our approach.

Multi-Agent Examiner: Evaluating Small Language Models on Financial Tasks

Analyzing financial documents such as 10-K filings, tabular disclosures, and macroeconomic reports demands expert reasoning and extensive time. However, existing Retrieval-Augmented Generation systems often struggle to process hybrid text–table structures or massive scale of financial documents. To address these challenges, we propose Hierarchical Reranker, a RAG framework designed to improve retrieval performance and generative reliability across large-scale financial datasets. The system integrates three key innovations: Pre-Retrieval Optimization, enhancing query clarity and search efficiency through normalization, keyword expansion, and table transformation; Hierarchical Reranker Architecture, improving retrieval precision through a two-stage ranking mechanism; and Long-Context Management, preserving reasoning accuracy through adaptive input partitioning and fusion under extensive contexts. Across multiple benchmarks, including FinQA, FinanceBench, and ConvFinQA, the proposed system achieved an NDCG@20 score of 0.7918 and demonstrated superior factual consistency. Its robustness was further validated by achieving second place in the ACM-ICAIF ’24 FinanceRAG Challenge. This work presents a deployable, domain-optimized RAG pipeline that enhances both the accuracy and scalability of financial reasoning, paving the way for automated audit reporting and quantitative investment analysis.

Hierarchical Reranking for Scalable Financial RAG System

This study investigated LLM–based automation for analyzing non-financial data in corporate credit evaluation. Two systems were developed and compared: a Single-Agent System (SAS), in which one LLM agent infers favorable and adverse repayment signals, and a Popperian Multi-Agent Debate System (PMADS), which structures the dual-perspective analysis as adversarial argumentation under the Karl Popper Debate protocol. Evaluation addressed three fronts: (i) work productivity (task completion time) compared with human experts; (ii) perceived report quality and usability, rated by credit risk professionals for system-generated reports; and (iii) reasoning characteristics quantified via reasoning-tree analysis (breadth, depth, and Reasoning Elaboration Index). Both systems drastically reduced task completion time relative to human experts (baseline ≈ 1,900 s/case; SAS ≈ 11.6 s; PMADS ≈ 92.0 s). Professionals rated SAS re-ports as adequate, while PMADS reports exceeded neutral benchmarks and scored significantly higher in explanatory adequacy, practical applicability, and usability. Reasoning-tree analysis showed PMADS produced deeper, more elaborated structures, whereas SAS yielded single-layered trees. These findings suggest that structured multi-agent debate enhances analytical rigor and perceived usefulness, though at the cost of longer computation time. Overall, the results demonstrate that reasoning-centered automation represents a promising approach for developing useful AI systems in decision-critical financial contexts.

Structured Debate Improves Corporate Credit Reasoning in Financial AI

Despite accounting for 96.1% of all businesses in Malaysia, access to financing remains one of the most persistent challenges faced by Micro, Small, and Medium Enterprises (MSMEs). Newly established or young businesses are often excluded from formal credit markets as traditional underwriting approaches rely heavily on credit bureau data. This study investigates the potential of bank statement data as an alternative data source for credit assessment to promote financial inclusion in emerging markets. Firstly, we propose a cash flow-based underwriting pipeline where we utilise bank statement data for end-to-end data extraction and machine learning credit scoring. Secondly, we introduce a novel dataset of 611 loan applicants from a Malaysian lending institution. Thirdly, we develop and evaluate credit scoring models based on application information and bank transaction-derived features. Empirical results show that the use of such data boosts the performance of all models on our dataset, which can improve credit scoring for new-to-lending MSMEs. Lastly, we intend to release the anonymised bank transaction dataset to facilitate further research on MSMEs financial inclusion within Malaysia's emerging economy.

Cash Flow Underwriting with Bank Transaction Data: Advancing MSME Financial Inclusion in Malaysia

Large Language Model (LLM)-based agent systems have shown strong potential for autonomous financial analysis. However, most multi-agent trading systems still fail to account for the diverse perspectives that drive real-market price movements, limiting their ability to capture the step-by-step reasoning and consensus-building seen in professional investment teams. We introduce AGORA-F: an AGentic Orchestrated Reasoning Architecture for Finance-that treats financial decision-making as a process in which agents gradually reach a shared conclusion through discussion based optimization. AGORA-F structures its workflow into specialized teams: Query Understanding Agents interpret user intent and derive data requirements; Analytical Agents collect and normalize real-time market data, financial statements, and sentiment signals to prepare detailed briefs; and Financial Analysis Agents evaluate the stock from multiple viewpoints, including return potential, risk exposure, macroeconomic context, and competitive positioning. A central Trading Agent coordinates iterative discussions among the financial analysts, refining information requests and aligning perspectives via an agentic, prompt-based gradient mechanism. When the discussion converges or a preset iteration budget is reached-a Manager Agent synthesizes the collective rationale and issues a final decision-buy, hold, or sell-with explicit evidence trails. Empirical results indicate that AGORA-F yields more consistent and explainable trading outcomes than comparable baselines, marking a step toward human-team-like collaborative reasoning in autonomous financial systems. Our code will be made publicly available.

Downloads

Next from AAAI 2026

ViG-LLM: Enhancing Visual Grounding Capabilities in Closed-Box LLMs for Document Information Extraction without OCR Dependencies

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES