United States

Industrial maintenance platforms contain rich but fragmented evidence, including free-text work orders, heterogeneous operational sensors or indicators, and structured failure knowledge. These sources are often analyzed in isolation, producing alerts or forecasts that do not support conditional decision-making: given this asset history and behavior, what is happening and what action is warranted?


We present Condition Insight Agent, a deployed decision-support framework that integrates maintenance language, behavioral abstractions of operational data, and engineering failure semantics to produce evidence-grounded explanations and advisory actions. The system constrains reasoning through deterministic evidence construction and structured failure knowledge, and applies a rule-based verification loop to suppress unsupported conclusions.


Case studies from production CMMS deployments show that this verification-first design operates reliably under heterogeneous and incomplete data while preserving human oversight. Our results demonstrate how constrained LLM-based reasoning can function as a governed decision-support layer for industrial maintenance.

ACL 2026

Evidence‑Driven Reasoning for Industrial Maintenance Using Heterogeneous Data

### Welcome to the 10th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.
#### On-Demand Access

* This is an on-demand site and all the recorded content from the workshop is available here. 
* Access to the recordings is exclusively available to registered participants.
* Each registrant has received an email notification, with instructions on how to access the content.

[![](https://assets.underline.io/markdown_image/1/image/60cd12aa376c191d3467ea2ee60944ea.png)](https://naml2026.org/)

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations

This is an on-demand site. All recorded content from NAML 2026 is available here and access to it is exclusively available to registered participants.

**Please log in using the email address you registered with.** If you have any issues logging in, please contact us at <naml2026@underline.io>.

Let’s celebrate the past decade by investing in the next. 🚀

Please log in to explore this event.

This is an on-demand site. All recorded content from NAML 2026 is available here and access to it is exclusively available to registered participants.

**Explore the latest advancements in machine learning tailored to Naval applications 🔎**

With on-demand access to NAML 2026, you can watch all lectures & browse poster presentations at your convenience, while also engaging directly with presenters via the Underline platform - ask questions, exchange ideas, and stay connected.

**If you still haven’t registered, you can do so now.**
👇
**Take advantage of a special one-time offer: full access for just $99, with complimentary registration for students, government, and military employees.**

**Get access here: https://naml2026-ondemand.paperform.co/**

Please check your spam folder, or contact our support team at **<naml2026@underline.io>** if you registered, but still haven't received an access email.

Let’s celebrate the past decade by investing in the next. 🚀

Please register!

NAML 2026

The 10th Annual Workshop on Naval Applications of Machine Learning (NAML 2026) was held in San Diego, California, March 2-5, 2026. NAML is a forum for interdisciplinary experts from government, industry, and academia to discuss results of ongoing research on theory and applications of machine learning to address some of the most challenging problems faced by our naval forces. 



poster

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. 
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) 
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

**Online Registration Form**: https://acl.swoogo.com/acl2026

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

The Model Context Protocol (MCP) has rapidly emerged as the dominant standard for connecting large language models to external tools, databases, and services. Yet this convenience introduces a fundamentally new attack surface that existing LLM safety measures fail to address: adversaries can now compromise AI agents not through the user prompt, but through the tools the agent trusts. We present ShieldMCP, a runtime security framework grounded in two contributions. First, we introduce a structured threat taxonomy derived from analysis of 80+ attack techniques catalogued under the SAFE-MCP initiative within the Linux Foundation's OpenSSF, spanning 14 tactical categories adapted from the MITRE ATT&CK methodology. Second, we describe an interception architecture that performs real-time validation of MCP tool calls and responses, combining structural analysis with semantic intent verification to detect tool poisoning, indirect prompt injection through tool outputs, and supply chain manipulation. In red-team evaluation across five popular LLM backends, ShieldMCP reduces attack success rates from 74% to under 9% for tool poisoning and from 47% to under 6% for indirect prompt injection via tool responses, while adding fewer than 120ms of median latency per tool call. We discuss deployment considerations, the tension between security and agent utility, and lessons applicable to any organization integrating MCP into production workflows. Our framework is categorized as an Emerging system intended for real-world deployment.

Securing the Tool Layer: A Threat Taxonomy and Runtime Defense Framework for Model Context Protocol Deployments

Modern data lakes contain vast and heterogeneous document collections, making Information Extraction (IE) a persistent and nontrivial challenge. Traditional IE systems are often rigid - i.e. domain-specific, require extensive supervision, or are limited to predefined entity types and data structures; LLM-based approaches are more flexible, but suffer from hallucinations, non-determinism, high computational costs - drawbacks inherent to all LLMs. To overcome these limitations, we introduce OSyTa, a novel scalable LLM-based IE system that processes documents of varying complexity, extracts relational tables, and can generate complex tables with Bi-dimensional metadata, enabling robust document-level IE.
We evaluated OSyTa on real user extraction tasks and \textcolor{blue}{achieved high F1 score (92.1\%) across three publicly available}, large-scale corpora from two distinct domains (medical and legal) each containing numerous documents.

OSyTa: Optimizer-based System for Table Generation from Documents

Large language models (LLMs) face a fundamental trade-off between factual consistency and reasoning diversity. Deterministic decoding strategies prioritize reliability but may fail to explore alternative solution paths, while high-temperature sampling increases exploration at the cost of factual accuracy. We present FROST (Factual Reasoning via Optimized Stochastic Trajectories), an inference framework that systematically balances exploration and exploitation for cost-efficient reasoning in production environments. FROST combines deterministic inference from large models with targeted stochastic sampling from smaller models, followed by multi-criteria validation to select factually grounded outputs. The framework operates entirely at inference time, requiring no additional training or fine-tuning. Across HotpotQA, CommonsenseQA, and MMLU, FROST achieves 3--5\% accuracy improvements over standard chain-of-thought prompting. Crucially, it maintains high cost-efficiency relative to other robust inference methods, achieving a 28\% cost reduction compared to Self-Consistency ensembles through strategic delegation to smaller models. Validation analysis demonstrates that FROST reduces unsupported outputs by 40\% relative to baseline methods. Task-stratified evaluation reveals that exploration benefits scale with problem ambiguity, with alternative reasoning paths improving accuracy in 23--34\% of multi-hop questions. These results establish FROST as a practical approach for deploying reliable and efficient reasoning systems in production environments.

FROST: Factual Reasoning via Optimized Stochastic Trajectories in Large Language Models during Inference

Extracting structured data from unstructured text using large language models (LLMs) becomes challenging when the target schemas are large and complex. In such cases, including the full schema in the prompt increases cost and latency, risks lost-in-the-middle performance degradation, and can exceed context length limits. We propose SchemaRAG, a retrieval-augmented generation (RAG) framework that dynamically prunes the output schema space for schema-conditioned information extraction tasks by leveraging schema metadata and few-shot examples (when available). We evaluate SchemaRAG on real-world healthcare and e-commerce datasets. Our results show that SchemaRAG can achieve up to an 8.8% increase in micro-F1, a 47% reduction in latency, and a 48% reduction in token costs, demonstrating its practicality for large-schema extraction.

SchemaRAG: Dynamic Large Schema Reduction for LLM-driven Structured Information Extraction

Enterprise IT support interactions are fundamentally diagnostic: effective resolution requires iterative evidence gathering from ambiguous user reports to identify an underlying root cause. While retrieval-augmented generation (RAG) provides grounding through historical cases, standard multi-turn RAG systems lack explicit diagnostic state and therefore struggle to accumulate evidence and resolve competing hypotheses across turns.


We introduce DQA, a diagnostic question-answering framework that maintains persistent diagnostic state and aggregates retrieved cases at the level of root causes rather than individual documents. DQA combines conversational query rewriting, retrieval aggregation, and state-conditioned response generation to support systematic troubleshooting under enterprise latency and context constraints.


We evaluate DQA on 150 anonymized enterprise IT support scenarios using a replay-based protocol. Averaged over three independent runs, DQA achieves a 78.7\% success rate under a trajectory-level success criterion, compared to 41.3\% for a multi-turn RAG baseline, while reducing average turns from 8.4 to 3.9. This improvement reflects the benefit of explicitly representing competing explanations and aggregating evidence across turns in unscripted troubleshooting.

DQA: Diagnostic Question Answering for IT Support

We introduce Agentic Economic Modeling (AEM), a framework that aligns synthetic LLM choices with small-sample human evidence for reliable econometric inference. AEM first generates task-conditioned synthetic choices via LLMs, then learns a bias-correction mapping from task features and raw LLM choices to human-aligned choices, upon which standard econometric estimators perform inference to recover demand elasticities and treatment effects. We validate AEM in two experiments. In a large scale conjoint study with millions of observations, using only 10\% of the original data to fit the correction model lowers the error of the demand-parameter estimates, while uncorrected LLM choices even increase the errors. In a regional field experiment, a mixture model calibrated on 10\% of geographic regions estimates an out-of-domain treatment effect of -65$\pm$10 bps, closely matching the full human experiment (-60$\pm$8 bps). Under time-wise extrapolation, training with only day-one human data yields -24 bps (95\% CI: [-26, -22], p$<$1e-5), improving over the human-only day-one baseline (-17 bps, 95\% CI: [-43, +9], p=0.2049). These results demonstrate AEM's potential to improve RCT efficiency and establish a foundation method for LLM-based counterfactual generation.

Agentic Economic Modeling

Reinforcement learning (RL) has become a cornerstone of the post-training pipeline for large language models (LLMs), enabling capabilities such as complex reasoning and tool use. However, standard RL approaches face significant challenges due to reward sparsity. Moreover, LLMs typically exhibit mode-seeking behavior, concentrating probability mass on high-likelihood regions. This lack of diversity biases the model toward premature exploitation, hindering the exploration necessary for optimal learning. To address this, we propose VEG (verbal $\epsilon$-greedy), a novel framework that leverages external feedback as a dynamic control variable to explicitly balance exploration and exploitation within the semantic space. This method not only supplements sparse final rewards with intermediate signals but also enforces sustained exploration throughout the training process. Experiments on Tau Bench and SearchQA demonstrate that our method achieves superior accuracy compared to standard RL baselines. Notably, the trained policy eventually outperforms the external feedback model itself, demonstrating that VEG enables the agent to effectively filter and improve upon the guidance it receives.

VEG: Verbal $\epsilon$-greedy for Semantic Exploration in Multi-Turn RL Agents

Traditional phishing website detection relies on static heuristics or reference lists, which lag behind rapidly evolving attacks. 
While recent systems incorporate large language models (LLMs), they are still prompt-based, deterministic pipelines that underutilize reasoning capability.
We present MemoPhishAgent (MPA), a memory-augmented multi-modal LLM agent that dynamically orchestrates phishing-specific tools and leverages episodic memories of past reasoning trajectories to guide decisions on recurring and novel threats.
On two public datasets, MPA outperforms three state-of-the-art (SOTA) baselines, improving recall by 13.6\%.
To better reflect realistic, user-facing phishing detection performance, we further evaluate MPA on a benchmark of real-world suspicious URLs actively crawled from five social media platforms, where it improves recall by 20\%.
Detailed analysis shows episodic memory contributes up to 27\% recall gain without introducing additional computational overhead.
The ablation study confirms the necessity of the agent-based approach compared to prompt-based baselines and validates the effectiveness of our tool design.
Finally, MPA is deployed in production, processing $\sim60K$ targeted high-risk URLs weekly, and achieving 91.44\% recall, providing proactive protection for millions of customers.
Together, our results show that combining multi-modal reasoning with episodic memory yields robust, adaptable phishing detection in realistic user-exposure settings.

MemoPhishAgent: Memory-Augmented Multi-Modal LLM Agent for Phishing URL Detection

Interactive large language model (LLM) agents operating via multi-turn dialogue and multi-step tool calling are increasingly used in production. Benchmarks for these agents must both reliably compare models and yield on-policy training data. Prior agentic benchmarks (e.g., tau-bench, tau^2-bench, AppWorld) rely on fully deterministic backends, which are costly to build and iterate. We propose Proxy State-Based Evaluation, an LLM-driven simulation framework that preserves final state-based evaluation without a deterministic database. Specifically, a scenario specifies the user goal, user/system facts, expected final state, and expected agent behavior, and an LLM state tracker infers a structured proxy state from the full interaction trace. LLM judges then verify goal completion and detect tool/user hallucinations against scenario constraints. Empirically, our benchmark produces stable, model-differentiating rankings across families and inference-time reasoning efforts, and its on- and off-policy rollouts provide supervision that transfers to unseen scenarios. Careful scenario specification yields near-zero simulator hallucination rates, as supported by ablation studies. The framework also supports sensitivity analyses over user personas. Human-LLM judge agreement exceeds 90%, indicating reliable automated evaluation. Overall, proxy state-based evaluation offers a practical, scalable alternative to deterministic agentic benchmarks for industrial LLM agents.

Toward Scalable Verifiable Reward: Proxy State-Based Evaluation for Multi-turn Tool-Calling LLM Agents

Large language models (LLMs) are increasingly being deployed in cost- and latency-sensitive settings. While chain-of-thought improves reasoning, it can waste tokens on simple requests. We study selective thinking for tool-using LLMs and introduce Adaptive Rejection Sampling (Ada-RS), an algorithm-agnostic sample filtering framework for learning selective and efficient reasoning. For each given context, Ada-RS scores multiple sampled completions with an adaptive length-penalized reward then applies stochastic rejection sampling to retain only high-reward candidates (or preference pairs) for downstream optimization. We demonstrate how Ada-RS plugs into both preference pair (e.g. DPO) or grouped policy optimization strategies (e.g. DAPO). Using Qwen3-8B with LoRA on a synthetic tool call-oriented e-commerce benchmark, Ada-RS improves the accuracy-efficiency frontier over standard algorithms by reducing average output tokens by up to ∼80% and reducing thinking rate by up to ∼95% while maintaining or improving tool call accuracy. These results highlight that training signal selection is a powerful lever for efficient reasoning in latency-sensitive deployments.

Downloads

Next from ACL 2026

Securing the Tool Layer: A Threat Taxonomy and Runtime Defense Framework for Model Context Protocol Deployments

Similar lecture

From Data to Diagnostics: How GenAI Revolutionizes Naval Maintenance

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES