United States

High Altitude Balloons (HABs) can leverage stratospheric wind layers for limited horizontal control, enabling applications in reconnaissance, environmental monitoring, and communications networks. Existing multi-agent HAB coordination approaches use deterministic methods like Voronoi partitioning and extremum seeking control for large global constellations, which perform poorly for smaller teams and localized missions. While single-agent HAB control using reinforcement learning has been demonstrated on HABs, coordinated multi-agent reinforcement learning (MARL) has not yet been investigated. This work presents the first systematic application of multi-agent reinforcement learning (MARL) to HAB coordination for distributed area coverage. We extend our previously developed reinforcement learning simulation environment (RLHAB) to support cooperative multi-agent learning, enabling multiple agents to operate simultaneously in realistic atmospheric conditions. We adapt QMIX for HAB area coverage coordination, leveraging Centralized Training with Decentralized Execution to address atmospheric vehicle coordination challenges. Our approach employs specialized observation spaces providing individual state, environmental context, and teammate data, with hierarchical rewards prioritizing coverage while encouraging spatial distribution. We demonstrate that QMIX achieves similar performance to the theoretically optimal geometric deterministic method for distributed area coverage, validating the MARL approach and providing a foundation for more complex autonomous multi-HAB missions where deterministic methods become intractable.

NAML 2026

Distributed Area Coverage with High Altitude Balloons Using Multi-Agent Reinforcement Learning

area coverage

multi agent reinforcement learning

high altitude balloon

distributed

deep reinforcement learning

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

The AAMAS conference series was initiated in 2002 in Bologna, Italy as a joint event comprising the 6th International Conference on Autonomous Agents (AA), the 5th International Conference on Multiagent Systems (ICMAS), and the 9th International Workshop on Agent Theories, Architectures, and Languages (ATAL).

Subsequent AAMAS conferences have been held in Melbourne, Australia (July 2003), New York City, NY, USA (July 2004), Utrecht, The Netherlands (July 2005), Hakodate, Japan (May 2006), Honolulu, Hawaii, USA (May 2007), Estoril, Portugal (May 2008), Budapest, Hungary (May 2009), Toronto, Canada (May 2010), Taipei, Taiwan (May 2011), Valencia, Spain (June 2012), Minnesota, USA (May 2013), Paris, France (May 2014), Istanbul, Turkey (May 2015), Singapore (May 2016), São Paulo (2017) and Stockholm, Sweden (2018), Montreal (May 2019), Auckland (May 2020, Virtual), London (May 2021, Virtual).
<br>
<br>



A ticket is required to attend this event, please register using the link below:

https://aamas2022-conference.auckland.ac.nz/attending/registration/

Registration Is Required

AAMAS 2022

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

technical paper

### Welcome to the 10th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.
#### On-Demand Access

* This is an on-demand site and all the recorded content from the workshop is available here. 
* Access to the recordings is exclusively available to registered participants.
* Each registrant has received an email notification, with instructions on how to access the content.

[![](https://assets.underline.io/markdown_image/1/image/60cd12aa376c191d3467ea2ee60944ea.png)](https://naml2026.org/)

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations
<br><br>

This is an on-demand site. All recorded content from NAML 2026 is available here and access to it is exclusively available to registered participants.

**Please log in using the email address you registered with.** If you have any issues logging in, please contact us at <naml2026@underline.io>.

Let’s celebrate the past decade by investing in the next. 🚀


Please log in to explore this event.

This is an on-demand site. All recorded content from NAML 2026 is available here and access to it is exclusively available to registered participants.

**Explore the latest advancements in machine learning tailored to Naval applications 🔎**

With on-demand access to NAML 2026, you can watch all lectures & browse poster presentations at your convenience, while also engaging directly with presenters via the Underline platform - ask questions, exchange ideas, and stay connected.

**If you still haven’t registered, you can do so now.**
👇
**Take advantage of a special one-time offer: full access for just $99, with complimentary registration for students, government, and military employees.**

**Get access here: https://naml2026-ondemand.paperform.co/**

Please check your spam folder, or contact our support team at **<naml2026@underline.io>** if you registered, but still haven't received an access email.

Let’s celebrate the past decade by investing in the next. 🚀

Please register!

The 10th Annual Workshop on Naval Applications of Machine Learning (NAML 2026) was held in San Diego, California, March 2-5, 2026. NAML is a forum for interdisciplinary experts from government, industry, and academia to discuss results of ongoing research on theory and applications of machine learning to address some of the most challenging problems faced by our naval forces. 



State-of-the-art autonomous systems often suffer from being size, weight, and power (SWaP)-prohibitive, coupled with a heavy reliance on extensive datasets for robust learning. Such limitations underscore the critical need for artificial intelligence (AI) to enable robots to learn from environmental interactions, adapt to dynamic conditions, and make informed decisions in novel situations without explicit programming. Neuromorphic computing, a specialized branch of AI, offers a promising solution to address the SWaP constraints of conventional autonomous systems. In this work, we showcase the engineering of a neuromorphic system tailored for autonomous exploration and adaptive learning, leveraging biologically inspired associative mechanisms to improve the performance of autonomous systems and reduce the amount of training data required. This computational approach demonstrates superior efficiency compared to conventional AI frameworks, exhibiting reduced computational overhead, enhanced power optimization, and improved adaptive response capabilities for robotic applications. In particular, our methodology utilizes both virtual simulation environments and physical experimental testbeds, both featuring a controlled arena equipped with chromatic visual markers and tactile perturbation elements to strengthen the sensory association. The hardware platform is made of an AI-ready NVIDIA processor and multiple sensor modalities, including LiDAR for spatial mapping and obstacle detection, depth imaging systems for chromatic pattern recognition, and inertial measurement units for tactile feedback acquisition. The associative learning algorithm facilitates a shift from rigid and preprogrammed robotic behavior towards adaptive and experiential learning. Such a technique reduces the need for exhaustive rule-setting and allows for more flexible responses to uncertain environments.

Neuromorphic Systems to Enable AI for Autonomous Systems

We present a novel approach to developing autonomous agents for the MCTF (Maritime Capture the Flag) competition by implementing both the environment and agents entirely in JAX. Our primary motivation is to leverage JAX hardware acceleration to significantly speed up reinforcement learning and machine learning training pipelines, enabling more extensive experimentation and faster iteration cycles. 

Our JAX-based implementation demonstrates decent performance in controlled settings, achieving 100\% win rates against random and purely offensive strategies red agents. However, the agent struggles to break ties against purely defensive opponents, a challenge consistent with results observed in previous competition years. These results validate that our JAX environment reproduces known strategic dynamics while offering substantially improved training efficiency. 

Furthermore, our work is important in linking the faster training back to the original environment, demonstrating sim-to-real issues often seen in reinforcement learning applications in the real world. A critical finding concerns the sim-to-real transfer gap: when deployed in the original Pyquaticus, agent performance degrades. Against random agents, we achieve 3 wins, 0 losses, and 7 ties. Notably, agents exhibit out-of-bounds navigation errors more frequently than in the JAX environment, highlighting fundamental differences between our accelerated training environment and the competition platform. 

This transfer problem represents the primary challenge for future work. We outline several promising directions to address these limitations: physics-based analysis to identify and minimize environmental discrepancies, followed by either fixing the training environment and retraining, or applying domain adaptation techniques during training or inference to compensate for environmental differences. Other methods include recent sim-to-real research such as progressive neural networks to mitigate catastrophic forgetting during cross-environment transfer, and multi-environment training strategies. These approaches collectively aim to preserve training efficiency while achieving robust competition performance.

Accelerated Multi-Agent Reinforcement Learning for Maritime Autonomous Agents: Towards Bridging the Sim-to-Real Gap

Large Language Models (LLMs) are transforming how we interact with computational tools, enabling natural language interfaces for complex workflows. The Model Context Protocol (MCP) provides a standardized way to connect LLMs with external services, allowing for execution of tasks without writing code. This session will introduce MCP and demonstrate how it can be used to perform data analysis through conversational interaction. We’ll explore the benefits of MCP and showcase a live example of connecting an LLM to a computational engine via MATLAB MCP Server to analyze and visualize data.

Highlights include:
-Overview of MCP and its role in LLM integration
-How LLMs enable code-free workflows for technical computing
-Live demo of a MATLAB MCP-enabled data analysis workflow

How to Talk to Your Data: Leveraging MCP Servers with LLMs for Code-Free Data Analysis

The integration of Large Language Models (LLMs) with physical systems is currently hindered by the fragmentation of hardware control interfaces and the complexity of real-time protocol management. To address this, we introduce a modular framework based on the Model Context Protocol (MCP) that abstracts physical capabilities into standardized, semantic tools accessible to AI agents. This ecosystem includes modules for vehicle control, computer vision, terrain analysis, and media streaming. By decoupling the high-level reasoning of the AI from the low-level implementation of hardware protocols, this architecture allows for more flexible and adaptable system designs. The system design emphasizes modularity, allowing distinct components to handle telemetry, geospatial data, and visual processing independently while maintaining a unified context for the agent. This work highlights the potential of protocol-based abstractions to accelerate the development of adaptable, general-purpose robotic systems.

A Modular Architecture for Unmanned Systems Control via the Model Context Protocol

As the number of new use cases on the Jupiter data platform has grown annually, so has the number of user questions that are sent to Jupiter’s technical support emails and dedicated office hours. To address this influx, “Jupiter Assist” was developed as the platform’s in-house large language model (LLM) chatbot. It is able to take advantage of years of documentation and technical support conversations using a retrieval-augmented generation (RAG) framework to provide accurate, domain-specific answers in real-time. Additionally, we cover the integration of generative models within Tableau tusing TabPy and Databricks MLflow, which provide the technical architecture for Jupiter Assist.

An LLM-Powered Technical Support Tool: Jupiter Assist

While the increasing power and complexity of Artificial Intelligence (AI) models has been a boon for automating rote human tasks (e.g., image classification) and identifying valuable patterns in massive amounts of data, the growing pervasive use of these models has also generated new attack-vectors which adversaries can exploit to compromise underlying model integrity to create adverse outcomes. Concern over these emerging AI-based vulnerabilities has given rise to new fields in cyber and software security that aim to harden and assure the resilience and robustness of AI algorithms prior to their deployment to production systems.

Noblis researchers are designing novel methods to exploit and test the robustness and resilience of various AI models, and using the results from said methods as part of an overall AI Risk Management Framework (AI-RMF) to assess a model’s suitability for deployment to mission-production systems.  

Our approach is novel, in part, in that it provides an extendable, scalable cloud architecture that accepts new model interrogation methods and applies them according to model type (e.g., transformer models for text classification, image-based models). The impact of this work is threefold. First (1), we aim to automate the risk-evaluation of AI models prior to their promotion to mission-production systems. Second (2), we seek to offer a repeatable, common method for assessing and comparing vulnerability risk across models to inform applied practitioners of potential model vulnerabilities and weaknesses.  Third (3), we aim to create an AI evaluation suite that can be used by multiple agencies and government to help them understand and assess the weaknesses and potential vulnerabilities of AI algorithms prior to their deployment to production.


Artificial Intelligence Assurance Implementation

Artificial Intelligence (AI) is playing an increasingly central role in the future of aerospace and defense systems. Across the aviation, space, and defense sectors, stakeholders are exploring how to safely and effectively deploy AI-enabled functionality in mission- and safety-critical systems. However, this shift is accompanied by a growing need to align AI development with evolving certification standards. In the defense and space domains, parallel frameworks such as NATO’s STANAG standards on Trustworthy AI and or ESA’s AI Assurance initiatives are beginning to emerge. These efforts underscore the challenges associated with verifying AI components including explainability, robustness, and traceability and their integration within larger certified systems.
At the same time, the demand for higher levels of autonomy continues to grow. Applications including auto-taxi systems and single-pilot operations along with systems like loyal wingmen, collaborative combat aircraft, and surveillance drones, require rapid, reliable autonomy under uncertain conditions. Meanwhile, the space industry has long relied on autonomous operations for deep-space missions, spacecraft fault management, and robotics. As such, the ability to develop AI-enabled systems that are robust, reliable, and certifiable is a strategic priority across domains.
To explore this challenge, this presentation will show a case study of an AI-enabled Airborne Collision Avoidance System (ACAS) derived from the ACAS Xu family of neural networks. This study demonstrates how advisory networks, which suggest safe maneuvers, can be integrated, verified, and tested within a closed-loop simulation framework. Using Simulink, the neural network is wrapped as an AI constituent model reference and embedded within a larger system model simulating real-time interactions between an aircraft and intruder. Verification is performed using formal methods from the CROWN family, applied both to MATLAB and Python-native implementations of the network. Complementary system-level testing leverages scenario-based simulation to assess performance across diverse encounter geometries. The workflow includes key steps for certification-readiness: capturing requirements, traceability to simulation and test artifacts, formal analysis of network properties, and generation of verification reports.
Finally, the presentation emphasizes the critical role of simulation in developing certifiable AI systems. As traditional test methods become infeasible for complex AI components, large-scale and distributed simulations offer scalable alternatives for validation and verification. These simulation frameworks enable exhaustive scenario coverage, integration, and parallelized execution of test campaigns. By combining Model-Based design, AI verification tools, and scalable simulation, the aerospace community can move toward a robust, more certifiable approach to deploying AI-enabled autonomy.


Towards the Certification of AI-Enabled Autonomous Systems in Aerospace and Defense: A Case Study on Airborne Collision Avoidance

Reinforcement learning (RL) has demonstrated remarkable success in solving complex decision-making problems, yet its adoption in critical domains is hindered by the lack of interpretability in its decision-making processes. Existing explainable AI (xAI) approaches often fail to provide meaningful explanations for RL agents, particularly because they overlook the contrastive nature of human reasoning—answering” why this action instead of that one?” To address this gap, we propose a novel framework of contrastive learning to explain RL selected actions, named VisionMask. VisionMask is trained to generate explanations by explicitly contrasting the agent’s chosen action with alternative actions in a given state using a self-supervised manner. We demonstrate the efficacy of our method through experiments across diverse RL environments, evaluating it in terms of faithfulness, robustness and complexity. Our results show that VisionMask significantly improves human understanding of agent behavior while maintaining accuracy and fidelity. Furthermore, we present examples illustrating how VisionMask can be used for counterfactual analysis. This work bridges the gap between RL and xAI, paving the way for safer and more interpretable RL systems

Why the agent made that decision: contrastive explanation learning for reinforcement learning

Modern naval operations increasingly demand high-precision analytic judgment, large-scale information processing, and rapid anticipation of adversary behavior. Human analysts excel at nuanced reasoning, rapid adaptation to changing scenarios, and the application of deep domain knowledge. However, their work is inherently limited in scale. Technology solutions like large-scale computational systems offer speed and capacity but lack the flexibility and contextual sensitivity required for complex and dynamic operational problems. Recent advances in AI agent architectures powered by LLMs and multimodal models offer a pathway to delivering highly precise analysis at never before seen scale while simultaneously enabling more faithful modeling of adversary cognition and decision-making.

We present the concept of Precision at Scale, a framework for understanding how AI agents can augment naval analysts and operators by combining expert-level reasoning with massive, policy-driven task decomposition across thousands of concurrent analytic threads. A key contribution of this framework is in the integration of adversary-aware modeling techniques. Agents can be grounded in adversary doctrine, cultural framing, linguistic priors, and historical patterns, allowing them to approximate the planning environment and strategic logic of competitors.

Our approach integrates four core elements: 

1. Adaptive agent workflows that allow agents to understand baseline expectations grounded in adversary doctrine while retaining the flexibility to shift strategies as mission context evolves.
2. Integrating multimodal data ingest pipelines spanning long-form text, geospatial data, open-source feeds, and additional high-fidelity modalities to adapt to the full range of relevant data.
3. Applying self-verification and proactively escalating to human operators to improve reliability and reduce analyst cognitive load.
4. Leveraging distributed agent teams to handle massive problem sets without sacrificing fine-grained reasoning quality.

This framework enables both precise outputs and scaleable adversary-behavior simulation, supporting use cases such as information environment analysis, network targeting, geospatial analysis, and wargaming. We report early experimental results and lessons learned from an operational deployment during Talisman Sabre 2025, during which the system produced multiple scenario predictions that later materialized. These findings highlight both the promise and limitations of deploying agentic systems in realistic operational settings.

We conclude by outlining open research challenges in modeling adversary cognition, scaling multimodal reasoning, and building distributed infrastructure capable of supporting real-time decision workflows. Together, these contributions establish an empirical foundation and technical roadmap for AI-driven systems that augment human planners and enhance quick-turn strategic foresight in contested environments.

Precision at Scale for Adversary-Aware Decision Support

Getting capability to the fleet on time is a critical aspect of NAVWAR’s
mission. The planning process may begin years before a modernization team steps
onboard a ship and involves many organizations along the way. As the
planning process proceeds, risks to on-time installation can develop. The
ability to quantify risks to on-time delivery during the planning process 
enables installation community leadership to make data-informed decisions to
mitigate risks before they become issues.
Can authoritative data be used to create a repeatable, analytical process that
identifies at-risk installs?
That question motivated NIWC Pacific’s Chief Engineer’s Office, in
collaboration with the NIWC Pacific Fleet Installation, Engineering & Logistics
Department, to embark on the On-Time Delivery Dashboard
project. Leveraging installation databases of record and machine learning
(ML), the team has created a ML model that predicts the likelihood that a given
installation will occur on time. The model also provides insight into which of
the model features most contribute to the prediction. The prediction results are visualized in a Power BI dashboard available on Flank Speed, and the tool is in use at FIELD, FRD, NAVSEA, and other organizations across the Navy.


Premium content

Downloads

Next from NAML 2026

Neuromorphic Systems to Enable AI for Autonomous Systems

Similar lecture

Disentangling Successor Features for Coordination in Multi-agent Reinforcement Learning

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES