United States

As large language model (LLM)-powered software is given greater freedom to act autonomously via tool calling and agentic frameworks, the need to monitor its behavior in order to minimize unintended actions continues to grow. The Multimodal Sensemaking in SeaVision (MUSES) project at NIWC LANT has used several open-source tools to test and evaluate LLM&#39;s and associated tools that are part of our software. MUSES is also researching using a separate LLM framework to continuously monitor and analyze T&amp;E results to enable autonomous self-correction when our systems fail or otherwise behave unexpectedly. This poster presents an overview of our T&amp;E work using several publicly releasable case studies.

NAML 2026

Open-source tools for LLM T&amp;E in a maritime awareness domain

seavision

tracing

test and evaluation

### Welcome to the 9th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
 Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

NAML 2025

The 9th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.

As large language model (LLM)-powered software is given greater freedom to act autonomously via tool calling and agentic frameworks, the need to monitor its behavior in order to minimize unintended actions continues to grow. The Multimodal Sensemaking in SeaVision (MUSES) project at NIWC LANT has used several open-source tools to test and evaluate LLM's and associated tools that are part of our software. MUSES is also researching using a separate LLM framework to continuously monitor and analyze T&E results to enable autonomous self-correction when our systems fail or otherwise behave unexpectedly. This poster presents an overview of our T&E work using several publicly releasable case studies.

Open-source tools for LLM T&E in a maritime awareness domain

poster

### Welcome to the 10th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.
#### On-Demand Access

* This is an on-demand site and all the recorded content from the workshop is available here. 
* Access to the recordings is exclusively available to registered participants.
* Each registrant has received an email notification, with instructions on how to access the content.

[![](https://assets.underline.io/markdown_image/1/image/60cd12aa376c191d3467ea2ee60944ea.png)](https://naml2026.org/)

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations

This is an on-demand site. All recorded content from NAML 2026 is available here and access to it is exclusively available to registered participants.

**Please log in using the email address you registered with.** If you have any issues logging in, please contact us at <naml2026@underline.io>.

Let’s celebrate the past decade by investing in the next. 🚀

This is an on-demand site. All recorded content from NAML 2026 is available here and access to it is exclusively available to registered participants.

**Explore the latest advancements in machine learning tailored to Naval applications 🔎**

With on-demand access to NAML 2026, you can watch all lectures & browse poster presentations at your convenience, while also engaging directly with presenters via the Underline platform - ask questions, exchange ideas, and stay connected.

**If you still haven’t registered, you can do so now.**
👇
**Take advantage of a special one-time offer: full access for just $99, with complimentary registration for students, government, and military employees.**

**Get access here: https://naml2026-ondemand.paperform.co/**

Please check your spam folder, or contact our support team at **<naml2026@underline.io>** if you registered, but still haven't received an access email.

Let’s celebrate the past decade by investing in the next. 🚀

Please register!

The 10th Annual Workshop on Naval Applications of Machine Learning (NAML 2026) was held in San Diego, California, March 2-5, 2026. NAML is a forum for interdisciplinary experts from government, industry, and academia to discuss results of ongoing research on theory and applications of machine learning to address some of the most challenging problems faced by our naval forces. 



Autonomous systems need environmental cues for orientation, navigation, and specialized tasks. Interpreting the layout of a space can be done by detecting objects and reading signs, which lead to enhanced scene understanding like identifying and assessing equipment in large spaces. We propose a pipeline for detecting, reading, and documenting relevant signs within a ship to aid in shipboard navigation and inventory tasks. This pipeline distinguishes between important Navy shipboard signage, including placards, exit signs, configuration labels, and bullseyes. Once the signs have been detected and classified, they are fed into specialized optical character recognition algorithms to read and decipher text according to the automated task. Finally, the sign classes and transcriptions are corrected via post-processing and mapped into a corresponding point cloud to form an inventory of the target space. This unique, proposed system comprises algorithms to be used in-situ for a set of inventorying and navigation tasks.

Shipboard Optical Character Detection And Recognition For Scene Understanding

In the shallow ocean, the seabed significantly influences sound propagation. Sound reaches and interacts with the seafloor, where seabed properties determine the amount of sound that is reflected or absorbed. Proper modeling of the properties of the seabed, then, is crucial for accurate source localization. While passive sonar inference of seabed parameters has often been accomplished using Bayesian optimizations, a deep learning approach may be an effective approach that simplifies the process in the long term. To begin evaluating the accuracy of this strategy, a 1D CNN is trained to perform the task of estimating seabed properties. The input data are on single frequency sound levels as a function of range from a towed source. Synthetic training data generated by a range-independent normal mode model for ocean environments corresponding to a wide range of seabed properties are used to train the CNN, which is then applied to measured data collected northeast of Scotland. A towed source produced the signal, which was recorded on a bottom-moored hydrophone. The recorded signal was then processed to obtain single-frequency power spectral density levels as a function of range. These data are input to the trained CNN and to a Bayesian optimization, which returns the posterior probability distributions (PPDs) for these parameters. A comparison between the CNN results and the PPDs is used to assess efficacy of the CNN in this generalization test and point to potential improvements in the deep learning approach, including future application of Bayesian networks. This search for deep learning approaches for the estimation of seabed properties has important implications for source detection, ranging, and tracking in the shallow ocean.

Evaluating Deep Learning for Seabed Properties Using Bayesian Optimization

Modern Navy maritime operations increasingly rely on advanced decision-aid tools and large-scale data analytics within Mission Operations Centers (MOC). However, a significant accessibility gap remains between these complex tools and MOC watchstanders who may lack domain-specific technical training. This work introduces an Artificial Intelligence (AI)-driven Control Flow framework that enables watchstanders to readily generate maritime network topology visualization and analysis using a custom-built network simulation tool named TOMO-Sim, through natural language, chat-style interactions. This project implements an agentic AI assistant designed to bridge the usability gap by allowing intuitive, plain-language communication with TOMO-Sim. Leveraging a state-of-the-art Large Language Model (LLM) orchestrated through the LangGraph python Library, the AI assistant interprets the user intent, gathers required parameters for network configuration, and autonomously interacts with the TOMOSim APIs to produce network topology visualizations. This approach providing the MOC watchstanders with simplified, context-aware insights, and maintaining a conversational workflow that enables intuitive, iterative refinement of the generated network models. The resulting workflow also offers a modular blueprint for integrating more advanced, future technical decision aids within an agentic AI framework.

Agentic AI Assistant for Maritime Network Topology Simulation

The OLIVE multi-media processing framework is in wide use across military and intelligence community applications. Capabilities include speech recognition, voice and face biometrics, translation, video and imagery summarization, language detection, deep fake audio and video detection and multiple other special purpose technologies. This presentation will focus on the integration of large language models (LLM) as both a source of natural language processing to extract information from, for example, automatically generated speech transcripts and visual captions to provide tools such as summarization, named entity recognition, sentiment, etc. and to post-process raw metadata on a massive scale to enable big-picture understanding of complex data streams. We will discuss the challenges and benefits of using LLMs in the processing of ISR and other multi-media data types, including the difficulties inherent in producing a reliable textual representation of non-textual data, strategies for prompting, limiting and constraining outputs, difficulties with completeness of coverage, etc. We also explore LLMs’ translation capabilities, especially mixed language input translations, which have long posed a problem for traditional AI systems. Finally, we look at the future of massively multi-modal models that heretofore have not yielded operationally-viable results but which clearly reflect the future of AI in multi-media processing.

Integration of Large Language Model Capabilities in a Media and ISR Processing Framework.

Artificial intelligence (AI) holds great promise in accelerating information processing in ways that can potentially help users make high-consequence decisions rapidly in operational environments. A key practical challenge is determining how to embed AI into command and control (C2) systems, especially those that are disconnected and have limited processing power. While large language models (LLMs) have demonstrated strong performance across various reasoning tasks, their size and computational requirements limit their suitability for deployment at the edge. The rapid growth of open-source small language models (SLMs) offers new opportunities for edge AI, but selecting the most appropriate model remains challenging, as standard benchmarks often fail to capture domain-specific tasks. This work reported here examines the viability of SLMs as support tools that enable users to make rapid decisions at the edge. As part of an Office of Naval Research-sponsored effort, we developed a novel evaluation framework combining insights from interviews with subject matter experts, workflow analyses, and guidance from naval doctrine to construct synthetic datasets and benchmarks representative of operational needs. 

We applied this framework to decision-making processes within the domain of operational law to evaluate the performance of SLMs in helping deployed commanders adhere to domestic and international regulations. While legal experts maintain the ultimate responsibility and authority in reviewing SLM outputs and making decisions, our results reveal benefits of SLM support for legal information retrieval and rules of engagement evaluation. 

These findings demonstrate that, with appropriate guardrails, SLMs provide a practical means to enhance decision support in resource-constrained C2 systems, improving both speed and reliability of critical actions. Finally, the proposed framework is generalizable to other mission domains and supports the development of AI-enabled C2 capabilities at the tactical edge. The poster will include illustrative workflow examples and SLM-assisted compliance checks to highlight practical operational applications.

Evaluating the Efficacy of Small Language Models as Decision Support Tools at the Edge

Sound propagation in the ocean depends on the parameterization of the seabed. The physical seabed properties are correlated and impact the data in different ways. To quantify the information content about the seabed properties in measured data, three tools for information geometry are used. These three methods are illustrated using bottom loss. Bottom loss represents the reflection coefficient of the seabed on a logarithmic scale and depends on both frequency and grazing angle. An analytical, multi-layered bottom loss model is used where the seabed layers are parameterized by their sound speed and density. Some combinations of layer properties produce similar bottom loss results, even if the specific values of their parameters are different. This ambiguity is investigated by applying information geometry to the model manifold: bottom loss as a function of frequency and grazing angle over a range of seabed properties. The resulting 2d model manifold is embedded in a data space with dimensions equal to the number of frequencies by the number of grazing angles. The edges of the model manifold correspond to bottom loss at the extreme values of one of the seabed parameters. To visualize this high dimensional data space, three information geometry techniques are used. For the first method, we use the Euclidean length of the edges of the manifold to indicate how much a parameter contributes to the overall model, which is referred to as “sloppy” or “stiff” parameters. The second method is to perform a Principal Component Analysis on the model manifold. Generally the first two principal components capture more than 90% of the model’s information and provide a simple way to evaluate the main sources of variation on the model manifold. The final method is to examine the Fisher Information Matrix (FIM). The FIM gives a quantitative measure of how much information a model carries about each parameter. To find the FIM, we perform automatic differentiation on the model for specific seabed properties. Using the FIM, we can find the conditional variance, Cramer-Rao bounds, and parameter correlations which can be examined to find how much information a parameter carries about the model. These information geometry techniques can be applied broadly to many types of models to quantify information content.

Information Geometry and Seafloor Bottom Loss

In this work, we present a multi-modal autonomous docking framework designed to operate robustly under unknown and changing dynamic conditions. The system supports three high-level operating modes: 1) human-in-the-loop supervision, 2) shared or assisted control, and 3) full autonomy, allowing seamless transitions across levels of operator involvement to support maritime operations 

Our approach integrates a deep-learning-based Koopman operator to obtain an accurate higher-dimensional linear representation of the vessel dynamics, enabling predictive control through an adaptive MPC formulation. Model mismatches and dynamic changes are handled using recursive least squares updates. Safety is enforced through a differentiable optimization–based Control Barrier Function (CBF) filter, which ensures real-time safe velocity commands across all modes of operation. 

Dock detection and localization are performed using multiple AprilTags, providing fast and reliable estimation of the docking structure and nearby obstacles. A simulation environment built in ROS and Gazebo enables systematic and scalable evaluation under a wide range of conditions. 

The proposed multi-modal control architecture is validated in simulation and through field experiments. Results indicate clear quantitative and qualitative improvements compared to baseline human drivers and alternative control strategies, demonstrating effective integration of differentiable CBFs with deep Koopman-based adaptive MPC for safe maritime docking.

Multi-Modal Safe Docking via Deep Koopman-Based Adaptive MPC and Differentiable CBFs

Biofilms form when ship hulls encounter naturally occurring microbial communities in marine waters. As these biofilms mature, their structure and microbiome evolve into a complex, self-sustaining microenvironment capable of supporting larger organisms such as barnacles and mussels. Environmental conditions (e.g., salinity, temperature, and water flow) strongly influence both the microscopic and macroscopic characteristics of developing biofilms. Because of this, biofilm composition can serve as a valuable indicator of geolocation. 

Raman microscopy provides a powerful means of examining biofilm microstructure by capturing vibrational signatures of molecular constituents. This technique yields non-destructive spectral fingerprints that are sensitive to spatial and developmental variations within biofilms, enabling precise identification. Traditional machine-learning approaches perform well on Raman data, allowing classification based on species composition, biofilm age, and responses to chemical or pharmacological stimuli. 

In this work, we present a method for determining environmental origin by distinguishing freshwater-derived biofilms from those grown in marine conditions. Although certain Raman spectral differences caused by salinity are known, the effectiveness of deep learning in detecting these subtle biochemical shifts remains largely unexplored. To investigate this, biofilms are being cultivated in both freshwater and seawater systems, with a target of 1,000 samples per condition. Each sample will be imaged, averaged to improve signal-to-noise ratio, and minimally preprocessed using baseline correction, smoothing, normalization, and cosmic-ray removal. The resulting spectra will be analyzed using a shallow 1D convolutional neural network to perform binary classification between freshwater and saltwater origins. Outcomes from this study will advance our broader goal of geolocating vessels based on biofilm signatures, with future plans to incorporate additional ecological influences on biofilm development. 



Classifying Freshwater and Marine Biofilms Using Raman Spectroscopy

This work covers the continued development of a concentration estimator for a contaminant plume generated by an unknown source in an underwater domain. The plant includes a stationary or mobile source, and the resulting plume is described using the 3D advection-diffusion equation. An autonomous mobile sensor moves through the plume and uses a process-state hybrid estimator to make real-time predictions of the plume from measurements of the concentration. The plant and estimator are discretized using a Non-Overlapping Domain Decomposition Explicit Finite Volume Method (NODDE-FVM) with constant time 4th order Runge-Kutta integration. Additionally, a novel method for measuring the gradient of a scalar field with a single mobile sensor is presented. This method is utilized by the mobile sensor to make estimations of the gradient of the plume without any information required from other sensors that may be present in the domain. This gradient evaluation method is based on a Taylor expansion of five or more scalar field measurements taken by the sensor along a 3D trajectory. The gradient results depend on both the conditioning of the resulting matrix equation and the error resulting from a Taylor series approximation of the field. Ongoing development of this work involves parameter estimation of the plume, focusing on source strength/release rate and localization in real time. This parameter estimation will follow similar work in implementing a neural-network framework to solve inverse problems for PDEs. In particular, we will utilize a physics-informed foundation to train a network on a general set of advective-diffusive scalar fields. This estimation scheme will be applied in tandem with the previously developed concentration estimator to complete a system that can estimate in real time the size, strength, and source location of a plume of contaminants in an underwater domain.

Real-Time Underwater Plume Estimation with a Mobile Sensor

Artificial intelligence models are increasingly integrated into naval systems for navigation, surveillance, decision support etc. These models can be targeted by adversarial attacks that can cause misclassifications. They also must perform under changing or degraded conditions such as lighting variation and changes in environment. In scenarios where AI is relied on for situational awareness and threat detection, the systems must continue to operate and provide credible outputs even in the presence of deliberate disruption.

This presentation describes a real-time defense and monitoring framework that keeps AI models detecting objects and targets under adversarial or degraded conditions while also flagging deliberate attacks. The system combines lightweight runtime analytics that track feature and confidence consistency across frames, anomaly detection that recognizes attack patterns in camera inputs, and analysis that separates adversarial interference from environmental noise. Our NAML 2025 presentation focused on fine-tuning models for improved robustness prior to deployment; this year’s work introduces an at-runtime defense that allows models to detect attacks in real-time.

Our team developed this work to meet the evaluation objectives of the Robust Artificial Intelligence Test Event (RAITE), a red-versus-blue exercise led by NSWC Crane that assesses AI robustness against cyber and physical threats. RAITE is scheduled for March 7-10, and internal evaluations aligned with its scenarios show that the system sustains object detection performance and identifies ongoing interference.

Our talk will share lessons learned in detecting and mitigating attacks on computer vision models, showing how to achieve real-time resilience without fine-tuning or major model changes. We will also discuss applications for shipboard surveillance, autonomous sensing, and expeditionary intelligence, surveillance, and reconnaissance (ISR) systems. All content is unclassified.

Premium content

Next from NAML 2026

Shipboard Optical Character Detection And Recognition For Scene Understanding

Similar lecture

MUSES: Multimodal Sensemaking in SeaVision

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES