United States

If two experts disagree on a test, we may conclude both cannot be 100% correct. But if they completely agree, no possible evaluation can be excluded. This asymmetry in the utility of agreements versus disagreements is explored here by formalizing a logic of unsupervised evaluation for classifiers. Its core problem is computing the set of group evaluations that are logically consistent with how we observe them agreeing and disagreeing in their decisions. Statistical summaries of their aligned decisions are inputs into a Linear Programming problem in the integer space of possible correct or incorrect responses given true labels. Obvious logical constraints, such as, “The number of correct responses cannot exceed the number of observed responses,” are inequalities. But in addition, there are “axioms”—universally applicable linear equalities that apply to all finite tests. The practical and immediate utility of this approach to unsupervised evaluation using only logical consistency is demonstrated by building no-knowledge alarms that can detect when one or more LLMs-as-Judges are violating a minimum grading threshold specified by the user.

NAML 2026

Logical Consistency Between Disagreeing Experts and Its Role In AI Safety

unsupervised evaluation

ai safety

formal verification

logic

2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics.

**Whova App** 
Stay in touch with your fellow conference attendees via the [Whova App](https://whova.com/portal/webapp/nacon_202106/)

**Conference Structure**
https://2021.naacl.org/blog/conference-structure/

**Walkthrough video of how to NAACL 2021** 

Please take a moment to view this video explaining how to navigate the platform, attend sessions network with other attendees. 


<figure class="video_container">
  <iframe src="https://screencast-o-matic.com/watch/crhwbGVh3vx?v=6&ff=1&title=0&controls=1" width=640  height=350  frameborder="0" allowfullscreen="true"> </iframe>
</figure>

<br>
<br>

NAACL 2021

technical paper

### Welcome to the 10th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.
#### On-Demand Access

* This is an on-demand site and all the recorded content from the workshop is available here. 
* Access to the recordings is exclusively available to registered participants.
* Each registrant has received an email notification, with instructions on how to access the content.

[![](https://assets.underline.io/markdown_image/1/image/60cd12aa376c191d3467ea2ee60944ea.png)](https://naml2026.org/)

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations
<br><br>

This is an on-demand site. All recorded content from NAML 2026 is available here and access to it is exclusively available to registered participants.

**Please log in using the email address you registered with.** If you have any issues logging in, please contact us at <naml2026@underline.io>.

Let’s celebrate the past decade by investing in the next. 🚀


Please log in to explore this event.

This is an on-demand site. All recorded content from NAML 2026 is available here and access to it is exclusively available to registered participants.

**Explore the latest advancements in machine learning tailored to Naval applications 🔎**

With on-demand access to NAML 2026, you can watch all lectures & browse poster presentations at your convenience, while also engaging directly with presenters via the Underline platform - ask questions, exchange ideas, and stay connected.

**If you still haven’t registered, you can do so now.**
👇
**Take advantage of a special one-time offer: full access for just $99, with complimentary registration for students, government, and military employees.**

**Get access here: https://naml2026-ondemand.paperform.co/**

Please check your spam folder, or contact our support team at **<naml2026@underline.io>** if you registered, but still haven't received an access email.

Let’s celebrate the past decade by investing in the next. 🚀

Please register!

The 10th Annual Workshop on Naval Applications of Machine Learning (NAML 2026) was held in San Diego, California, March 2-5, 2026. NAML is a forum for interdisciplinary experts from government, industry, and academia to discuss results of ongoing research on theory and applications of machine learning to address some of the most challenging problems faced by our naval forces. 



Presenting PEO C4I as an acquisition command, how S&T is done at PEO C4I and how industry/academia/government can help with AL/ML.

PEO C4I Science and Technology (S&T) at NAML

The motion response of a surface vessel in following or stern quartering seas may include a sudden uncontrollable turn, known as broaching. The physical reason for broaching is an instability in yaw caused by a combination of the hull forces, thus the rudder action may not have an effect at all. Broaching-to is not a new problem; it has been known since the Age of Sail [1], so experienced operators are aware of it.

A commonly known precursor to the broaching phenomenon is surf-riding. During a surf-ride, a ship accelerates or decelerates to match the wave celerity such that the ship travels with the incident wave profile. In the realistic sea, these periods of matching the speed of the wave profiles appear and disappear with acceleration, making analysis complex. Given the common frequencies inherent to the sea and typical ship speeds, surf-rides are typically linked with periods of “high-run” [2]. It follows that the instantaneous speed of the ship as well as the speed of the wave profile at the ship’s center of gravity are important parameters that inform surf-riding and potentially broaching.

The proposed method introduces a machine learning framework that links time series realizations of ship speed and wave celerity to the instantaneous yaw of the ship. A Long-Short Term Memory (LSTM) [3] neural network is trained with various combinations of tri-chromatic wave time series and surge velocity time series simulated with the reduced-order SimpleCode [3] as input and with yaw time series generated by SimpleCode as targets. The results of the trained neural network are evaluated against SimpleCode in ability to estimate large yaw events.

References

[1] Spyrou, K. J. “Perceptions of Broaching-To: Discovering the Past”, Chapter 22 of Contemporary Ideas on Ship Stability, Neves, M.A.S., Belenky, V., de Kat, J. O., Spyrou, K. and N. Umeda, eds., Springer, ISBN 978-94-007-1481-6, pp. 399-414, 2011.

[2] Themelis, N., Spyrou, K., and Belenky, V. “Surf-riding in Multi-chromatic Seas: “High-runs” and the Role of Instantaneous Celerity,” in Ocean Engineering, Vol. 120, pp 230-237, 2016.

[3] Hochreiter, S., and Schmidhuber, J., “Long Short-Term Memory,” Neural Computation, Vol. 9, No. 11, pp. 1735–1780, 1997.

[4] Weems, K. and Belenky, V. “Extended Fast Ship Motion Simulations for Stability Failures in Irregular Seas,” Proceedings of the 13th International Conference on Stability of Ships and Ocean Vehicles, Kobe, Japan, 2018.

Ship Broaching Prediction with Machine Learning

Reliable sonar interpretation is essential for many naval applications, from autonomous undersea vehicles to undersea surveillance and mine detection. However, deep learning models that achieve high performance in this domain often rely on large, resource-intensive architectures that are impractical for size, weight, and power (SWaP) constrained autonomous and edge-deployed buoy systems. Knowledge distillation (KD) provides an efficient alternative by transferring useful knowledge from large teacher networks to lightweight student models, allowing smaller models to perform the same task with reduced computational cost. While KD has shown promise across various audio tasks, its limited exploration in sonar arises from the challenge that existing methods often neglect the fine-grained texture cues critical for reliable detection and classification in complex, undersea acoustic environments. To address this gap, we leverage the Structural and Statistical Audio Texture Knowledge Distillation (SSATKD) framework, designed to enhance sonar perception across both passive and active modalities. SSATKD fuses high-level contextual features with low-level texture cues using two complementary modules: an edge detection module that identifies structural textures reflecting periodic patterns (i.e., repetitions) present in both spectrograms and sonar imagery (e.g., side-scan), and a statistical texture knowledge extractor that characterizes signal variability and distributional differences across environments. In passive and active sonar, SSATKD improves vessel classification and target segmentation respectively by leveraging structural and statistical texture cues that help improve with target recognition and precise localization. Across real-world sonar datasets, SSATKD extends prior work by providing stronger empirical validation in operational domains and improving texture modeling through a redesigned objective function, resulting in significant performance gains while reducing model parameters and computational cost for deployment on embedded, SWaP-constrained naval platforms. These results highlight the potential of texture-aware knowledge distillation to bridge perception tasks across sonar modalities and advance autonomous sensing capabilities for undersea operations.

LINE PROGRAM CUSTOM MARKING (LIMITED-RELEASE ONLY) - STANDARD MARKINGS ARE NOT APPLICABLE. USE THE FOLLOWING INSTEAD: Distribution authorized to U.S. Government agencies and their contractors. Other requests for this document must be referred to the MIT Lincoln Laboratory Technology Office. This material is based upon work supported by the Under Secretary of War for Research and Engineering under Air Force Contract No. FA8702-15-D-0001 or FA8702-25-D-B002. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Under Secretary of War for Research and Engineering.

Texture-Aware Knowledge Distillation for Efficient Passive and Active Sonar Analysis

Bottlenose dolphins produce various whistle types, including signature whistles, shared whistles, copies of conspecifics’ signature whistles, and variant whistles. While signature whistles have been well studied, due to their individual distinctiveness, much less has been done to identify individuals from non-signature whistle types. This study trained m18 convolutional neural networks (CNNs) to classify dolphin whistle producers from the 1D time-series inputs of whistle recordings of 20 dolphins at the U.S. Navy Marine Mammal Program. The model trained exclusively on signature whistles achieved 93.7% accuracy (random chance = 5%). The same model trained using all whistle types achieved 85.2% accuracy overall and performed slightly better on signature whistles (96.1%), suggesting the broader repertoire and larger dataset improved classification. Shared whistles were classified with 92.5% accuracy and variant whistles still were classified at 72.1%, suggesting that some individually identifying information is encoded across these types. Signature whistle copies proved difficult to classify (17.2%). With localization remaining a challenge in social groups, this study offers a promising step toward caller identification in well-studied populations. These results suggest that deep learning applied to raw audio time series data can learn individual whistle repertoires, offering new opportunities for acoustic monitoring of odontocetes.

Identifying Dolphin Whistle Producers with Deep Learning: Moving Beyond Signature Whistles

Accelerating Naval Decision Superiority through AI-Driven Multi-Domain Awareness
The future fight demands faster, smarter decisions—and Avathon AI is delivering the tools to make that happen today. Naval forces face unprecedented challenges: contested environments, distributed operations, and an explosion of sensor data across seabed-to-space domains. The question is: How do we turn that data into decisive action at the tactical edge?
Naval operations face two critical challenges: detecting and classifying threats at the edge of detection and transforming massive, multi-domain sensor data into actionable intelligence. The Office of Naval Research (ONR) and NIWC Pacific have prioritized these capabilities to enable distributed maritime operations and accelerate decision superiority.
Undersea Signal Processing for Long-Range Vessel Classification
Acoustic sensing platforms provide unique advantages for detecting surface vessels beyond radar and visual ranges. However, classification is complicated by non-stationary signals, environmental clutter, and limited labeled data. Avathon’s solution fuses physics-based modeling with AI techniques to overcome these challenges. Key innovations include:
• Hybrid Physics + AI Models for improved acoustic imaging and classification.
• Mass Feature Construction (MFC) across MFCC, spectral, wavelet, and time-domain features to identify predictive patterns at scale.
• Transformer-Based Architectures, including Masked Spectrogram Modeling (MSM-MAE), for unsupervised labeling and resilient embeddings in cluttered environments.
• Data Augmentation leveraging large public acoustic datasets to bootstrap models for naval-specific signatures.
Multi-Domain Awareness Advisor (MDAA)
Beyond undersea detection, Avathon’s MDAA delivers real-time fusion of radar, sonar, satellite, acoustic, and environmental inputs to detect, classify, and track threats across seabed-to-space domains. MDAA accelerates OODA loops, enhances battlespace awareness, and aligns with Project Overmatch objectives for distributed maritime operations. Designed for denied and degraded environments, MDAA integrates seamlessly with Navy DevSecOps ecosystems (COSMOS, NR&DE Cloud, Overmatch Software Armory) and supports edge deployment for tactical vehicles, mobile C3 facilities, and shipboard systems.
Operational Impact
Both solutions have been prototyped in joint service environments and are ready for transition into Navy programs of record. They directly support NIWC’s thrusts in C4ISR modernization, ISR fusion, and resilient logistics. By integrating advanced signal processing and multi-domain AI, Avathon delivers the decision advantage needed to outpace adversaries and sustain mission readiness across all domains.


Avathon AI-Advanced Decision Making

Inverse problems in deep learning that infer initial parameters from a final output are challenging, particularly when the generative process is a "black-box" function. These problems are often ill-posed and non-bijective, meaning a single output can map to multiple valid inputs. 

We introduce a novel framework that employs a conditional denoising diffusion model to predict the multimodal distribution of rendering parameters from a 2D image. We demonstrate our methodology on a practical use case: inferring camera and lighting parameters for a rendered image of a 3D vehicle model within a complex, non-differentiable Blender pipeline. Our approach trains a conditional diffusion model on a dataset of image-parameter pairs generated by the black-box. 

The architecture uses a pre-trained ResNet encoder to extract image features and a custom 1D U-Net with cross-attention to predict the noise in a corrupted parameter vector. By conditioning the denoising process on image features, the model learns the non-linear mapping between visual cues (e.g., shadows, highlights, and perspective) and the corresponding rendering parameters. 

We demonstrate how our trained model can generate a diverse, accurate distribution of plausible parameter sets from a single rendered image. By sampling the reverse diffusion process multiple times, the model captures the solution space's inherent multimodality. Quantitative analysis, measured by the Mean Squared Error between the ground truth and the best-predicted mode, illustrates our approach's high fidelity. Future work will focus on reducing training queries to the computationally expensive black-box, enhancing efficiency in applications like scientific simulation and procedural content generation.

Inverse Black Box Diffusion Modeling

Space-based Electro-Optical (EO) imaging sensors continued to be launched at an increasing rate. These systems may be used to help detect illicit maritime activity. However, challenges exist in the use of deep learning-based computer vision algorithms to derive maximum value from this growing data source. In particular, maritime ship detection and classification algorithms have demonstrated performance loss in the presence of novel imagery classes or out-of-distribution images. One means for mitigating this issue is the incorporation of Automated Identification System (AIS) vessel track data. Relatively little work has explored the use of multi-modal computer vision models to make use of satellite imagery and time coincident AIS data for image classification. This paper develops novel multi-modal imagery and AIS models to explore whether the addition of AIS data can improve model robustness. Image pixels are dropped to zero at varying rates to emulate varying degrees of out-of-distribution data in order measure robustness. It is shown that the multi-modal imagery and AIS models can yield an up to a 13.3% increase in the average precision to identify a ship in imagery when 67% of pixels have been dropped from an image. In all models examined, the multi-modal models outperformed the imagery-only models across the vast majority of cases examined. This confirms that the additional AIS modality can improve model robustness.

Characterizing the robustness of multi-modal AIS and imagery models for vessel image classification

By exploiting the inherent redundancy that exists across sensing modalities and advances in generative AI, one can enable reliable perception even in environments that traditionally degrade optical sensors, such as marine space. In this work, we present a generative adversarial network (GAN)-based framework for transforming sonar-style imagery to the corresponding RGB image in cases where there is intermittent camera occlusion by generating a ‘ghost’ clone of the camera image. Our model is pretrained on simulated RGB and sonar-style imagery, then fine-tuned on the limited in-situ data captured before camera failure. These modalities were selected because although sonar is a much more robust underwater sensor, and is useful for many underwater tasks such as navigation, mapping, and target detection, RGB cameras are still critical when visual detail is needed for tasks such as underwater scene reconstruction, imaging, infrastructure inspection, and marine life surveying. A key advantage of our approach is its ability to make use of very limited paired in-situ data, which is valuable because of how difficult and expensive it can be to collect large underwater datasets. Our results demonstrate that AI-based cross-modal translation between sonar and RGB domains significantly enhances the robustness of perception systems operating in highly dynamic and challenging marine environments. The images generated from our model have improved average PSNR, SSIM, and very similar LPIPS when compared to the current state-of-the-art. This indicates that our images have stronger pixel-level and structural fidelity to the ground truth images than the current state-of-the-art, while maintaining similar perceptual fidelity. When running SIFT feature extraction, our generated images produce feature counts and distributions that more closely match the ground truth than those of current state-of-the-art models, indicating improved performance. This performance points to the opportunity for success in downstream applications that depend on robust feature matching, such as structure-from-motion, SLAM, and underwater 3D reconstruction pipelines in future work.

Chasing Marine AURORAs: AI-Based Underwater Reconstruction and Robustness Augmentation

The rapid evolution of Machine Learning (ML) and Artificial Intelligence (AI) particularly Large Language Models (LLMs) and agentic technologies, presents exceptional opportunities and new complexities for Navy and Marine Corps mission planning and execution. For leaders, planners, and system owners, the path to successful, scalable AI adoption starts with a clear understanding of the foundational elements that make enterprise AI truly impactful. 
This session will explore critical infrastructure considerations, from dataset size evaluations and interpreting tokenization metrics to anticipate GPU resource needs, to mapping storage solutions for high-performance AI model training and inference. We’ll cover the nuances of vector storage requirements, common “gotchas” encountered when upscaling from pilot to production, and the operational benefits of multi-cloud and hybrid architectures in delivering both resilience and flexibility across diverse DoN environments. 
Illustrative scenarios will showcase how agentic AI workflows and LLMs can enhance everything from intelligence analysis to logistics and communications, demonstrating how specific environmental and architectural choices directly influence operational outcomes. 
Examples of these applied AI use cases include:
• Course of Action development and decision support
• Battlefield awareness and mission readiness 
• ISR and geospatial analysis
• predictive maintenance 
• recruiting retention, promotion, attrition, etc.

Attendees will also receive insights on optimizing data architectures and compute environments to support continuous innovation, agility, and mission alignment. 
Special consideration will be briefly given to AI at the edge, highlighting how edge-based solutions that integrate with cloud can support secure, high-performance AI even in disconnected and austere environments. 
By participating in this session, DoN stakeholders will leave with the tools and frameworks needed to systematically define requirements and prepare their environments, enabling the successful integration and scaling of AI-powered capabilities for sustained mission advantage.


Building the Foundation: Infrastructure Readiness for Mission-Critical AI and Multi-Cloud Operations

Radio Frequency Machine Learning will be essential for automated analysis of RF signals, and particularly in the adaptation of RF signals to their environment. At present, RFML, and machine learning (ML) methods for general time series problems, are underdeveloped, relative to text and image-oriented ML methods. In addition, as in other ML domains, there is a lack of openly accessible datasets, necessitating the use of synthetic data and domain adaptation to create ML models. However, training on synthetic data has been found to be problematic in virtually every area of ML.
We developed a new synthetic radar data generator, PyRadarWF, to explore the sensitivity of deep learning methods to different properties of radar waveforms. We labeled the data series with multiple labels, each set corresponding to a different property of interest. These labels were then used for training and inference to assess the sensitivity of ML methods in classification problem. We have labeled for modulation 18 class, and pulse width and SNR ranges. For deep learning ML methods, we employ 1D adaptations of resNet18 and a modified transformer NN.
Finally, we use a radar emulation system to generate similar datasets to test the quality of this domain generalization. By training on the synthetic data and inferring on the emulated data, we discuss how well the ML methods capture each of the labeled properties.


Premium content

Downloads

Next from NAML 2026

PEO C4I Science and Technology (S&T) at NAML

Similar lecture

Generating Negative Samples by Manipulating Golden Responses for Unsupervised Learning of a Response Evaluation Model

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES