United States

The Navy&#39;s Cyber Security Technical Authority is using LLMs to generate cybersecurity Standards. The implementation of LLMs has fundamentally disrupted the Navy&#39;s process for developing cybersecurity standards. Our efforts have transformed a multi-year, multi-million dollar single purpose task into a reusable agile process that realizes efficiencies to cost, schedule, and performance with improved operational security and increased fidelity of the resulting products. 


NAML 2025

Transforming Naval Cybersecurity Standards Development: The role of Scripting and Large Language Models to deliver standards tailored towards warfighter needs. 

llms

standards

cybersecurity

The Navy's Cyber Security Technical Authority is using LLMs to generate cybersecurity Standards. The implementation of LLMs has fundamentally disrupted the Navy's process for developing cybersecurity standards. Our efforts have transformed a multi-year, multi-million dollar single purpose task into a reusable agile process that realizes efficiencies to cost, schedule, and performance with improved operational security and increased fidelity of the resulting products. 


poster

### Welcome to the 9th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
 Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations

Please log in to explore this event.You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

**Get Your Tickets for NAML 2025 - Complimentary to students, government, and military**

Dive into the world of machine learning research relevant to Naval applications.

Now you can have access to all the lectures & posters and even engage with the speakers (send them emails, ask questions) on the Underline platform. 

We are offering a one-time only discounted on-demand access to this NAML 2025 content **for only $99** (complimentary to students, government, and military). 
 
**To register, please click** [**here**](https://naml25ondemand.paperform.co/)

Please register!

The 9th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.

Throughout the Fleet, operators are faced with complex tasks that require collaborative interactions between heterogeneous teams and software systems. This is especially embodied by Naval planners working through the Naval Planning Process (NPP). Although recent work has focused on modernizing the way this process unfolds through the introduction of digital systems like the Advanced Multi-echelon Planning Tool (AMPT), we believe that empowering planners with more natural ways to interact with tools like this will result in better user experiences and more streamlined workflows to bring them from initial concept to the final planning artifact. Achieving this requires the concurrent development of machine learning (ML) enabled processes such as automated speech recognition (ASR), user intent recognition, information extraction and synthesis, and in-context learning. Ultimately our goal brings all these pieces together such that Naval planners can employ the unstructured domain specific language they use with one another to directly interface with the planning systems needed to complete their jobs.

Natural Language Interfaces for Naval Planners

Mission planning is a complex and dynamic process, requiring robust systems and procedures to ensure the successful planning and execution of missions. Recent developments in generative AI have the potential to revolutionize mission planning, introducing new capabilities that enhance the speed and quality of mission plans. One such capability is the generation of aerial refueling schedules with the assistance of a Large Language Model (LLM).
On their own, LLMs struggle to reliably generate any type of plan due to the complexity of the problem [1]. Often, there are many implicit and explicit constraints that must be adhered to when creating aerial refueling schedules, that the LLM may not follow. This could lead to plans that look correct at a first glance but have errors in them upon deeper inspection. There are two primary contributing factors to this problem: hallucinations and a lack of reasoning skills. Hallucinations occur due to the probabilistic nature of LLMs, since they are not grounded in truth, and rely on their training data to produce text [2]. This also means that LLMs can imitate some basic reasoning skills, but quickly struggle as soon as any complexity is added to a problem [3]. Our project exploits the strengths of LLMs while minimizing its weaknesses to create a more robust aerial refueling planning system.
Using a LLM in conjunction with an optimization algorithm, we can take a natural language description of a mission, use the LLM to translate it into a structured format, pass it into an optimization algorithm, and generate an aerial refueling schedule. This utilizes the LLMs ability to parse natural language and output it into a specialized format. The optimization algorithm is developed specifically for generating aerial refueling plans, but this could be replaced by any algorithm-based tool that requires a structured input. By using this type of framework, the LLM is now solving a much simpler problem of translating natural language into a structured format such as JSON or a Pydantic model. Additionally, structured data such as JSON or Pydantic models can easily be validated, which significantly reduces the impact of hallucinations. Our results show that this approach is successful in developing aerial refueling schedules with several types of aircraft, departing from varying locations, with several constraints regarding their fuel level. By developing this framework, we are demonstrating how to safely deploy LLMs in an operational environment.
1. Xie, Jian, et al. "Travelplanner: A benchmark for real-world planning with language agents." arXiv preprint arXiv:2402.01622 (2024).
2. Waldo, Jim, and Soline Boussard. "GPTs and Hallucination: Why do large language models hallucinate?." Queue 22.4 (2024): 19-33.
3. Yao, Shunyu, et al. "React: Synergizing reasoning and acting in language models." arXiv preprint arXiv:2210.03629 (2022).


Using Large Language Models to Assist in Air Combat Mission Planning

Propulsion and flow control design for Navy vehicle systems require rigorous analysis of the fluid dynamics around the structures. The most common experimental approach for characterizing these complex flow fields is Particle Image Velocimetry (PIV), in which image sequences are used to estimate velocity flow fields. Traditional PIV methods can be time-consuming and resource-intensive, requiring significant manual calibration and post-processing. This is a significant bottleneck in the design cycle. In this program, we develop a machine learning PIV approach to accurately estimate velocity fields from complex real-world data for existing underwater flow control and propulsion programs. This could lead to an order-of-magnitude speedup in PIV processing, accelerating from months to hours.

FLAP-PIV: Fast, Learning-based Approach to Processing for Particle Image Velocimetry

The increased utilization of Unmanned Aerial Vehicles (UAVs) in diverse missions, from humanitarian aid to combat operations, underscores the necessity for an efficient and cost-effective development workflow for autonomous systems. Especially for defense purposes, building autonomous target recognition systems capable of detecting, identifying, and classifying adversarial agents with machine learning models requires extensive data for training. Consequently, simulation software has become an essential tool for developers seeking to assess autonomous system performance and collect data across various environments. Furthermore, the transition to real-world, application-ready systems necessitates a simulation platform that replicates not only the vehicle control algorithms but also environmental factors that affect system performance, such as lighting conditions and sensor noise. 

In response to these requirements, we introduce ‘SiDG-ATRID’ (Simulator for Data Generation for Automatic Target Recognition, Identification and Detection), a simulation platform that enables the collection of high-fidelity imagery data, powered by Unreal Engine 5. The simulator supports multi-agent simulations using the AirSim API library for UAV controls and simulates commercial aircraft traffic. This framework allows for customized camera placements to record videos or photos and manage environmental conditions such as weather and lighting. Additionally, by leveraging the Cesium API for geospatial mapping, it can accurately recreate real-world environments, enhancing the realism and applicability of simulations. This integrated approach enhances the efficiency and effectiveness of synthetic data generation for training machine learning and computer vision models.


SIDG-ATRID: Simulator for Data Generation for Automatic Target Recognition, Identification and Detection

The U.S. Marine Corps (USMC) faces significant challenges in adapting its operational frameworks to embrace modern technologies, notably in the integration of advanced data sciences and AI/ML. For example, finding the root causes for risks such as mishaps or incidents regarding a warfighting system is critical for designing policies and taking actions in strategic, operational, and tactical levels to prevent future risks.
These risks may attribute to multiple factors, especially human factors such as who operates/drives an equipment. The data sources detailing the incidents, systems, and operators might reside in distributed environment, sometime they are difficult to access without necessary policies and rules to justify the needs. For example, operators’ data may include personal identifiable information (PII), demographics, and personal history. In the context of WHI, different end users may not have the rights to share data directly because of sensitive nature of human elements. Furthermore, these data sets can be structured such as age, gender, education, military occupational specialty (MOS), etc., or unstructured such as descriptions of incidents. 
In this paper, we will demonstrate a systematic methodology using advanced data sciences and AI/ML to address these challenges in the following steps:
• Provide a distributed AI/ML agents infrastructure such as collaborative learning agents (CLAs) to analyze data sources separately and then fuse the patterns and models using shared vocabularies in the patterns instead of sharing raw data.
• Apply causality analysis including counterfactual reasoning to discover root causes of risks from structured data.
• Apply and compare natural language analysis methods including lexical link analysis (LLA), AI transformers such as bidirectional encoder representations from transformers (BERT), retrieval augmented generation (RAG), knowledge graphs (KG), and large language models (LLMs) to extract themes and causes from unstructured data to characterize the risks. 
The U.S. Marine Corps (USMC) faces significant challenges in adapting its operational frameworks to embrace modern technologies, notably in the integration of advanced data sciences and AI/ML. For example, finding the root causes for risks such as mishaps or incidents regarding a warfighting system is critical for designing policies and taking actions in strategic, operational, and tactical levels to prevent future risks.
These risks may attribute to multiple factors, especially human factors such as who operates/drives an equipment. The data sources detailing the incidents, systems, and operators might reside in distributed environment, sometime they are difficult to access without necessary policies and rules to justify the needs. For example, operators’ data may include personal identifiable information (PII), demographics, and personal history. In the context of WHI, different end users may not have the rights to share data directly because of sensitive nature of human elements. Furthermore, these data sets can be structured such as age, gender, education, military occupational specialty (MOS), etc., or unstructured such as descriptions of incidents. 
In this paper, we will demonstrate a systematic methodology using advanced data sciences and AI/ML to address these challenges in the following steps:
• Provide a distributed AI/ML agents infrastructure such as collaborative learning agents (CLAs) to analyze data sources separately and then fuse the patterns and models using shared vocabularies in the patterns instead of sharing raw data.
• Apply causality analysis including counterfactual reasoning to discover root causes of risks from structured data.
• Apply and compare natural language analysis methods including lexical link analysis (LLA), AI transformers such as bidirectional encoder representations from transformers (BERT), retrieval augmented generation (RAG), knowledge graphs (KG), and large language models (LLMs) to extract themes and causes from unstructured data to characterize the risks. 
We will demonstrate the methodology using the use case from the mishap and incident reports of a marine transportation equipment, e.g., motorcycles or PMV2 and related data resources from the I Marine Expeditionary Force (IMEF).


Discovering Root Causes of Risks for Warfighting Systems & Human Factors Integration (WHI) Capabilities 

Neural network verification provides model robustness guarantees in the presence of noise. Our results show promise in using a combined approach to significantly boost verification efficiency and lower the threshold for operational viability of AI in mission- and safety-critical applications. We showcase two disparate, but complimentary, efforts for efficient formal verification and model robustness, resulting in increased performance and decreased verification effort.
Efficiency of this NP-complete problem is studied using α,β-CROWN. So-called smoothed models are benchmarked against their original versions.

Efficient Formal Verification of Neural Networks for Mission Assurance

Artificial Intelligence (AI) has become integral to critical applications, yet the lack of robust mechanisms for safety and explainability poses significant security concerns. To address this, ObjectSecurity LLC has developed a novel AI/ML trust analysis tool—funded through SBIR and internal R&D—that detects vulnerabilities in AI systems, enhances explainability, and enables targeted patching of identified weaknesses within an AI model.

This poster presents the technical mechanisms behind our approach, demonstrating how it detects and mitigates vulnerabilities in AI/ML models. The tool conducts a layer-by-layer analysis to identify key risks, including susceptibility to adversarial attacks and inefficiencies caused by overfitting. It improves model transparency by uncovering hidden relationships between tokens and optimizing neuron utilization, enabling targeted fine-tuning for maximum efficiency. By generating actionable metrics and applying custom LoRA masks, the tool facilitates rapid model adjustments, enhancing robustness, accuracy, and adaptability.

By fostering trust, confidence, and accountability, our solution supports the deployment of AI systems in high-stakes environments.

Vulnerability Analysis of AI/ML Systems

Background: This study explores factors influencing attrition in the U.S. Marine Corps by analyzing participant data from the Recruit Assessment Program (RAP) and employing machine learning techniques. 
Methods: The primary focus is to predict contract fulfillment, defined as attrition, fulfillment, or retention, and secondly, to identify predictors of different types of discharge. The study population included 53,156 male recruits at Marine Corps Recruit Depot, San Diego, who enlisted between 2013 and 2018, with a five-year follow-up period. 
Results: The analysis revealed that 15.0% of recruits did not meet their contractual obligations, 58.6% fulfilled them, and 26.4% exceeded them. Machine learning models identified in-service factors—such as promotion status, military occupation, and deployment rate—as the strongest predictors of both attrition and retention, whereas pre-service mental and behavioral health factors had minimal influence. Promotion status, as reflected by grade expectations, emerged as the most important predictor, with failure to meet early career promotions increasing the risk of attrition. In contrast, pre-service mental health variables, including depression, PTSD, ACE, and problematic anger, were not predictive of career outcomes. 
Conclusions: These findings suggest that in-service experiences and career progression play a more substantial role in determining long-term success in the military than pre-service mental health factors. These results highlight the potential for improving retention through enhanced support services during early military service rather than relying primarily on pre-service health screening.

Comparing Pre-service and In-service Predictive Factors that Lead to Attrition Among Active Duty Marine Recruits

Delivered by Matt Beekman, AFCEA San Diego Chapter President and Katie Rainey, director of Science and Technology, Intelligence, Surveillance & Reconnaissance Department, Naval Information Warfare Center Pacific and NAML 2025 General Chair.

Opening Remarks

Power in the modern world - military, economic, geopolitical - is built on a foundation of computer chips. America has maintained its lead as a superpower because it has dominated advances in computer chips and all the technology that chips have enabled. (Virtually everything runs on chips: cars, phones, the stock market, even the electric grid.) Now that edge is in danger of slipping, undermined by the naive assumption that globalising the chip industry and letting players in Taiwan, Korea and Europe take over manufacturing serves America`s interests. Currently, as Chip War reveals, China, which spends more on chips than any other product, is pouring billions into a chip-building Manhattan Project to catch up to the US.

In Chip War economic historian Chris Miller recounts the fascinating sequence of events that led to the United States perfecting chip design, and how faster chips helped defeat the Soviet Union (by rendering the Russians’ arsenal of precision-guided weapons obsolete). The battle to control this industry will shape our future. China spends more money importing chips than buying oil, and they are China`s greatest external vulnerability as they are fundamentally reliant on foreign chips. But with 37 per cent of the global supply of chips being made in Taiwan, within easy range of Chinese missiles, the West's fear is that a solution may be close at hand. 

Premium content

Transforming Naval Cybersecurity Standards Development: The role of Scripting and Large Language Models to deliver standards tailored towards warfighter needs.

Next from NAML 2025

Natural Language Interfaces for Naval Planners

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES