United States

Cognitive radar systems must efficiently allocate limited resources to maintain accurate situational awareness in dynamic environments. Traditional rule-based approaches often struggle to adapt to rapidly changing conditions, resulting in suboptimal target-tracking performance. In this study, we present a reinforcement learning-based framework for resource allocation in multi-target tracking tasks. By employing Deep Q-Networks (DQN), the radar system learns to dynamically allocate scanning resources across multiple tasks.

Our approach utilizes extended Kalman filter (EKF) estimates of target states, including position, velocity, and covariance matrix elements to inform the decision-making process. The reward function is designed to promote uncertainty reduction and maximize detection efficiency. Performance is evaluated in a Python-based simulation environment with varying numbers of targets, comparing DQN-based strategies against traditional equal allocation methods.

Results demonstrate that the DQN-based radar agent achieves improved target detection times and reduced covariance matrix traces in certain scenarios with moderate target density. However, performance declines when the environment becomes highly dynamic with many targets. Ongoing work focuses on refining reward structures, exploring alternative reinforcement learning architectures, and introducing multi-objective optimization to enhance radar performance across diverse operational conditions.

This research highlights the potential of reinforcement learning to significantly improve cognitive radar resource allocation, offering promising avenues for future applications in aerospace and defense contexts.

NAML 2025

Reinforcement Learning Based Resource Allocation for Radar Scanning and Multi-target Tracking

target tracking

cognitive radar

resource allocation

reinforcement learning

poster

### Welcome to the 9th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
 Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations

Please log in to explore this event.You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

**Get Your Tickets for NAML 2025 - Complimentary to students, government, and military**

Dive into the world of machine learning research relevant to Naval applications.

Now you can have access to all the lectures & posters and even engage with the speakers (send them emails, ask questions) on the Underline platform. 

We are offering a one-time only discounted on-demand access to this NAML 2025 content **for only $99** (complimentary to students, government, and military). 
 
**To register, please click** [**here**](https://naml25ondemand.paperform.co/)

Please register!

The 9th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.

Tremendous variations coupled with large degrees of freedom in UAV-based imaging conditions lead to a significant lack of data in adequately learning UAV-based perception models. Using various synthetic renderers in conjunction with perception models is prevalent to create synthetic data to augment the learning in the ground-based imaging domain. However, severe challenges in the austere UAV-based domain require distinctive solutions to image synthesis for data augmentation. In this work, we leverage recent advancements in neural rendering to improve static and dynamic novel-view UAV-based image synthesis, especially from high altitudes, capturing salient scene attributes. Finally, we demonstrate a considerable performance boost is achieved when a state-of-the-art detection model is optimized primarily on hybrid sets of real and synthetic data instead of the real or synthetic data separately.

UAV-SIM: Enhanced UAV-based Human Detection Using Novel Neural Rendering

Setting up a local Large Language Model (LLM) on a Navy RDT&E laptop can leverage open-source tools and models approved for DoD use. The integration of the LLM into Visual Studio Code as a chat bot, tab autocomplete, and code editor, can increase efficiency writing code and improve research capabilities. This enhanced workflow allows for accessing tools in offline environments and for asking questions of project documentation using RAG, while ensuring confidentiality of sensitive information.

This poster presentation provides details on setting up a local LLM and highlight the benefits for Navy software developers. Showcasing the challenges and solutions associated with using a local LLM on an RDT&E machine can lead to more Navy software developers, both ashore and afloat, utilizing this useful tool.

Local LLMs for Software Development on a Navy RDT&E Laptop Asset

Autonomous robot scanners can map and annotate environments for a variety of DoD relevant purposes, such as supporting installations, reverse engineering, and maintenance of facilities and ships. As scanners perform their scan in a fixed location, one key aspect is finding the optimal scanning waypoints to cover a given environment by locating landmark structures, such as stairs, doors, desks, ladders, and other prominent features. In this work, we use a SPOT robot to create our own maps and an Oak-D camera for “in-real-time” object detection. We show that object detection can be used to enhance the traversal of mapped environments by annotating landmarks and using the detections as insight for object avoidance.

Object Detection And Map Annotation For Autonomous Robot Scanning

In this work, we discuss the development and deployment of a robust autonomous system capable of performing diverse maritime tasks, including autonomous docking with human-in-the-loop interaction, under unknown dynamic conditions. Our approach leverages a data-driven modular design to facilitate a seamless transfer of autonomy across various maritime surface vessel platforms. This design addresses challenges associated with a priori system model identification, particularly under evolving system behaviors and shifting environmental conditions. The proposed learning-based platform integrates a deep Koopman system model to capture system dynamics and enable predictive control. A safety filter ensures the generation of safe velocities, which the recursive least squares model predictive controller (RLS-MPC) follows to achieve robust and reliable motion control. Dock detection is performed using a camera and AprilTag, providing precise and reliable localization for docking tasks. To support development and testing, we created a simulation environment in ROS and Gazebo, enabling realistic and scalable evaluation of the system under various scenarios. We introduce safety-aware Koopman-based adaptive MPC control architecture. The framework's efficacy is validated in both simulation and real-world experiments, demonstrating its capability in autonomous docking tasks. Results show consistent improvements in accuracy and robustness compared to approaches that overlook dynamic system changes, underscoring its potential for advancing maritime autonomy.

Safety-aware Koopman-based Adaptive MPC for Docking of Autonomous Surface Vehicles

"The increasing complexity of machine learning (ML) models used in longitudinal data analysis often hinders effective interpretability, a critical barrier in high-stakes environments like the US Navy. To address this, we propose a novel framework: Embedding Model Agnostic Human Readable Attribution (EMAHRA) into longitudinal analyses. This approach one-hot encodes the trends of longitudinal variables into human-readable descriptors, such as ""slow rise,"" ""abrupt jump,"" or ""steady decline,"" to provide model explanations in language comprehensible to human decision-makers.

By transforming raw data trends into descriptive, categorical variables, EMAHRA simplifies the interpretive layer of explainable AI (XAI). Importantly, this method enhances not only the explainability but also the consistency of model outputs. Our results demonstrate that encoding trends as human-readable descriptions mitigates the influence of minor fluctuations in data, focusing the analysis on overriding trends. This improvement was observed to reduce noise in the data, resulting in models that were both more stable and more consistent in their predictions.

Applications in a US Navy setting underscore the value of EMAHRA. For instance, in ship maintenance, this approach enables personnel to interpret predictive models identifying components at risk of failure. Instead of opaque outputs, models could explain predictions with statements like ""The hydraulic system shows a steady decline in pressure, indicating wear and tear."" Similarly, in operational readiness, longitudinal health data of personnel could be described as ""gradual improvement in fitness metrics,"" helping leaders allocate resources more effectively."

Embedding Model Agnostic Human Readable Attribution into Longitudinal Analyses for Better XAI

"Neuromorphic processing mimics the processing of neural structures in the brain. Neuromorphic computer architectures enable the development of sophisticated and novel capabilities in pattern recognition, event reasoning, resilient decision-making, machine learning, and unsupervised tasking for energy-efficient mobile combat vehicles. Neuromorphic sensors with several benefits are event-based cameras. They feature little latency and low power consumption and are offered as COTS solutions. Spiking neural networks (SNN) can be trained with the event-based datasets that are produced. This paper offers a comprehensive overview of neuromorphic computing, encompassing the latest hardware and software platforms as well as some datasets in the field. This effort is based on a NISE project in which we surveyed neuromorphic computing and SNNs and collected real-time data using DAVIS cameras.
DVS cameras are a type of event-based camera -- asynchronous sensors that sample light based on the scene dynamics, rather than on a clock that has no relation to the viewed scene. Their advantages are: very high temporal resolution and low latency (both in the order of microseconds), very high dynamic range3 (140 dB vs. 60 dB of standard cameras), and low power consumption. Hence, event-based cameras have a large potential for robotics and wearable applications in challenging scenarios for standard cameras, such as where high speed and high dynamic range are critical. Spikes that indicate a change in brightness are returned by DVS cameras, which concentrate on using brightness to identify an event. SNNs and DVS cameras are complementary in that they both handle inputs by default using event-based triggers. Therefore, for algorithmic implementation, DVS cameras and neuromorphic computing hardware may result in a small form factor AI-based sensing paradigm that is both natural and incredibly efficient.
Lava is an open-source SW platform for creating AI apps for Intel Loihi and other neuromorphic hardware designs.  Lava currently supports Intel's Loihi architecture and conventional CPUs, but its compiler and runtime can be extended to support other architectures. Third-party libraries such as Nengo, ROS, YARP, and others can be added to and modified with Lava. Additional tools enable users to profile workload power and performance, visualize complex networks, and assist with the float to fixed point conversions needed by many low-precision devices, including neuromorphic hardware.
SNNs were trained in the present investigation using dynamic vision sensor (DVS)-based datasets using LAVA and other open-source tools. This is mainly because, once available, LAVA hardware will offer significant performance benefits. We examined MNIST and N-MINST, two widely used opensource datasets that serve as a benchmark for creating Navy-specific proprietary datasets and are used to train SNNs. The use of these datasets for SNN training and implementation was covered in this paper. Information on the SNN MNIST Classifier: Using a multi-layer feed-forward classifier neural network that is easily constructed and uses pre-trained weights from the LAVA repository, without the need for convolutional layers. 2) SNN testing with twenty-five MNIST dataset images 3) A classification accuracy of 88% was obtained. 2) The second portion of the SNN training phase took a long time. By adding a few more open-source spiking neural component primitives to PyTorch, users can use the most recent and stable setup created with PyTorch, as well as spiking neural network components suitable with deep learning.Using these primitives, an SNN was built and trained on the entire N-MNIST dataset. The accuracy and runtime performance were documented during five epochs. The outcomes are extensive and impressive.
Although the aforementioned event-based datasets are publicly available, none of them were designed with dod data—such as drones—in a generic format in mind. Any project that requires a drone data set must be built from the ground up. This might mean not just spending the time and employing the drones to record, but also investing in a DVS camera. In cooperation with the SAUSR range at the Naval Weapons Station, drone movement footage was recorded. We were able to set them up so we could record their plans, which they required tested. "

Neuromorphic Computing & DVS Cameras

This project's objective is to analyze how methods of control versus reinforcement can be applied to DC motors that actuate unmanned aerial vehicles (UAVs). In a model-following approach more akin to adaptive control methods, the recursive least-squares method and other least-squares method regression are performed in this endeavor. This is compared to that of the use of reinforcement learning on a deterministic method that is also used to control output to a trajectory. A stochastic method is also an aim when the latter method is accomplished in order to determine how physics-based models perform under different probability-based models in terms of training. Examples include the optimal feedback learning for altering the control signal, in addition to exploring different types of controllers (i.e. proportional, derivative, integral, PID, etc.) for seeing how parameters influence transient outputs and inputs.

DC Motor and Altitude Control for UAVs with Adaptive/Reinforcement Learning + Deterministic AI

Given a heterogenous set of agents, we examine the use of a convolutional neural network to decide which agent should be deployed to perform a set of tasks in an environment with dynamically changing terrain. Our work takes in an initial map of the environment to make predictions of how the environment will change as an agent moves. In our simulations, we designated every cell within a grid as having water, mud, or land terrain. Our agents were specialized as either water-efficient travelers or land-efficient travelers, but all agents could move across any terrain type with corresponding speeds derived from their aptitude for that particular terrain. We used deep reinforcement learning to train the agents to explore a flood environment towards eventually reaching a target. These agents were then evaluated based on the number of steps taken to achieve success for 5,000 different scenarios. In each scenario, only the starting map of the environment and the agent that completed the task in the least number of steps was recorded and compiled into a dataset. A convolutional neural network was then trained on the dataset to predict which agent (water-efficient or land-efficient) should be chosen for a mission. Through evaluation on separate test cases, the model averaged a 90% accuracy in its prediction of which agent to send for a flood-rescue mission.

Exploratory Navigation of Dynamically Changing Flood Environments With Autonomous Delegation of Tasks

Through the application of innovative Naval GenAI foundation, based on historical fleet and vessel maintenance data – including maintenance notes and other unstructured data elements – naval personnel can use machine learning and natural language processing technologies to determine the specific inspection required as well as precise repair procedures and parts required for that necessary repair. Accordingly, the GenAI models will provide instant and data-driven precision diagnosis, along with recommended courses of action including suggested steps for inspections, repairs and swap out or overhauling of equipment. The building of a Navy GenAI model for fleet and vessel maintenance will lead to improvements in maintenance and operational availability as the model has several advantages over traditional schedule-based preventive maintenance methods. Through the amalgamated use of historical data along with maintenance notes, Sailors, maintainers and tech support personnel will be able to learn from past performance related to repairs, automate diagnostics processes, and receive trustworthy guidance regarding the ideal course of action when completing maintenance for fleets and vessels. Additonally, a use case for welding is presented.The Shipbuilders and the DoD Maritime Industrial Base (MIB) experiences delays and bottlenecks from weeks to months, with both Welding and Non-Destructive Testing documentation reviews and approvals. These delays result in 2nd and 3rd orders affecting the downrange in the planning, resourcing, and construction process. To meet the needs of our Nation, IBM has begun development, and will be deploying Minimal Viable Products (MVPs), by leveraging GenAI to deliver to the MIB:
1) User-friendly Procedure Look-up ChatBot
2) AI-Procedural Review Assistant
To accomplish this, IBM designed an innovative Welding Documentation Assistant by ingesting 3ea Technical Publications, along with 1ea MIL-Standard starting the End-to-End Process journey. Though many more Welding Specifications are available (>100ea), a strong foundation for the model at the start was a must. This enables Stakeholders to quickly understand terminology, definitions, quickly look-up the correct specification required based on the Weld DNA, to helping a user write a weld procedure. This is a very complex and time-consuming process, so this assistant will significantly reduce the time and guesswork to get to the correct answer. There will even be time savings for the most experienced with Welding knowledge to find efficiencies. Not to mention all those with less experience or perhaps are new(er) to Navy Standards.


Streamlining the complexity of Welding and Inspection Quality Documentation through GenAI and LLMs.

As part of the Intelligence Advanced Research Projects Activity (IARPA) BRIAR program, Accenture has developed, BLAU, a video-based biometric algorithms to extract accurate and reliable identity recognition and identification under non-cooperative scenarios including long-range, through atmospheric turbulence or from aerial platforms. Leveraging both image-based and temporal-based features, BLAU extracts signatures from the subject’s face, body and gait to build a robust whole-body representation.
With imagery (images or videos) captured of subjects, BLAU first performs body detection to find the individual(s) in the scene. Our body detector is capable of finding subjects that are fully (head-to-toe) and partially (head-to-waist, head-to-knees, etc.) visible. From the image crop of the body, BLAU then performs face detection to find the face. Using the body and face crops, BLAU generates biometric signatures for each modality (face, body, and gait): Face is analyzed with a quality-modulated CNN with adaptive margin loss that has been trained using embedding space augmentation, facial attribute and soft biometric expansion, facial alignment, geometric test time adaptation, among other techniques. Body is analyzed with a ResNet-based architecture that is customized with transient-appearance adversarial loss; and gait is analyzed with a pyramid spatial-temporal aggregation network to process full-motion video. The gait module learns spatio-temporal representations that can discriminate individual’s unique motion patterns.
A key module of BLAU is its fusion engine. An advanced fusion framework intelligently fuses the different biometric signatures, amplifying the strengths of some modalities while mitigating weakness with other modalities. This fusion operation is learned from training data. It works by attending to several factors (image quality, face/body pose, etc.) that are used to weight each modality adaptatively for each video. This enables BLAU to perform accurate recognition even when minimal biometric information is present, and media quality is heavily degraded.
In this presentation we will provide an overview of the BLAU pipeline, highlighting some of the innovations developed by our team, along with our academic partners, and show performance results for several use cases.


Premium content

Reinforcement Learning Based Resource Allocation for Radar Scanning and Multi-target Tracking

Next from NAML 2025

UAV-SIM: Enhanced UAV-based Human Detection Using Novel Neural Rendering

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES