United States

With the proliferation and rapidly improving endurance of next-generation unmanned undersea systems, long-term deployment of human-on-the-loop subsea and seabed resident fleets are fast-approaching. More specifically, wave-generated power and other renewable power sources, autonomous docking capabilities, and increased edge processing capabilities enable subsea residence of both static and mobile underwater systems. In this research, we evaluate human-on-the-loop adaptive mission planning algorithms for managing a fleet of unmanned understeer systems in simulation. These systems could provide persistent monitoring, maintenance, and repair of critical undersea infrastructure vulnerable to grey zone operations; such a system could estimate and track infrastructure health, including detection and sensor data acquisition of significant changes. Moreover, subsea resident systems could perform continuous mine survey of a particular (even mobile) area of responsibility and conduct environmental surveys over extended periods of deployment. It is often impractical or impossible for these systems to maintain consistent contact with human operators, necessitating novel approaches to mission planning that bridges the gap between constant human oversight and fully autonomous operation of the platforms. 
These persistent, autonomous systems are expected to adapt to increasingly complex environments and tasking, collaborate with other autonomous systems, and to respond appropriately to the preferences of human operators without requiring unnecessary oversight. The mission planning algorithms used by these systems must decompose high level objectives into discrete tasks that can be described in terms of their cost to complete, and their efficacy can be evaluated by the computational time required to create a solution, as well as the efficiency of the solution itself. To study these algorithms, a multi-agent problem is posed in simulation, wherein a simulated group of systems (agents) complete a number of (potentially mobile) tasks in the shortest time possible without colliding with other agents, or with dynamic obstacles drifting through the environment. By implementing a multi-stage approach to the problem decomposition, this work proposes an algorithm that outperforms state-of-the-art algorithms, such as the Consensus-Based Bundle Algorithm, with respect to both the computational time to create the plan and the efficiency of the plan itself. 


NAML 2023

Human On the Loop Adaptive Mission Planning 

human-machine-teaming

seabed

autonomy

mission planning

multi-agent

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

NAML 2021

poster

### Welcome to the 7th annual workshop on Naval Applications of Machine Learning!

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

**General Chairs:**

Katie Rainey, Naval Information Warfare Center Pacific

Nicole Isoda, Naval Information Warfare Center Pacific<br><br>

<button name="button" onclick="window.open('https://sites.google.com/go.spawar.navy.mil/naml/','blank')">**NAML 2023 Website**</button>

<br><br><br>

To access the site please register [**here**](https://events.r20.constantcontact.com/register/eventReg?oeidk=a07ejmzkc5g71584a82&oseq=&c=&ch=). 

The 7th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


The need for abundant, diverse, and relevant training data continues to increase as deep machine learning based computer vision models grow in complexity. We have seen that tasks such as image classification and object detection benefit from large and varied datasets. Unfortunately, the cost of obtaining accurate labeled data at scale is prohibitive especially for more complex tasks such as object detection which require both categorical labels and accurate bounding box annotations. Active learning (AL) seeks to alleviate the need for brute-force data annotation by analyzing and automatically sampling a small but informative subset of data for labeling. With an effective sampling strategy, the trained model can achieve comparable performance to fully supervised methods at a fraction of the labeling cost. Sampling strategies generally fall into three categories: uncertainty-based, diversity-based, and hybrids of the two. For satellite imagery applications of computer vision, AL is an attractive potential solution: vast volumes of unlabeled satellite imagery exist, but annotation is expensive and tedious. For object detection in synthetic aperture radar (SAR) imagery, labeling is particularly challenging. Labeling SAR images is more costly than electro-optical (EO) images because the average person lacks the domain expertise to interpret and label SAR imagery. Because of this, crowdsourcing data annotation is not a viable option. Thus, we must intelligently allocate scarce labeling resources to the data most likely to improve the performance and robustness of the model being developed. The goal of AL is to automate and optimize this dataset curation process. We propose to evaluate active learning methods applied to maritime object detection in SAR imagery. Recent work has shown the efficacy of active learning for common EO image datasets like PASCAL VOC and COCO, but how do these methods perform in SAR imagery with unique image properties and much smaller objects? We consider two active learning methods: CALD selects images based on model uncertainty cues, and VAAL selects diverse images from the full dataset independent of the task or model being trained. With uncertainty-based methods the uncertainty cues are gathered from the model under training, so there is an inherent coupling between the model and the selected data. When a labeled dataset acquired for one model (the acquisition model) is used to train another model of a different architecture (a successor model), what are the model performance impacts? Given the costs of labeling datasets, it is desirable for the labeled set to transfer well across a variety of model architectures, since datasets are likely to be reused throughout model development. Our evaluation comprises two parts:
• We measure AL algorithm performance across EO and SAR image datasets and three deep object detection models. 
• We examine transferability of AL-acquired data for training different successor models.

Evaluating Active Learning Methods for Synthetic Aperture Radar Maritime Object Detection

Model cards organize and portray information
about a model’s training data, hyper parameters, and behavior,
such as predictive accuracy and bias for machine learning (ML)
algorithms or models. Consumers use these cards to determine
whether a model is well suited for a particular use case.
However, the current design of model cards does not include
resource utilization metrics, which are important when models
are being optimized for specific hardware. The main objective
of this study was to determine which set of hardware-centric
performance metrics provided the most value to users when
comparing among different algorithms for Internet of Things
(IoT) edge devices. The Department of Defense (DoD) can utilize
these metrics to select the best model for key mission tasks
within the Command and Control (C2) missions that require
AI/ML deployment on edge devices. Our study focused on
finding correlations between resource availability and machine
learning models’ computational footprint using three key metrics:
energy consumption, execution time, and memory utilization.
The data was gathered from simulated environments hosted
by Docker containers (i.e., lightweight, stand-alone, executable
package of software that includes everything needed to run an
application) with various memory sizes. Two different series of
experiments were performed. The first series of experiments
simulated environments with 256 MB to 8GB (steps of the
powers of 2) of available memory. For each memory size, we
deployed 50 different containers, calculated the metrics from
each container, and then recorded the average result of each
metric. Two ML models from Microsoft’s EdgeML library that
are designed to run more efficiently on edge devices were used:
Bonsai, a decision tree based model, and ProtoNN, a KNN based
model. Using Spearman’s correlation, we found that none of
the metrics showed promising correlations for the Bonsai model,
while the ProtoNN model demonstrated a correlation of 0.5 for its
computational time. In order to investigate the lack of adequate
results, we ran a second series of experiments in a similar fashion
but with smaller memory sizes from 250MB to 950MB (in steps of
50MB). Using Spearman’s correlation for the Bonsai model with
smaller memory sizes, we found that RAM energy consumption
showed the most promising correlation value of 0.75. While for
ProtoNN model with smaller memory sizes, both the RAM energy
consumption and total energy consumption showed promising
correlation values of 0.63 and 0.71 respectively.

Incorporating Hardware Metrics to Model Cards to Evaluate the Utilization of Machine Learning Models on IoT devices

From 2021 to 2022 participation at Carderock in the Congressionally mandated Defense Organizational Climate Survey (DEOCS) increased 76%!  Great for us, right?!  The survey is mostly multiple choice, but there are ten free response questions on the survey as well.  Five questions are the same for everyone in the Navy, and five are specific to each Command.  The 2,974 responses are 365 pages long when printed out.  Historically, a team from the Commander’s office must read through all of those responses and assign classes to each one.  The categories for each comment are: Requires Redaction, Sentiment (Positive, Negative, Neutral), Requires Follow-up Action by Command, and Risk Factor.  Then there are also 20 Sub-Factors and 27 Sub-Categories.  Sub-Factors include, Cohesion, Inclusion, Stress, Morale, etc.  Sub-Categories include Compliance, Diversity, Workload, Food, Gym, Parking, etc.  Therefore, the Commander’s Office initiated a project to evaluate the feasibility of classifying the DEOCS responses using natural language processing (NLP) models.  We successfully used the 2021 survey results and classifications to implement transfer learning on a variety of BERT, RoBERTa, and Distillbert pretrained NLP models using the Hugginface API.  We found that a model pretrained on 5 million Tweets performed best for sentiment classification.  Sentiment and all the binary classifications performed well (>90% accuracy).  We identified that if we corrected errors in only the smallest classes of each category that we can further improve the final accuracy with at 70% reduction in labor.  We found that we do not yet have enough training data to achieve acceptable acuracy for the Sub-Factor (19%) and Sub-Category (66%) multiple classification tasks. We would like to work with other Warfare centers to combine datasets for the five common DEOCS questions in order to improve performance with more data. 

DEOCS Survey Response NLP Analysis

There are large selections and offerings of online courses in the field of machine learning, computer science, programming language, database management, cloud computing, and many more. In addition to the selection of topics for courses, the courses are tailored for specific skill level and data competency, which includes whether the course provides expert level knowledge or working level knowledge in the domain to the individual completing the courses. Identifying relevant information from course description is important to recommend the correct courses to an individual for workforce development. To this end Naval Surface Warfare Center, Carderock Division (NSWCCD) implemented a machine learning and python approach that finds various online courses, classifies the course based on skill level and data competency, and recommends a selection of courses for workforce development within a given data competency and skill level. NSWCCD started with scraping the internet using python package Beautiful Soup to find courses and downloading the associated text from the webpages. The skill level required for the course was extracted from the web scraped text, but to determine data competency requires human in-the-loop and this was accomplished through Watchful. Watchful.io is an interactive solution that uses regular expression and statistical model to speed up the labeling of text data. Once the courses and the associated web-scraped text were labeled, then various NLP models were employed to classify the courses to the data competency level. The authors evaluated training separate NLP models per classification label and using single NLP model to classify for any label. In addition, the authors compared various deep learning models’ accuracy to a random forest classifier model.

Data Science Course NLP Classification

The Power Projection Using Process Technologies Tool (PPUPTT) enables data-driven tactics, techniques, and procedures (TTP) for enduring 21st Century Competition. Developed originally for exploiting fleeting opportunities in space ecosystems at scale, PPUPTT aligns with the Space Force’s “Combat Power Projection'' strategic capability. Importantly, however, PPUPTT may be adapted for use within any ecosystem of interest. Governments and military organizations need to transition from linear, kinetic approaches to a continual competition mindset. Better, faster data-driven decisions advantage one player over another. PPUPTT uses explainable process technologies which predict complex systems next-state, and identify fleeting opportunities to maneuver and increase adversary costs of operations. Process technologies are core to PPUPTT’s architecture. Developed originally in a business context for organizations to model the processes which run the business, PPUPTT uses process mining to model complex systems as temporal, unfolding processes. Using process models outputs, PPUPTT derives time series information entropy measures and depicts entropy with intuitive, engaging dashboards fueled by explainable artificial intelligence--making it easy for those without extensive data science training to use the software, understand the analyses, and communicate recommendations. Information entropy, which is linked to Ashby’s Law of Requisite Variety, is a measure of the state of possibilities within complex systems. Changes in information entropy reflect changes in the distribution (variety) of activities, which uncover planning and execution cycles inferences, cognitive state comparisons among agents, and ecosystem stimulus-response dynamics. Creative TTPs built using these measures improve the ability for blue to pursue goals by increasing blue variety of TTPs while simultaneously tarnishing red’s variety of activities. It does this by monitoring ecosystems of interest continually and providing operators with actionable insights tailored to their specific needs. Finally, PPUPTT speeds decision making and automates rote analyses, which frees up time for human sense making. PPUPTT operationalizes an aspect of complexity science by synthesizing research from noteworthy thinkers. For example, Shannon’s Information Theory and simple entropy calculation is used to identify surprising confluences of events which can be used for persistent monitoring, alerting, and decision making based upon recent and historical activity. Information entropy describes the variety of activities or state of possibilities within complex systems. Inputs and outputs are in open data standards which facilitates collaboration with adjacent systems. PPUPTT enables maneuver warfare grounded in complexity science, can be operationalized quickly, and will increase in value as more statistical insights are discovered and TTPs developed.

There is a pressing need to exploit fleeting opportunities continually--real time anticipatory ecosystem entropy offers a lens for these kinds of exploitations. We want to measure ecosystem entropy continually to make better, faster data-driven decisions which advantage one player over another. Combining entropy measures with process models, we seek to understand how dissipative structures interact with one another. Continual sliding window entropy measurements identify exploitation opportunities when a player’s ecosystem is especially chaotic or orderly. Process models provide turnkey simulation parameters which predict ecosystem next-states and identify exploitation actions. This concept may be applied to space, social media, critical infrastructure, urban, supply chain, macro-economic, natural, and geo-political ecosystems, as well. Efforts to date have neither examined ecosystems in a temporal process mining context using categorical data event logs nor analyzed interactions between ecosystem players in order to anticipate outcomes and improve decision making. A highly adaptable, domain agnostic information maneuver capability is possible. It will be possible to drive entropy purposefully by anticipating ecosystem changes or inflection points and make decisions which favor one party over the other.

Power Projection Using Process Technologies Tool (PPUPTT)

Modeling for safety risk often involves infrequent events with sparse data, and there is simultaneously a low tolerance for both type 1 and type 2 errors. Frequent type 1 errors will cause a numbness to develop with respect to any warnings generated by the model, while frequent type 2 errors are a symptom of chronic underestimation of risk. Managing type 1 and type 2 errors in a balanced way is the objective of ensemble modeling. This method can help reduce type 1 errors so that a more sensitive model can be produced, which also in turn remediates the chronic risk underestimation. Ensemble modeling can leverage multiple techniques that tend to agree on the true positive and true negative results, while tending to have a diverse response with respect to false positive and false negative results. When these results are unified under a top layer model, each lower layer model's classification results are interpreted in context with all lower layer model results. The outcome of this process is a more robust model which amplifies the true positive and negative results that are common between ensemble components while deemphasizing the results that the components disagree on. This paper will review some generally accepted ensemble modeling strategies and provide a practical walkthrough of a layered ensemble process using Naval safety data and the R programming language, along with an evaluation of the limits associated with the improvement that can be expected from this technique.

Ensemble Modeling: Using Advanced Learning Techniques for Naval Risk Analysis and Prediction

Keynote Talk: Rear Admiral Douglas Small

Automating Safety of Navigation with Machine Learning Technologies

Our project focuses on using machine learning and sensor data to predict quantities such as the refractive-index structure constant (Cn2), scattering, and extinction. Meteorological sensor data including temperature, relative humidity, wind speed, and surface pressure was input as predictors into RegressionLearner in MATLAB, with Cn2 and RSSI being the response variables. We provide a comparison of all the available machine learning algorithms. Principal Component Analysis (PCA) was included to determine the number of variables that explain the most variance. This presentation covers the initial results for our optical turbulence prediction trained on datasets from our collaborators at the Hellenic Naval Academy. 

Initial Results for Optical Turbulence Prediction Using a Bagged Decision Tree Model Trained on Geographically Diverse Datasets

First-principles computer models or simulations are often used as forward models for training reinforcement learning (RL) agents because training on real-world systems is too expensive, slow, or otherwise impractical. Even the highest-fidelity models can, however, fail to account for all physical variables and these modeling errors can lead to poor performance when the simulation-trained agent is exposed to the real-world. A potential solution is to use data from the physical system to directly train a deep neural network as an approximation (surrogate) of the system’s underlying state-transition function. Given difficulties inherent in acquiring sufficient real-world training data, one advantageous approach is to first train a neural network surrogate using a first-principles model and then use transfer learning and a smaller set of real-world data to fine-tune the surrogate’s approximations. An RL agent can then be quickly and cheaply trained using the well-trained surrogate and can exhibit better real-world performance compared to an agent trained on a first-principles model alone. We propose a unique algorithm for generating training data appropriate for this type of surrogate, explain several important considerations in posing the learning objective function, describe the training techniques used to achieve outstanding performance for several surrogate models before and after transfer learning, and present results for an RL agent trained using a surrogate as the forward model. More specifically, we consider the problem of training RL agents to control an Unmanned Underwater Vehicle (UUV) in an environment characterized by current flow fields. We include discussion of how our novel data generation algorithm can be extended to work for a UUV navigating such an environment and explain how an egocentric state-space definition can improve the performance of an RL agent trained to control the UUV.

Premium content

Human On the Loop Adaptive Mission Planning

Next from NAML 2023

Evaluating Active Learning Methods for Synthetic Aperture Radar Maritime Object Detection

Similar lecture

FARLAND (Framework for Advanced Reinforcement Learning for Autonomous Network Defense)

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES