United States

The future success of the Navy will depend, in part, on artificial intelligence. In practice, many artificially intelligent algorithms, and in particular deep learning models, rely on continual learning to maintain performance in dynamic environments. The software requires adaptation to maintain its initial level of performance in unseen situations. However, if not monitored properly, continual learning may lead to several issues including catastrophic forgetting in which a trained model forgets previously learned tasks when being retrained on new data. The authors created a new framework for safely performing continual learning with the goal of pairing this safety framework with a deep learning computer vision algorithm to allow for safe and high-performing automatic deck tracking on carriers and amphibious assault ships. The safety framework includes several features, such as an ensemble of convolutional neural networks to perform image classification, a manager to record confidences and determine the best answer from the ensemble, a model of the environment to predict when the system may fail to meet minimum performance metrics, a performance monitor to log system and domain performance and check against requirements, and a retraining component to update the ensemble and manager to maintain performance. The authors validated the proposed method using extensive simulation studies based on dynamic image classification. The authors showed the safety framework could probabilistically detect out of distribution data. The results also show the framework can detect when the system is no longer performing safely and can significantly extend the image classifier’s working envelope. Future follow-on work will include increasing the fidelity of the images to mimic views from the Integrated Launch and Recovery Television.

NAML 2023

Assurance for Continual Learning Systems

domain shift detection

out of distribution data

validation and verification

machine learning

continuous learning

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

NAML 2021

poster

### Welcome to the 7th annual workshop on Naval Applications of Machine Learning!

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

**General Chairs:**

Katie Rainey, Naval Information Warfare Center Pacific

Nicole Isoda, Naval Information Warfare Center Pacific<br><br>

<button name="button" onclick="window.open('https://sites.google.com/go.spawar.navy.mil/naml/','blank')">**NAML 2023 Website**</button>

<br><br><br>

To access the site please register [**here**](https://events.r20.constantcontact.com/register/eventReg?oeidk=a07ejmzkc5g71584a82&oseq=&c=&ch=). 

The 7th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


Bats have evolved highly advanced biosonar systems that combine information encoding in the physical domain by virtue of complex ear motions with neural extraction of information. In order to emulate this behavior, prior research has approximated the dynamics of bat pinna using fixed deformation patterns, e.g., by virtue of a lever deforming a silicone replica of a bat pinna. However, these systems could capture little of the known and suspected variability in the pinna motions that may allow bats to adapt to different sensing situations in their natural environments. As a first step towards exploring the use of variability in the acoustic characteristics of the bat pinnae, we have designed a deep-regression network that can predict the
beampattern of deformed pinnae. To enable this, we have rigged a 3D tomographic reconstruction model of a hipposiderid bat’s (Hipposideros pratti) pinna with three tendons that were used to test 10 actuator states each. All resulting shape deformations were then exported into a frequency-domain BEM model which predicted the ultrasonic beampatterns as function of the actuator states. Finally, we have reproduced the BEM beampattern prediction with two deep neural network architectures: a deep radial basis function neural network and a multilayer perceptron neural network. Both architectures were able to reproduce the BEM results. Since inference is computationally much less costly than BEM simulations, these deep-learning models for the link between pinna actuation and acoustic properties can be used for finding ways to harness the variability in the bats’ pinna shapes using expensive training methods such as reinforcement learning.

Deep-Learning Models of the Dynamic Periphery of Bat Biosonar

For general-sum, n-player, strategic games with transferable utility, the COCO value provides a computable method to both 1) quantify the strategic value of a player; and 2) make cooperation rational through side payments. We give a simple formula to compute the COCO value in normal-form games. Next, we provide two methods to generalize the COCO values to stochastic (or Markov) games, and show that one of them may be computed using generalized Q-learning algorithms. Finally, an empirical validation is performed on stochastic grid-games with three or more players.

N-Player COCO-Q Reinforcement Learning

Cyclostationary features such as high-order moments and cumulants are often applied in detecting and classifying digitally modulated signals. However, these predictor variables may not be mutually uncorrelated, raising the concern that potential multicollinearity may lead to redundant information and significantly affect quality prediction. This work explores machine learning and statistical methods to mitigate multicollinearity issues. We first perform multicollinearity diagnostics by evaluating variance inflation factors and eigendecomposition of the correlation matrix of the predictor variables. To combat multicollinearity, we apply feature transformation using the principal component analysis (PCA), a traditional dimensionality reduction technique that has been well-studied and is suitable for small feature sets. Our empirical results show that the chosen diagnostic tools effectively detected cyclostationary multicollinearity. Future work will include assessing PCA against existing methods for selection of moments and cumulants.

Multicollinearity Diagnostics on Cyclostationary Features

Navy divers are exposed to health risks while operating in marine waters contaminated with a wide variety of bacterial, viral, and protozoan pathogens. Timely and accurate analysis of measured bacterial levels to predict illness is rarely possible due to the costs involved. Available data integrated through the means of knowledge graphs can be transformed into knowledge that can help divers avoid exposure to biological hazards. A knowledge graph (also known as an ontology) is a graph-based data model that can express and integrate structural knowledge from multiple domains in the form of (head-entity, relationship, tail-entity) triples, where the entities represent nodes in the graph and the relationships represent edges that connect entities. Knowledge graphs can encode complex rules which are useful in defining and reasoning about the semantics of the entities and can lead to new knowledge and discoveries. For example, given the following two triples in a knowledge graph (Pseudomonas aeruginosa, causes, acute respiratory infection), (acute respiratory infection, isa, communicable disease), one can infer that (Pseudomonas aeruginosa, causes, communicable disease). While knowledge graphs can be useful in inferring new knowledge, extracting triples from large-scale data manually can be a tedious task and the resulting graphs can suffer from incompleteness. Currently, there are many knowledge graphs or ontologies readily available, including a water quality ontology (DOCE), an epidemiology ontology (EPO), an infectious disease ontology (IDO) and an exposure ontology (ExO), among others. However, there is no knowledge graph that links water quality to health risks posed by water activities such as diving. To fill in this gap, we have focused on the construction of a knowledge graph that links water quality to health risks by leveraging existing ontologies and large-scale data available in PubMed. We constructed a knowledge graph based on semantic triples automatically extracted from the Semantic MEDLINE database (SemMedDB), a PubMed-scale repository of biomedical triples. The entities in the SemMedDB triples correspond to Metathesaurus concepts in the Unified Medical Language System (UMLS), specifically concept unique identifiers (CUIs), while the relations are drawn from the UMLS semantic network. Using a set of manually identified CUIs that we found to be most relevant to water quality and health risks, we constructed a graph of approximately 75,000 semantic triples which contain at least one entity from the manually identified set. We subsequently used the constructed knowledge graph to study machine learning approaches for knowledge graph completion (KGC), to help identify new valid triples or new relationships between existing entities. Specifically, we experimented with the KG-BERT approach for graph completion by employing pre-trained BERT (Bidirectional Encoder Representations from Transformers) and RoBERTa (Robustly Optimized BERT Pre-training Approach) models and also equivalent models fine-tuned on a collection of water quality and health-risks abstracts retrieved from the Web of Science. Experimental results showed that the KG-BERT approach with BERT/RoBERTa models fine-tuned on the domain-specific corpus improves the performance of KG-BERT with pre-trained models. Furthermore, the best results overall were obtained with a fine-tuned RoBERTa model, which has an accuracy of 88.61% on the valid triple classification task.

Leveraging Existing Literature and Machine Learning to Build a Knowledge Graph Focused on Water Quality and Health Risks

Recent operational challenges have prompted a renewed interest in mission-engineering tools that can advance decision-making and innovation in defense. These tools have the ability to simulate high-fidelity combat engagements to analyze operational concepts, platforms, systems, and capabilities in multiple domains to develop optimal solutions to the intricate and difficult issues within the field of mission engineering. One such tool is the Advanced Framework for Simulation, Integration, and Modeling (AFSIM). AFSIM is a standard combat simulation tool that includes a variety of applications and plugins that can be used for systems engineering as well as engagement, mission, or campaign-level simulation. Due to software challenges, using AFSIM in combination with Reinforcement Learning (RL) poses particular challenges that must be overcome before RL will gain widespread use in the AFSIM community. This study proposes a novel way to utilize synthetic data created through AFSIM by using offline, or data-driven, RL.

Data-Driven Reinforcement Learning for Mission Engineering and Combat Simulation

We constructed a scenario using a previously developed framework to educate junior developers on the importance of incorporating statistical analysis in the development of autonomy behaviors. We added rudimentary perception and obstacle avoidance to the previously developed framework, adding variable parameters for the range the autonomous agent could perceive its environment, the braking response curve, and the maximum speed through which the agent could traverse its enivronment. By simulating the agent avoiding obstacles through an environment it has no a priori knowledge of and analyzing the frequencies with which the agent either collided or nearly collided with obstacles, we were able to identify patterns in the design parameters. For example, One-way ANOVA testing showed a statistically significant difference in collision avoidance success rate between the different braking curves across identical sensor ranges and maximum speeds (p=7.88e-24). Additionally, thresholds for agent survivability were identifiable by analyzing the sensor ranges and maximum speeds allowed in simulations. When combined with other core development best practices such as unit testing and version control, we believe we can better define intended behaviors and consequently improve the reliability of the behavior by conducting relevant statistical analysis on the design parameters of behaviors.

Learning How To Learn: Emphasizing Iterative Design in Good Autonomous Framework Development

The objective of this research project is to improve the warfighter’s maritime domain situational awareness by designing and developing a prototype payload for deployment on an unmanned or dismounted operator’s platform in support of various missions. The payload would have the ability to detect, track, and identify Signals of Interest (SOIs) by utilizing Convolutional Neural Networks (CNN) and Digital Signal Processing (DSP) techniques. The 1st year of this effort entailed developing a benchtop prototype capable of performing real-time SOI classification. The components of the prototype consisted of a Software Defined Radio (SDR) to stream the raw digitized In-phase and Quadrature (I/Q) components of the signal and a System on a Chip (SoC) development kit to perform real-time DSP and SOI classification via the CNN. The SOIs were classified based on modulation type, wireless protocol, and emitter type. Two network architectures were developed using a standard image approach and a complex value I/Q approach to evaluate the benefit of I/Q input format. DSP techniques such as power normalization, down sampling, up/down converting, and time averaging were implemented to elicit better generalization and accuracy of the trained CNN. This research project effort has been selected to continue into a second year. Future work will focus on Low Probability of Intercept (LPI) signal as the SOI and applying machine learning to both detection and classification of low and negative Signal to Noise Ratio (SNR) signals.

Spectrum Awareness Persistence Payload

Synthetic data generation is crucial to performative and predictive autonomy technology research. The efficient generation of robust synthetic data requires 3D Computer-Aided Design (CAD) models that correctly represent the target but sufficiently small as to promote agile computation. "Watertight" CAD models - models with no gaps or surface flaws - are necessary for computer simulations, generation of synthetic datasets for machine learning, and accurate synthetic data analysis. This is joint work with Dr. Igor Ternovskiy (AFRL/RYAT) and Ms. Elizabeth Sudkamp (AFRL/RYAP).

A Facet to Watertight CAD Converter

Naval Aviation Support Equipment (SE) PMAs lack an automated methodology to produce a failure mode and system identification. An automated capability would facilitate a consistent, repeatable, and time savings process across the sixteen SE PMA’s portfolio of equipment. The NAVAIR methodology of recording equipment maintenance entails completing a Maintenance Action Form (MAF). Deficiencies in the MAF process allow maintainers to enter overly generic systems and root causes for the maintenance action. However, the MAF includes underutilized fleet narratives that describe the equipment discrepancy and the corrective maintenance action. This research investigates whether an unsupervised machine-learning algorithm can process fleet text narratives and identify SE Failure Modes and system affected. 

The research proceeded by employing Natural Language Processing (NLP) to the MAF unstructured text data. The researchers acquired data comprising of several pieces of equipment (contained in separate files) of NAVAIR equipment that contains a variety of failure modes and systems. The unstructured maintenance text was preprocessed utilizing lemmatization, lowercasing characters, removing punctuations and stop words, replacing acronyms/abbreviations, and stripping white space. The researchers plan to apply a Latent Dirichlet Allocation (LDA) topic algorithm to the processed unstructured text iterated across a sequence of topic quantity hyper-parameters. A harmonic mean function can be used to identify the optimal number of hyper-parameter topics. The researchers will apply the LDA algorithm with the optimal number of hyper-parameter topics against the dataset. 

The objective of this project consists of determining whether NLP techniques can identify SE Failure Modes and System affected using unstructured text fields found in the MAF. The project is currently in progress and should result in an automated solution to extract accurate labels from the text data. This solution will allow the stakeholders to identify root causes of Support Equipment failure and apply the necessary resources to rectify the failure mode quickly and more efficiently.

Unsupervised Support Equipment (SE) Failure Mode Identification

We demonstrate an end-to-end neural network approach for acoustic source detection and classification on a linear sensor array. We show that the neural network learns optimal spatial and frequency filters as it performs beamforming-based signal enhancement and finite impulse response (FIR) filtering to isolate source-specific energy signatures. The trained network shows robust performance as a function of signal-to-noise ratio.

Premium content

Assurance for Continual Learning Systems

Next from NAML 2023

Deep-Learning Models of the Dynamic Periphery of Bat Biosonar

Similar lecture

Applying a Decision Framework to Increase Machine Learning Impact on the V-22 Osprey Program

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES