United States

The US Submarine Navy relies on SONAR for detection and classification of contacts to support navigation and other mission related tasks. The field of deep learning (DL) could be useful in this problem space given that it is the current state of the art approach for image and speech recognition. Supervised learning however is a data hungry process and many Navy problems have limited labeled data. An underwater acoustic classification problem that is likely to have enough labeled samples to investigate the application of DL is recognizing marine mammals from individual clicks. The objective of this project is to apply DL to underwater acoustic transient classification and determine how much training data is required for a DL classifier to compete with an existing baseline approach. This is a collaborative effort between the Naval Undersea Warfare Center (NUWC) Division, Newport, RI (NUWCDIVNPT) and the Naval Surface Warfare Center Carderock Division (NSWCCD). The NUWC research team was able to use a one dimensional convolutional neural network (1D CNN) to ingest raw time-series data previously isolated by a power law detector and achieve classification accuracy as high as 93.7% with minimal optimization of hyper-parameters. As expected, classification accuracy increased as more training data were used. Further analysis into the performance of the network raises concerns on generalization because classification accuracy is not consistent across class or by recording location of sample data. Findings from fiscal year 2020’s efforts highlight the importance of scrutinizing classifiers that show overall good performance, especially in deep learning applications where intractability prevents intuitive awareness of knee cases.

NAML 2021

deepBioNet: Acoustic Transient Classifier

data-limited

biologic

echolocation

marine mammal

acoustic

sonar

performance evaluation

convolutional neural network

time series

deep learning

classification

generalization

AIAI 2020 (16th International Conference on Artificial Intelligence Applications and Innovations) should be held in Halkidiki, Greece. Due to current Covid-19 situation the conference will be held virtually. Underline will host a repository of video talks and preserve them for the future. Videos will be released on Underline one day after the conference.

AIAI 2020

poster

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

Passive identification of underwater sound sources is critical for a variety of applications including surveys of marine mammals and maritime domain awareness. Currently, the Navy can collect large sets of underwater acoustic signals. The cost and challenge lies in correctly labeling the data, which is time consuming and requires subject matter experts. The advent of machine learning makes the labeling process ripe for automation into a semi-supervised workflow. We begin with a labeled set of recordings released by the Scripps Institution of Oceanography (DCLDE 2015 and 2018) from hydrophones placed offshore in Southern California and US East Coast. In DCLDE 2015, five species were identified by experts across three sites with seven hydrophones, providing substantial variability in context and acoustic content. The DCLCE 2018 data is similar, though with nine different species identified across a variety of East Coast sites. Here the intersection of these two data sets is the presence of the Risso’s Dolphin, allowing us to test a classifier for this class on both different backgrounds and with different groups of dolphins that generated the vocalizations. Given the variety of approaches that currently exist for classification, we explore the effect on labeling performance of three fundamentally different models: transfer learning with AlexNet (image processing and classification), an SVM with precomputed acoustic features, and Temporal Convolutional Networks (TCN) with processed and raw timeseries data. Specifically, we explore generalization of these methods (robustness to different backgrounds or placement contexts) when trained in one environment and tested in another. Additionally we studied background suppression strategies and the effect of the sample distribution used to train the model. We found that generalization of the TCN and SVM model was dependent on mitigating the background and providing the correct blend of labeled training data. AlexNet transfer learning performance was found to be consistent despite changes in class balancing and the three background treatments tested.

Automated classification for multiple species of echolocating whales

NAVWAR's Digital Transformation Learning Continuum Program supports the Commander's strategic goal to rapidly build, strengthen and exercise data analytics capabilities across the NAVWAR HQ and affiliated PEO workforces. The Program delivers NAVWAR-mandated training for all hands (Foundations for Data Citizens) and Supervisors (Foundations of Data Analytics). Additionally, the Program offers a direct, "hands-on" workforce experience with advanced analytics through a theory-to-practice Workshop (2-Day), Laboratory (Case Study) and Incubator (Competency/PEO-led Applied Analytics Projects) development progression.


NAVWAR Digital Transformation Learning Continuu

Source localization and characterization are important tasks of passive sonar systems. The variability of the ocean environment alters the propagation of the acoustic signal and therefore affects the overall accuracy of passive sonar if not accounted for. To achieve more accurate results, this study presents a deep learning approach to seabed classification using the noise from surface ships on a single hydrophone. Ship noise spectrograms are used as input for a convolutional neural network to classify between 34 types of seabeds. This work aims to determine the optimal time duration and frequency band of the ship noise spectrogram to predict seabed type with 3-layer convolutional neural networks. The performance using both 1-dimensional and 2-dimensional convolutional kernels is compared. The networks are trained on synthetic data created with a broadband source model for the ship noise and a range-independent, normal mode propagation model to simulate propagation in the shallow ocean for the 34 seabed classes and a wide range of ship speed and closest point of approach. Training is augmented with a broadband model for the wind noise. The neural networks are verified using five-fold cross validation, and then tested using data from a 2017 seabed characterization experiment. Seabed class predictions are compared to measure the impact of altering the time window and frequency band of the input spectrograms. Finding the optimal spectrogram size for use with these networks can improve the overall accuracy and efficiency of using surface ships for seabed classification and, thus, improve passive sonar. [Work Supported by the College of Physical and Mathematical Sciences, Brigham Young University]


Finding Optimal Spectrogram Length 
for CNN Seabed Classification from Surface Ship Noise

Digital maps are important in many naval applications for situational awareness and decision making. Attempts have been made to automate digital map generation, but these regularly fall short in quality and usefulness. For example, digital maps often contain labels that can get occluded at certain zoom levels or as new layers are added. Algorithms that optimize label placement are computationally complex, and automated solutions must rely on heuristic or statistical methods to choose how the labels are distributed. Completely automated solutions are often inferior to maps designed by trained experts and still require extensive editing by an analyst. To address this challenge, we describe an interactive map generation approach that leverages cognitive models and interactive machine learning to improve label placements over purely manual or automated solutions. 

Interactive machine learning (IML) creates a continuous loop of online feedback to improve a machine learning model. These approaches start with little to no data and iteratively improve through interactions with the user. This is useful for digital maps, where optimal label placements often depend on features that cannot always be clearly defined. Using IML, analysts can adjust labels in a map view, supplying the algorithm with feedback the algorithm can use to identify the relevant features and improve its recommendations over time. Due to digital maps having a large feature space, IML may require a great deal of analyst feedback before the generated maps are considered acceptable.

We propose using a cognitive model to train an IML algorithm, decreasing the amount of required time and effort needed from real users. Cognitive architectures, such as ACT-R, are commonly used in human-computer interaction research to simulate human behavior and to generate synthetic training data for machine learning. Cognitive architectures provide a framework for modeling behavior using established cognitive models of perception and memory. This is relevant in determining label placements, because an analyst's decision of where a label should be placed is based on perceptual information (i.e., the location and density of map colors and textures) and heuristic rules about when labels should be moved. A cognitive architecture can simulate an analyst's decision processes about label placements using cues such as saliency, spatial distribution, and color density. Using defined procedural rules, the architecture simulates an analyst iteratively identifying poorly placed labels or cluttered areas and then moving labels as needed. By using a cognitive model to simulate an analyst making label adjustments on a digital map, the IML algorithm can be initialized with some basic information about features important for generating good label placements, without taking an analyst's time. In this way, the system generates better initial recommendations, while still continuing to be fine-tuned by analysts going forward.

Interactive Map Generation with Cognitive Models of Label Placement

The MK 18 program of record uses unmanned underwater vehicles (UUVs) equipped with Side Scan Sonar (SSS) to survey the sea floor looking for ordinance. The SSS data is reviewed by operators to look for ordinance. It is well known that machine learning algorithms are used to automatically detect the ordinance in the SSS data either though post mission analysis or in real time. What is less known and what this talk will cover is how the Unmanned System Simulator (USS) uses ML and AI to create simulated SSS embedded with ordinance to train the operators that evaluate the data. This presentation will describe a novel approach to insert realistic objects into SSS imagery using an AI/ML algorithm to provide operator training and algorithm training

The Unmanned System Simulator: using AI and ML to create simulated sonar data and mine like objects to train fleet operators.

Despite existence of large amounts of unclassified documentation, describing military engagements such as TTPs, historic books etc there is a shortage of formal event data that can be used for M&S purposes. We propose a hybrid AI/ML methodology to extract low-level event data from documents and aggregate them to higher level of abstraction that M&S systems can operate on. The same exact methodology can be used to extract low-level event data in real-world situations from text messages that soldiers exchange on the battlefield to “light the simulation decision tree”, thus linking simulation and actual events together. 


Acquiring realistic training data for modeling and simulation(M&S)

In order to achieve large-scale acceptance of ML/AI across the Navy and DoD, algorithms must be efficient and work well with available hardware. Currently, a large portion of the ML/AI workload is dominated by cloud-based server farms (Google, Amazon, etc.), where massive datasets and computational resources can be leveraged. However, many DoD applications cannot use these services due to limitations in memory and connectivity. Recent growth in the Internet of Things has led to the commercial development of low-SWaP hardware platforms specifically designed for many flavors of ML, including GPUs, neural network accelerators, and neuromorphic processing cores. However, the underlying operation of ML/AI algorithms on these platforms are not well understood. 
Warfighting of the future will be waged not only with bombs and bullets, but also with smart systems through information warfare at the tactical edge. The use of ML/AI as part of information warfare is an area of projected growth for the Navy and DoD. While there is a strong growing workforce with experience in the software and computer science aspects of ML/AI, implementation of this technology into DoD systems will require expertise in the implementation of ML/AI hardware as well. An understanding of the hardware platforms currently available, along with the current trajectory of S&T efforts, will allow scientists and engineers to implement systems with ML/AI technology currently being developed improved SWaP.
The decision of which hardware platform to deploy and implement specific ML/AI algorithms is not always clear. They each have their own advantages and limitations that allow for their maximum use and performance for training and inference but there are relevant metrics such as performance, power efficiency, and usability to aid in a successful implementation of research for real world applications. This paper will explore some of the unique capabilities and advantages of familiar platforms including the NVIDIA Jetson Nano, Raspberry Pi with the Intel Neural Compute Stick, and Google Coral Dev Board. 


Evaluating ML/AI Edge Computing Devices for Real World Applications

The Predictive Cost of Readiness Engine (PCORE) expands on the Digital Aviation Readiness Technology Engine (DARTE) by combining financial information with traditional readiness data (manning, training, and equipment) to focus on predicting an innovative new aircraft-level readiness metric for the Naval FA-18 fleet: return on investment (ROI). For this metric, the investment into an aircraft is defined as the cost of people (maintenance man hours) and parts, while the return from an aircraft is flight hours. A temporal pattern attention mechanism enhanced long short-term memory (LSTMA) network is used to predict future aircraft ROI. Predicting ROI allows maintainers and leadership to more accurately make decisions regarding part priority, whether to perform costly maintenance actions, and how best to allocate finite resources such as material, labor, and funding.

Improving the Efficiency of the Naval FA-18 Fleet Using The Predictive Cost of Readiness Engine (PCORE)

Machine Learning Approach to Corrosion Maintenance Identification

Underreporting fleet corrosion maintenance records results in the data consumers, analyst, and decision makers with inaccurate information to pursue corrosion mitigations in order to maintain equipment mission readiness. The NAVAIR methodology of recording equipment maintenance entails completing a Visual Information Display System/Maintenance Action Form (VIDS/MAF). The VIDS/MAF includes underutilized fleet narratives that describe the equipment discrepancy and the corrective maintenance action. This research investigates whether an unsupervised machine-learning algorithm can process fleet text narratives and identify the corrosion maintenance records.

The research proceeded by employing Natural Language Processing (NLP) to the VIDS/MAF unstructured text data. The researchers acquired data comprising of a single piece of NAVAIR equipment that contains a generous quantity of both corrosion and non-corrosion related maintenance records. The unstructured maintenance text was preprocessed utilizing lemmatization, special character removal, lowercasing characters, removing punctuations and numbers, and stripping white space. The researchers applied a Latent Dirichlet Allocation (LDA) topic algorithm to the processed unstructured text iterated across a sequence of topic quantity hyper-parameters. A harmonic mean function identified the optimal number of hyper-parameter topics. The researchers applied the LDA algorithm with the optimal number of hyper-parameter topics against the dataset. This LDA model identified additional potential corrosion maintenance records by a factor of five.

The objective of this project consisted of determining whether NLP techniques can identify corrosion maintenance records beyond the capability of the structured fields found in the VIDS/MAF. The research was successful in identifying additional probable corrosion maintenance records. Furthermore, this research has the potential for utilization in analogous applications, such as identifying underreported wiring issues that is applicable across the Navy enterprise.


Machine Learning Approach to Corrosion Maintenance Identification

This approach shows the use of robotics operating system (ROS) framework to improve the processing time performance of stochastic optimization systems as they are applied to large data sets for robotics applications. As a case study, we implement the genetic algorithm using ROS to optimize its processing time performance. Genetic algorithms are commonly used to find optimal solutions to optimization problems with different levels of complexity, however, they require lengthy data processing time when applied to large data sets. By making use of ROS action messages we can perform parallel processing in asynchronous mode which allows us to breakdown the processing of large datasets into smaller ones which can then be computed in parallel at separate servers therefore greatly reducing computation time. Another advantage of using ROS action messages, is their asynchronous aspect in which they communicate results among clients and servers therefore allowing the client to perform other tasks while waiting for results from other servers. Once these data sets have been evaluated and processed, they can be put back together into a large data set, and fed back into the genetic algorithm for further optimization therefore improving the processing speed of the evaluation module for the genetic algorithm while also avoiding constraints in memory resources due to a single server handling all the data processing. While past research to improve computation time of Genetics Algorithms have made use of parallel processing, using the ROS framework allows us to have the advantage of having feedback about the status of each separate processing task in a dynamic manner without having to add much complexity to code development.

Premium content

deepBioNet: Acoustic Transient Classifier

Next from NAML 2021

Automated classification for multiple species of echolocating whales

Similar lecture

Transfer Learning using Convolutional Neural Network Architectures for Brain Tumor Classification from MRI Images

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES