United States

Accurate and efficient corrosion detection is a difficult but important issue with immediate relevance to maintenance of Naval ships. Current practice requires an inspector to search for defects in a space, however the process requires a time-consuming manual inspection and physical access to the space. Considering both the inspector&#39;s and the ship’s schedule, this is not always a straightforward process. 
There is a significant amount of research into automatic detection of corrosion via computer vision algorithms but performing pixel level segmentation introduces added difficulty. There are two key reasons for this: the lack of annotated data and the inherent difficulty in the type of problem. In this work, we utilized a combination of annotated data from a different domain and a small hand labeled dataset of panoramic images from our target domain: the inside of empty ship tanks. 
We trained two High-Resolution Network (HRNet) models for our corrosion detector; the first with a dataset outside our target domain, the second with our hand annotated panoramic tank images. By ensembling our two models, we were able to increase the F1-Score by ~120% and mIOU score by ~176% with respect to the single baseline corrosion detector. 
The data collection process via LiDAR scanning allows the inspection process to be performed remotely. Additionally, the setup of the detector leads to a natural expansion of the corrosion dataset as panoramas are continually fed through the detector and detections are validated. This allows for the corrosion models to be later retrained for potential improvement in accuracy and robustness.

NAML 2023

Shipboard Tank Corrosion Segmentation via Deep Learning 

corrosion detection

lidar

segmentation

computer vision

deep learning

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

NAML 2021

Accurate and efficient corrosion detection is a difficult but important issue with immediate relevance to maintenance of Naval ships. Current practice requires an inspector to search for defects in a space, however the process requires a time-consuming manual inspection and physical access to the space. Considering both the inspector's and the ship’s schedule, this is not always a straightforward process. 
There is a significant amount of research into automatic detection of corrosion via computer vision algorithms but performing pixel level segmentation introduces added difficulty. There are two key reasons for this: the lack of annotated data and the inherent difficulty in the type of problem. In this work, we utilized a combination of annotated data from a different domain and a small hand labeled dataset of panoramic images from our target domain: the inside of empty ship tanks. 
We trained two High-Resolution Network (HRNet) models for our corrosion detector; the first with a dataset outside our target domain, the second with our hand annotated panoramic tank images. By ensembling our two models, we were able to increase the F1-Score by ~120% and mIOU score by ~176% with respect to the single baseline corrosion detector. 
The data collection process via LiDAR scanning allows the inspection process to be performed remotely. Additionally, the setup of the detector leads to a natural expansion of the corrosion dataset as panoramas are continually fed through the detector and detections are validated. This allows for the corrosion models to be later retrained for potential improvement in accuracy and robustness.

poster

### Welcome to the 7th annual workshop on Naval Applications of Machine Learning!

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

**General Chairs:**

Katie Rainey, Naval Information Warfare Center Pacific

Nicole Isoda, Naval Information Warfare Center Pacific<br><br>

<button name="button" onclick="window.open('https://sites.google.com/go.spawar.navy.mil/naml/','blank')">**NAML 2023 Website**</button>

<br><br><br>

To access the site please register [**here**](https://events.r20.constantcontact.com/register/eventReg?oeidk=a07ejmzkc5g71584a82&oseq=&c=&ch=). 

The 7th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


An overview of the NAVWAR Data Science Learning Program discussing course available to all Navy personnel

NAVWAR Digital Transformation Learning Program

Neural networks and ensemble techniques achieve state-of-the-art results but often incur a considerable resource cost. Decision trees, in contrast, require comparatively fewer resources, to the detriment of model performance. To bridge this gap, soft trees are introduced. Soft trees represent a novel variant of decision trees which incorporate additional contextual information into the predictions. Under this methodology, predictions comprise a summation of leaf values weighted by threshold distance and probability. Such predictions provide a significant improvement over the performance of standard trees while retaining trees’ reduced-resource properties. Soft tree models may then allow applications to realize the resource benefits of using a decision tree while also producing improved results within a constrained environment.

Soft Trees: Smoothing Decision Thresholds for Improved Performance

Unsupervised Degrader Ranking

An algorithm that can accurately rank Naval Aviation Support Equipment (SE) top degraders is a desired product to allocate resources effectively. This project utilized Factor Analysis of Mixed Data (FAMD), Isometric Mapping (ISOMAP), and Magnetic Properties and Correlation (URMC) to calculate the ranking of top SE degraders for two different Program/Project Manager (PMA) programs. The current methodology for ranking top degraders relies on a z-scores and Subject Matter Expert (SME) provided weights used by the existing degrader ranking products. The goal of the project would be to optimize the degrader ranking solution to use a method that is less subjective in nature and more flexible, which can be automated and applied across a variety of equipment.

Research to date in this project tackled the main issue by looking into two ways to calculate degrader ranking in an unsupervised fashion: FAMD and ISOMAP. Factor Analysis of Mixed Data reduces the dimensionality of both categorical and numerical data to that of N-dimensions specified by the user before finding a best way to fit all data points on a line in each dimension. ISOMAP, like FAMD, reduces the dimensionality of the data, but in a non-linear methodology, and is only used for numerical variables that are statistically significant. For both algorithms, data obtained from the current degrader products is reduced to a varying range of two to four dimensions while being assigned a degrader percentile label (Rank/Total) to aid in the algorithms’ potencies.

At this point in time, only a FAMD implementation has been fully tested, and with reduction to three components, can explain 84% of the incoming data from the current degrader product’s rankings. Additional exploration and research is needed to evaluate the URMC algorithm’s performance and comparison against both FAMD and ISOMAP models.

Unsupervised Degrader Ranking

Longitudinal data, which is widely used in many disciplines to study cause and effect, poses significant computational challenges to both modeling and analysis. Longitudinal data is composed of readings on the same variable collected over time and is often high-dimensional with correlated features. The combinatorial search approach for identifying the optimal features is unrealistic for most applications. The alternative approaches, such as heuristics, greedy searches, and regularization techniques, including LASSO, can result in models that suffer from both low accuracy and unclear feature attribution. In this presentation, we propose a binary transformation on the data before applying LASSO for feature learning. As demonstrated in the paper, the binary transformation enhances signal in the data, resulting in highly accurate feature attribution, including associated time lags. It avoids the typical shortcomings of the LASSO algorithm, including saturation of the feature space and arbitrary or inconsistent sparse feature selection. Both synthetic data and real-world data sets were used to demonstrate the value of the proposed transformation and in every case substantial improvements in feature learning were seen. In addition, the scalability of the solution is superior to that of the standard LASSO since an integrated high performance computing (HPC) solution would utilize integer math for most of the computational effort.


LASSO Logic Engine: Harnessing the Logic Parsing Capabilities of the LASSO Algorithm for Longitudinal Feature Learning 

GPS has proven to be a valuable, highly accurate Positioning, Navigation, and Timing (PNT) solution in many military applications. Its susceptibility to jamming and interference, however, has sparked a great deal of research in PNT alternatives to address vulnerabilities in GPS-denied environments. In this study, we utilize micro-electro-mechanical system (MEMS) IMU sensors mounted on different parts of the body to estimate movement speed and position. MEMS IMUs are small and light enough to be wearable on a person and can accurately estimate orientation, but fails to accurately estimate position. To address this, we create a system that uses GNSS, motion capture, and inertial data to train a machine learning speed estimator. The estimated speed is paired with a main IMU and an extended Kalman filter to estimate position. We will then compare our results with two other IMU-based navigation techniques: step counting and zero-velocity update algorithm (ZUPT). Step counting works well for normal walking or running, but fails for more complex movements like crawling, strafing, or walking backwards. ZUPT requires high end IMUs mounted on the foot, which can be very cumbersome. We believe our proposed method addresses these problems and performs more accurately.

Machine Learning Aided Gait Recognition for Inertial Navigation and Orientation (MARIO)

Carrier Flight Deck Information Updates using Speech Recognition 

Naval aviation assets such as aircraft, catapults and arresting gear are tracked through information systems that provide situational awareness to sailors on aircraft carriers. The current concept of operations requires highly skilled workstation operators to manage everything from aircraft status to flight schedules in order to keep information up to date. Information is given to the operator through personnel on the flight deck by using an audio communication pipeline. This requires the operator to be dedicated to listening to updates and manually entering what they hear into workstations. Many times, there are loud noises from jets and other equipment, which will only make the operator’s job more difficult. This project proposes a way to read into the audio communication pipeline and completely automate the process of adding information into computer systems using automated speech recognition models. 

The research in this use case accomplished acquiring a model that can understand the standardized phrases said by carrier personnel and correctly identify the written equivalent of what was said. There were three Transformer-based models in total that were acquired/trained to test accuracy on pre-generated flight-deck phrases: Speechbrain, Whisper, and NemoASR. While Speechbrain was fine-tuned on our flight deck phrases, NemoASR and Whisper were used in the base forms to test relative accuracy. In order to bolster their accuracies, research was also conducted into noise cancelling and speech enhancement pipelines that almost completely erase all background noise and make the spoken instructions as loud as possible, so the model has a low fail rate. These pipelines are possible by analyzing the Short-time Fourier Transforms of the audio in question and running them through a Long Short-Term Memory (LSTM) or Generative Adversarial Network (GAN) to erase unwanted frequencies from the audio. 

The goal of this project was to obtain a high enough accuracy in providing a written equivalent to an audio flight deck phrase. At this moment in time, the team has a word error rate of less than 5% utilizing the state-of-the-art model, Whisper. While the team is content with the accuracy of the model, they are considering conducting additional fine-tuning on both NemoASR and Whisper to get as close to a perfect model as possible while working on syncing the model with carrier flight deck information systems.

Carrier Flight Deck Information Updates Using Speech Recognition

Video sensors are ubiquitous in the realm of security and defense. Successive image data from those sensors can serve as an integral part of early-warning systems by drawing attention to suspicious anomalies while leveraging object detection, computer vision, and machine learning. Automating some of those detection and classification tasks aids in maintaining a consistent level of situational awareness in environments with ever-present threats. Specifically, the ability to detect small objects in video feeds would help people and systems to protect themselves against far away or small hazards. This work proposes a way to accentuate smaller features in video stills by using pixel subtraction of surrounding frames to augment the current input frame. Our method allows us to highlight changing pixels and extract motion information from video data that can be used to boost detection capabilities for objects of interest.

Frame-Differencing Object Detection Enhancement in Videos

As the digitization of modern life continues full steam ahead, criminals
and state-sponsored actors continue to target government systems. The impact of these attacks on the Navy range from the exposure of highly sensitive data to the disruption of services and the exploitation of back doors for future attacks. Modern cybersecurity teams have to defend on-premises and multi-cloud footprints. Effective threat detection and response are challenged by the cost and complexity of siloed tools that neither scale nor provide capabilities for advanced threat detection. These advanced threats are causing federal organizations and agencies to reevaluate their security posture and architectures being used to enable the
analysis of petabytes of data in real time and apply advanced analytics to reduce their cyber risk and operational costs.

Databricks - Defending Hybrid Cloud Environments with Advanced Threat Detection through Machine Learning

Automatic target recognition (ATR) is a critical task in a wide range of naval applications such as surface vessel protection and port security via radar, sonar, and optical sensor data processing. Data-driven methods, such as deep learning (DL), have been successfully applied to a variety of ATR tasks. Despite their strong performance, current state-of-the-art DL ATR applications are reliant upon aggregating data from several devices to a central server for training. This process can be computationally costly and presents a potential security risk if the data itself is sensitive. Moreover, current state-of-the-art DL ATR models also produce point estimates at test time with no associated measure of uncertainty. Uncertainty quantification (UQ) on model predictions is critical for accurately characterizing and assessing the reliability of model outputs. In this research, under the Office of Naval Research (ONR) Threat and Situational Understanding with Networked-Online Machine Intelligence (TSUNOMI) program, we develop a novel ATR framework, which is capable of both producing UQ measurements and eliminating the need for data centralization. This novel approach begins by considering an ensemble of ATR models that are jointly utilized to produce model predictions and UQ estimates. The prediction given by the ensemble is characterized by the distribution of outputs from the individual models rather than a point estimate produced from a single DL ATR model. Characteristics of the ensemble, such as its prediction interval and coverage probability are used to express model uncertainty. Using Federated Learning, the ensemble is trained in a distributed manner, where each vessel with contributing data trains a local ensemble using its own computational resources. Local model parameters are periodically transmitted to a centralized location for aggregation with the other ensembles. These parameters are generally much smaller-sized communications payloads than the raw data itself by several orders of magnitude and are used to perform the same shared learning task as centralized DL. We evaluate the efficacy of this ATR framework on a prototype task and share preliminary results.

An Uncertainty Quantification-Based Automatic Target Recognition Framework Over Distributed Surface Vessels

With the proliferation and rapidly improving endurance of next-generation unmanned undersea systems, long-term deployment of human-on-the-loop subsea and seabed resident fleets are fast-approaching. More specifically, wave-generated power and other renewable power sources, autonomous docking capabilities, and increased edge processing capabilities enable subsea residence of both static and mobile underwater systems. In this research, we evaluate human-on-the-loop adaptive mission planning algorithms for managing a fleet of unmanned understeer systems in simulation. These systems could provide persistent monitoring, maintenance, and repair of critical undersea infrastructure vulnerable to grey zone operations; such a system could estimate and track infrastructure health, including detection and sensor data acquisition of significant changes. Moreover, subsea resident systems could perform continuous mine survey of a particular (even mobile) area of responsibility and conduct environmental surveys over extended periods of deployment. It is often impractical or impossible for these systems to maintain consistent contact with human operators, necessitating novel approaches to mission planning that bridges the gap between constant human oversight and fully autonomous operation of the platforms. 
These persistent, autonomous systems are expected to adapt to increasingly complex environments and tasking, collaborate with other autonomous systems, and to respond appropriately to the preferences of human operators without requiring unnecessary oversight. The mission planning algorithms used by these systems must decompose high level objectives into discrete tasks that can be described in terms of their cost to complete, and their efficacy can be evaluated by the computational time required to create a solution, as well as the efficiency of the solution itself. To study these algorithms, a multi-agent problem is posed in simulation, wherein a simulated group of systems (agents) complete a number of (potentially mobile) tasks in the shortest time possible without colliding with other agents, or with dynamic obstacles drifting through the environment. By implementing a multi-stage approach to the problem decomposition, this work proposes an algorithm that outperforms state-of-the-art algorithms, such as the Consensus-Based Bundle Algorithm, with respect to both the computational time to create the plan and the efficiency of the plan itself. 


Premium content

Shipboard Tank Corrosion Segmentation via Deep Learning

Next from NAML 2023

NAVWAR Digital Transformation Learning Program

Similar lecture

Multi-modal Dataset Generation using Domain Randomization for Object Detection

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES