United States

Recent budget constraints require the Naval Aviation Enterprise to maximize readiness while minimizing cost. As such, it is important to know which aircraft are the most expensive to keep in a mission capable (MC) state and what factors affect this cost. An AI/ML model is developed as an extension to the Digital Aviation Readiness Technology Engine (DARTE) family of models to determine the factors that lead to a high average cost per day of MC for an aircraft in the Naval FA-18 fleet. This model then predicts if the average cost per day of MC will be above a fixed cost threshold in the future. Various model architectures are explored, including tree-based ensemble methods and long short-term memory (LSTM) networks. These models are trained on a combination of traditional readiness datasets (manning, training, equipping) and financial datasets. The resulting models are then examined with explainable artificial intelligence (XAI) techniques to obtain a rigorous and comprehensive feature importance which may be used to determine past and predict future aircraft status.

NAML 2021

Understanding and Predicting the Future Cost of Readiness for an Aging Naval FA-18 Fleet

predictive analytics

readiness

finance

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

poster

The Strike Force Interoperability Office (SFIO) assembles and filters large quantities of data from a number of sources to produce Task Group readiness briefs, providing guidance in assisting mission planners with their recommendations. This process is historically manually intensive and significantly impacts personnel workload and time better spent conducting analysis.

PRIOR, a cloud native application, provides:

Secure and reliable semi-automated data driven analysis which leads to time and cost savings
Flexible microservice architecture and use of Natural Language Processing (NLP) techniques which support the transformation of large quantities of disparate data to assist with mission readiness assessment.


Planning Resources for Interoperability Options and Readiness (PRIOR)

The current state of machine learning and artificial intelligence mirrors where digital technology was 20 years ago; systems are being developed without rigorous testing and with limited (if any) security mechanisms. This lack has created challenges across both the DoD and industry with respect to the amount of trust in the safety, robustness, and performance of intelligent systems, resulting in degraded capabilities and technological utility for the warfighter. We argue for early, often, and specific consideration of security and robustness as part of a comprehensive DevSecOps process – one that incorporates tooling, methodology, and changes to acquisition strategy.

Critical Gaps in Trusted AI

We discuss the Chinese strategy in AI research in development. It is trying many things simultaneously with the weakness of not doing any of the adequately. Much of their development is top-down and they lack a culture of innovation, both of which may impede progress.

China's Artificial-Intelligence Strategy: Human Neural Networks

NAUTICA, a NISE funded workforce development project, intends to become a catalyst in development of AI talent across the Navy. NAUTICA, a loose acronym for "Naval AI Unity Training and Challenge", will offer a NIWC Pacific designed/customized AI/ML curriculum tailored towards our budding DoD AI community throughout FY21 Q2 and Q3. NAUTICA team will provide a series of online/live and recorded training lectures to provide its students in practical AI know-how in Unity ML Agents (includes Reinforcement and Imitation Learning), OpenAI Gym, genetic algorithms, Statistical Forward Planning algorithms, Monte Carlo Tree Search/AlphaZero, NEAT/HyperNEAT, NGTS integration of trained agents, supervised/unsupervised learning, neural networks, explainable AI and more. Weekly exercises will be provided following each of the lectures, as well as technical AI support. Graduates of training will be awarded certificates of completions. 
After conclusion of the series of Unity courses, we will then hold the NAUTICA Challenge for participants to train ML agents to compete within a simple wargame environment during the summer of 2021 (FY21Q4). Certificates are awarded to all participants with tournament performance statistics. Trophies will be awarded to the top three winners. The details about the NAUTICA Challenge are still in planning and early design, but we are considering using the Unity RIDE environment developed by USC ICT, and is available to all DoD government employees and FVEY partners. 


NAUTICA Naval AI Unity Training and Challenge Poster

Text localization and recognition in 3D point cloud environment are challenging tasks. In this paper, we introduce a method to adapt 2D text detection and recognition techniques in panorama images with appropriate map-ping; combined with some heuristic methodologies such as artificial camera to create a fast and efficient 3D text localization and recognition system. The simplistic nature of this system is attractive and computationally inexpensive because it focuses on 2D processing instead of 3D. The system shows promising results
with over 90% accuracy for detection, localization tasks and 95% on recognition task.

Unsupervised Text Recognition And Localization From 3D Point Cloud Via
Feature Clustering In Correspondence 2D Projection

As sensor and algorithmic capabilities improve, understanding the world in 3D is becoming an ever greater focus of computer vision. For the Naval community, there is a growing treasure trove of shipboard data available for developing new capabilities. Several commands are actively scanning ships for purposes of installations and maintenance using Light Distance and Ranging (LiDAR) and photogrammetry. They are collecting geometrically accurate 3D point cloud data sets and associated 2D images. We present some applications of this data and provide an overview of our own research efforts -- which are separately submitted to NAML as individual deeper dives. We remind the community about the existence of this data and our ongoing collection in hopes of inviting further collaboration.

Naval Applications of Shipboard Point Cloud Data

Adversarial examples are inputs to a machine learning system intentionally crafted by an attacker to fool the model into producing an incorrect output. These examples have achieved a great deal of success in several domains such as image recognition, speech recognition and spam detection. In this paper, we study the nature of the adversarial problem in Network Intrusion Detection Systems (NIDS). We focus on the attack perspective, which includes techniques to generate adversarial examples capable of evading a variety of machine learning models. More specifically, we explore the use of evolutionary computation (particle swarm optimization and genetic algorithm) and deep learning (generative adversarial networks) as tools for adversarial example generation. To assess the performance of these algorithms in evading a NIDS, we apply them to two publicly available data sets, namely the NSL-KDD and UNSW-NB15, and we contrast them to a baseline perturbation method: Monte Carlo simulation. The results show that our adversarial example generation techniques cause high misclassification rates in eleven different machine learning models, along with a voting classifier. Our work highlights the vulnerability of machine learning based NIDS in the face of adversarial perturbation.

Evaluating Evasion Attack Methods on Binary Network Traffic Classifiers

In our research, we successfully created a custom neural trojan by modifying the weights of the model itself. Additionally, we successfully defended against this attack by adding a TensorFlow Extension to encrypt and hash custom checkpoints used in the training and testing of the model. 


Designing Secure Machine Learning Systems

Predictive analytics offers a potential game changing capability for Naval tactical decision superiority. Tactical operations could take a significant leap in progress with the aid of a real-time automated predictive analytics capability that provides predictions of second and third order effects of possible courses of action. This future capability would accompany current developments in the use of artificial intelligence and data analytics to improve battlespace knowledge and offer automated battle management aids to the tactical warfighter. As the automated battle management aids develop tactical course of action options the predictive analytics capability could predict how the adversary might respond to each course of action option. The predictive analytics capability could continue to “wargame” possible blue force/red force actions and responses—generating predictions of second and third order effects. These predictions offer the tactical warfighter a more strategic perspective in making tactical course of action decisions. By performing this analysis using an automated aid with artificial intelligence, it allows the capability to sup-port real-time decisions and to analyze great amounts of data (both sensor data and historical data) and handle highly complex tactical environments This real-time wargaming translates into high order computations that would be impossible to be performed manually in the short reaction times given. This paper discusses the results of a study of predictive analytic capabilities in the naval maritime do-main.

Predictive Analytics in the Maritime Domain

In recent years, there have been significant parallel advances in two areas: laser systems and artificial intelligence (AI). The Navy is rapidly moving forward with the development and deployment of laser systems as weapons and sensors. In order to realize their full potential, laser systems must adapt to their environment and be utilized in concert with existing kinetic weapons and tactical sensors for effective engagements and military operations. Decision-making regarding the use of laser systems will be challenging in complex environments. AI methods, and automation in general, including machine learning, data analytics, and predictive analytics have the potential to create a “cognitive” laser that can assess a threat situation in real-time and adapt itself to overcome environmental challenges, select a specific target aimpoint, and tailor its beam to specific target characteristics.

The Naval Postgraduate School (NPS) is developing concepts for a cognitive laser to support the operation of shipboard laser weapon systems in the maritime domain. NPS is conducting a model-based systems engineering (MBSE) approach to understanding the laser system’s complex decision space, studying human-machine teaming methods, developing and analyzing models of decision aid concepts, and performing proof-of-concept demonstrations of machine learning methods. This presentation will cover the NPS research team approach as well as discussing concepts, progress, and preliminary analysis results.

Premium content

Understanding and Predicting the Future Cost of Readiness for an Aging Naval FA-18 Fleet

Next from NAML 2021

Planning Resources for Interoperability Options and Readiness (PRIOR)

Similar lecture

Forecasting Obsolescence Model Optimization

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES