United States

A common focus across the DoD is: How can our tools and information help improve fleet readiness? It is evident that Data Science plays an immense role when it comes to decision making and the analysis of issues. The increase of data digitization only furthers these efforts. One avenue of interest is Natural Language Processing (NLP), which is a subset of Artificial Intelligence (AI). NLP focuses on developing models to understand human language. This work focuses on a specific area of NLP called Word Embeddings, which maps words to vectors of real numbers. These numerical representations of words helps enhance modeling tasks such as multiclass classification. Our use-case demonstrates how Word Embeddings were able to enhance classification results of maintenance narratives.

The maintenance narratives are free-form text that the sailors input when completing maintenance actions. The dataset of about 60,000 records is labeled and has six different categories of maintenance. The process is then broken down into two approaches, the classical approach and then the word embedding approach. The classical approach involves the textual data being broken down into tokens and forming a Document Frequency Matrix (DFM). This DFM is then used to build supervised classifiers such as Naive Bayes and Gradient Boosting Machines. This is then compared to the Word Embedding approach, which also tokenizes the textual data, but then uses it to form word vector representations of each token. These vectors are then used to build a supervised classifier that yields better results than the classical approach.

Word Embedding models allows for a deeper understanding of human language as opposed to classical approaches. It is able to absorb more context and syntactic similarity from the text. The corpora of data that is used to train it determines how the language model will behave. In our case, the model is centric to naval maintenance data. However, such work could be applied in various fields where multiclass classification would be beneficial.


NAML 2021

Maintenance Classification using Word Vectors

multiclass classification

word2vec

word embedding

# Welcome everyone to ACL 2022!

The 60th Annual Meeting of the Association for Computational Linguistics is taking place May 22-27, 2022 as a hybrid event, in Dublin and online. We are happy to welcome all of you to this anniversary edition with an almost 50-50 in-person and virtual participation. 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, three amazing keynote events and two new initiatives of invited talks: Spotlight Talks for Young Rising Stars (STIRS) and The Next Big Idea Talks. Posters (including Findings of ACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop will have an oral session and a poster session as part of Poster Session 1. The program also features eight Tutorials and 28 Workshops. 

 
We wish you a wonderful conference! 
[**The ACL 2022 Organizing Committee**](https://www.2022.aclweb.org/organisers)
 
[**Conference Handbook**](https://drive.google.com/file/d/1_BUCMfhMVrjG9E2e71aHdHeE28KSje0l/view?usp=sharing) 
[**Mini Handbook**](https://drive.google.com/file/d/1qlBKl0wzmlVF1oCeMQl3BahLd9nLP5Ce/view?usp=sharing) 
[**Posters and Demo guides**](https://drive.google.com/file/d/1UucMAoCNncIOaH1rMMDa0owuG9qgvJTG/view?usp=sharing)

ACL 2022

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing (NLP). 

poster

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

Recent budget constraints require the Naval Aviation Enterprise to maximize readiness while minimizing cost. As such, it is important to know which aircraft are the most expensive to keep in a mission capable (MC) state and what factors affect this cost. An AI/ML model is developed as an extension to the Digital Aviation Readiness Technology Engine (DARTE) family of models to determine the factors that lead to a high average cost per day of MC for an aircraft in the Naval FA-18 fleet. This model then predicts if the average cost per day of MC will be above a fixed cost threshold in the future. Various model architectures are explored, including tree-based ensemble methods and long short-term memory (LSTM) networks. These models are trained on a combination of traditional readiness datasets (manning, training, equipping) and financial datasets. The resulting models are then examined with explainable artificial intelligence (XAI) techniques to obtain a rigorous and comprehensive feature importance which may be used to determine past and predict future aircraft status.

Understanding and Predicting the Future Cost of Readiness for an Aging Naval FA-18 Fleet

The Strike Force Interoperability Office (SFIO) assembles and filters large quantities of data from a number of sources to produce Task Group readiness briefs, providing guidance in assisting mission planners with their recommendations. This process is historically manually intensive and significantly impacts personnel workload and time better spent conducting analysis.

PRIOR, a cloud native application, provides:

Secure and reliable semi-automated data driven analysis which leads to time and cost savings
Flexible microservice architecture and use of Natural Language Processing (NLP) techniques which support the transformation of large quantities of disparate data to assist with mission readiness assessment.


Planning Resources for Interoperability Options and Readiness (PRIOR)

The current state of machine learning and artificial intelligence mirrors where digital technology was 20 years ago; systems are being developed without rigorous testing and with limited (if any) security mechanisms. This lack has created challenges across both the DoD and industry with respect to the amount of trust in the safety, robustness, and performance of intelligent systems, resulting in degraded capabilities and technological utility for the warfighter. We argue for early, often, and specific consideration of security and robustness as part of a comprehensive DevSecOps process – one that incorporates tooling, methodology, and changes to acquisition strategy.

Critical Gaps in Trusted AI

We discuss the Chinese strategy in AI research in development. It is trying many things simultaneously with the weakness of not doing any of the adequately. Much of their development is top-down and they lack a culture of innovation, both of which may impede progress.

China's Artificial-Intelligence Strategy: Human Neural Networks

NAUTICA, a NISE funded workforce development project, intends to become a catalyst in development of AI talent across the Navy. NAUTICA, a loose acronym for "Naval AI Unity Training and Challenge", will offer a NIWC Pacific designed/customized AI/ML curriculum tailored towards our budding DoD AI community throughout FY21 Q2 and Q3. NAUTICA team will provide a series of online/live and recorded training lectures to provide its students in practical AI know-how in Unity ML Agents (includes Reinforcement and Imitation Learning), OpenAI Gym, genetic algorithms, Statistical Forward Planning algorithms, Monte Carlo Tree Search/AlphaZero, NEAT/HyperNEAT, NGTS integration of trained agents, supervised/unsupervised learning, neural networks, explainable AI and more. Weekly exercises will be provided following each of the lectures, as well as technical AI support. Graduates of training will be awarded certificates of completions. 
After conclusion of the series of Unity courses, we will then hold the NAUTICA Challenge for participants to train ML agents to compete within a simple wargame environment during the summer of 2021 (FY21Q4). Certificates are awarded to all participants with tournament performance statistics. Trophies will be awarded to the top three winners. The details about the NAUTICA Challenge are still in planning and early design, but we are considering using the Unity RIDE environment developed by USC ICT, and is available to all DoD government employees and FVEY partners. 


NAUTICA Naval AI Unity Training and Challenge Poster

Text localization and recognition in 3D point cloud environment are challenging tasks. In this paper, we introduce a method to adapt 2D text detection and recognition techniques in panorama images with appropriate map-ping; combined with some heuristic methodologies such as artificial camera to create a fast and efficient 3D text localization and recognition system. The simplistic nature of this system is attractive and computationally inexpensive because it focuses on 2D processing instead of 3D. The system shows promising results
with over 90% accuracy for detection, localization tasks and 95% on recognition task.

Unsupervised Text Recognition And Localization From 3D Point Cloud Via
Feature Clustering In Correspondence 2D Projection

As sensor and algorithmic capabilities improve, understanding the world in 3D is becoming an ever greater focus of computer vision. For the Naval community, there is a growing treasure trove of shipboard data available for developing new capabilities. Several commands are actively scanning ships for purposes of installations and maintenance using Light Distance and Ranging (LiDAR) and photogrammetry. They are collecting geometrically accurate 3D point cloud data sets and associated 2D images. We present some applications of this data and provide an overview of our own research efforts -- which are separately submitted to NAML as individual deeper dives. We remind the community about the existence of this data and our ongoing collection in hopes of inviting further collaboration.

Naval Applications of Shipboard Point Cloud Data

Adversarial examples are inputs to a machine learning system intentionally crafted by an attacker to fool the model into producing an incorrect output. These examples have achieved a great deal of success in several domains such as image recognition, speech recognition and spam detection. In this paper, we study the nature of the adversarial problem in Network Intrusion Detection Systems (NIDS). We focus on the attack perspective, which includes techniques to generate adversarial examples capable of evading a variety of machine learning models. More specifically, we explore the use of evolutionary computation (particle swarm optimization and genetic algorithm) and deep learning (generative adversarial networks) as tools for adversarial example generation. To assess the performance of these algorithms in evading a NIDS, we apply them to two publicly available data sets, namely the NSL-KDD and UNSW-NB15, and we contrast them to a baseline perturbation method: Monte Carlo simulation. The results show that our adversarial example generation techniques cause high misclassification rates in eleven different machine learning models, along with a voting classifier. Our work highlights the vulnerability of machine learning based NIDS in the face of adversarial perturbation.

Evaluating Evasion Attack Methods on Binary Network Traffic Classifiers

In our research, we successfully created a custom neural trojan by modifying the weights of the model itself. Additionally, we successfully defended against this attack by adding a TensorFlow Extension to encrypt and hash custom checkpoints used in the training and testing of the model. 


Designing Secure Machine Learning Systems

Predictive analytics offers a potential game changing capability for Naval tactical decision superiority. Tactical operations could take a significant leap in progress with the aid of a real-time automated predictive analytics capability that provides predictions of second and third order effects of possible courses of action. This future capability would accompany current developments in the use of artificial intelligence and data analytics to improve battlespace knowledge and offer automated battle management aids to the tactical warfighter. As the automated battle management aids develop tactical course of action options the predictive analytics capability could predict how the adversary might respond to each course of action option. The predictive analytics capability could continue to “wargame” possible blue force/red force actions and responses—generating predictions of second and third order effects. These predictions offer the tactical warfighter a more strategic perspective in making tactical course of action decisions. By performing this analysis using an automated aid with artificial intelligence, it allows the capability to sup-port real-time decisions and to analyze great amounts of data (both sensor data and historical data) and handle highly complex tactical environments This real-time wargaming translates into high order computations that would be impossible to be performed manually in the short reaction times given. This paper discusses the results of a study of predictive analytic capabilities in the naval maritime do-main.

Premium content

Maintenance Classification using Word Vectors

Next from NAML 2021

Understanding and Predicting the Future Cost of Readiness for an Aging Naval FA-18 Fleet

Similar lecture

Roadblocks in Gender Bias Measurement for Diachronic Corpora

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES