United States

Congress recently created middle tier acquisition (MTA) programs, which provide the military services rapid prototyping and fielding pathways with new program flexibilities and an explicit schedule constraint. The services are executing multiple MTAs, resulting in a set of MTA experiments related to development, execution, and governance. There is little published information on MTA performance; we used public data to quantify planned schedules. We introduce a quantified schedule risk measure based on Monte Carlo simulations. The simulations provide insights into MTA program schedule risk and program performance relative to a statistically-based reference.

NAML 2022

Estimating Middle Tier Acquisition schedule risk

schedule simulation

middle tier acquisition

monte carlo

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers.<br><br><br>
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities.  This includes talks to attend in person, talks to attend remotely, breaks with colleagues  and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).<br><br><br><br>

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


AAAI 2023

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

poster

##### For full functionality of Underline, please SHUT OFF your VPN, or use a personal device.


Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

To spread awareness of current machine learning research relevant to Naval applications

To connect machine learning researchers with experts in military requirements

To build and strengthen collaborations within the defense research community


<figure class="video_container">
  <iframe src="https://s3.amazonaws.com/pf-user-files-01/u-59356/uploads/2022-03-22/hk13mxo/NAML_ReceptionWithtitle.mp4" width=826  height=526  frameborder="0" allowfullscreen="true"> </iframe>
</figure>

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

To spread awareness of current machine learning research relevant to Naval applications

To connect machine learning researchers with experts in military requirements

To build and strengthen collaborations within the defense research community



Ransomware is a rising threat that utilizes malicious software that prevents a user from accessing computer files, systems, or networks demanding a ransom for their return. This poster introduces a novel technique to measure the impact of a ransomware attack. Specifically, factors such as the ransom paid, recovery time or duration, and critical infrastructure targeted, are used to define the impact of an attack. To this end, a dataset of 980 ransomware attacks worldwide from October 2013 to June 2021 is analyzed and a general framework is established. The final results show that organizations in the five lifeline critical infrastructures (water, energy, transportation, communication, and information technology) have the largest impact scores. Moreover, this new metric aims at appraising the effectiveness of cybersecurity measures at a given organization.

Measuring the impact of a ransomware attack

Today’s investments in aircraft and Naval Aviation Support Equipment (SE) sustainment lack a coordinated response. Current metric sustainment reports address aircraft or SE top degrader equipment separately. To our knowledge there is no current association on the SE degrader investments that will have the greatest impact on aircraft readiness. This research will allow for the transition of these findings into an integrated metric data product report.

Naval Aviation Support Equipment Impact on Aircraft Readiness

Rapidly advancing fields of software technologies and the development of software autonomy frameworks present new opportunities and challenges for the DoD. While these technologies allow the DoD to rapidly augment capabilities, integration complexity increases, which oftentimes overwhelms Autonomy Practitioners, or the human end users of autonomy frameworks. This paper describes the utilization of Research and Autonomy Innovation Development Environment and Repository (RAIDER), a Navy maintained repository of autonomy frameworks, intended to leverage the promise of rapid integration while mitigating integration complexity. In particular, RAIDER encourages coordination of acquisition efforts through the reuse and repurposing of existing products, which ultimately improves system interoperability and frees up resources (e.g. staff, schedule). The acquisition strategy and tools laid out in this paper can be applied to implement new mission capabilities and integrate technologies such as machine learning on existing hardware platforms.

Streamlining Software Acquisition Processes Through the Use of an Autonomy Framework Repository and Environment

We would like to present a use case very relevant to most predictive analytic situations in military data environments where high predictive accuracy is required with less-than-ideal data integrity (limited data availability, missing data, inconsistencies, specialized vocabulary, small training data sets, etc.)

In our presentation we’ll illustrate a proprietary algorithm, Higher Order Low-Resource Learning (HO-LRL™) which is based on a suite of technologies facilitating learning models on both dynamic and static data by transforming data representations into a “higher order” format based on latent relationships between data items. This transformation can be incorporated into a number of data agnostic algorithms providing a significant functional and exponential advantage over currently used algorithms in the NLP field when confronted with limited data on a project. 

Data quality, consistency and integrity remains a common challenge across Artificial Intelligence use cases. This is especially true within Natural Language Processing (NLP). Most algorithms and state of the art machine learning methods rely on large numbers of training examples to provide high accuracy percentages. 

Modeling from a previous predictive analytics project Serco performed for customers in the defense industry, Serco and Intuidex partnered to develop a use case to better address low data environments. The previous project presented various challenges around data quality, integrity, with minimal ability to use Natural Language Processing against unstructured data providing context for generating predictions. 

Our use case replicated these conditions in the lab to test Intuidex’s technology and innovations in algorithms. The data was randomly generated using a similar data profile and only 0.2% of the original data volume provided to compare against various front-runners for this use case. Intuidex technology was able to overcome the roadblocks previously experienced through data cleansing, validation, utilization of innovative algorithms such as high order learning and low-resource learning, predictive analytics, and pattern of life modeling. 

In our use case we compared the HO-LRL algorithm with the algorithm from the original NAVSEA implementation custom developed by a team of data scientists, along with standard Stanford NLP GloVe, Facebook AI RoBerta, and Google AI BERT. The largest benefit noted was in the amount of data required to reach a higher accuracy measure which was 0.2% of what was used by the algorithm in the original implementation, along with other competitors. Accuracy and MAPE also improved dramatically by at least 13 precision points over the original and competitors while requiring exponentially less data (180 samples minimum versus 3 million plus used in original use case), training time (<20 seconds), and time to implement the solution (2 weeks). 

Our use case focused on predictive maintenance, but our findings supported related previous work, where it was shown that leveraging higher order dependencies improves the performance of a number of different applications including threat detection in streaming message traffic, anomaly and threat detection in network traffic, classification of radio nuclear signals in border protection, as well as applications in ecommerce, law enforcement, and counterterrorism. 

In conclusion, our use case illustrates how the HO-LRL algorithm provides a significant advantage when confronted with limited data which can be a very common situation on projects in military environments using Natural Language Processing (NLP). 


Rapid Exploitation of Human Language in a Low-Resource Environment

Electromagnetics Aircraft Launch System (EMALS) Anomaly Detection

Detection of Anomalous Merchant Vessel Trajectories

Traditionally, data is labelled by those willing to do the work. This can lead to large amounts of available training samples, but often falls short as popular models can take tens of thousands of labelled data points to train. With active learning, the burden of labelling is minimized. The human provides targeted answers to decrease uncertainty in the model. With the human and model working together, labelling automation through prediction is enabled. 

Active Learning: Super Charge Data Labelling

AI-Enabled Naval Tactical Kill Chain

One Enterprise Platform from the Cloud to the Edge | From securing against Cyber Kill Chain Attacks to enabling Machine Learning Systems | Key objective is to illustrate how the islands of Machine Learning activities across the Navy can be aligned with the evolution of mission areas to a DevSecOps way of delivery new software capability via the industry leading Enterprise Kubernetes platform, Red Hat OpenShift Container Platform. The end result is better information for the Warfighter to understand, decide and act (i.e. close kill chains).

Closing Kill Chains by Combining MLOps and DevOps

Forecasting algorithms allow for predictions of future changes of systems. A forecasted Inventory quantity can empower leadership to preemptively adjust funded resource levels via a forecasted data demand signal. Naval Aviation Support Equipment have significant Ready for Use deficits that have a direct impact to their Naval Aviation support. Current decisions made to manage these resources are based on historical and current data reports. With predictive information that allows for proactive resource management, plans for mitigating and handling deficits can be developed before readiness is impacted. 
Current research is focused on utilization of univariate forecasting algorithms such as Auto Regressive Integrated Moving Average (ARIMA) and Exponential Smoothing. These algorithms are applied to time series values such as Ready for Use (RFU) and total on hand. The data is split into a training set consisting of between the first 70%-80% of the data to build the models. The models are then used to forecast a time series equivalent in length to the remaining 30%-20% test set, and the model forecast is compared to the test set. Algorithm performance is judged by deriving accuracy measures such as Mean Absolute Percentage Error (MAPE) and Root Mean Squared Error (RMSE). Currently, more than half of forecasts performed on the metrics of the selected pieces of support equipment have a MAPE of less than 20%, however several have had MAPE values of over 100%. In order to improve forecasting performance further analysis of the time series will be performed. For instance grouping time series with common behaviors using clustering algorithms will be used to generalize insights of forecasts. Another objective is to examine other attributes of the candidate support equipment to determine additional factors for unexpected changes in the trends of the time series and identify opportunities for optimization.
The objective of this project is to develop a forecasted inventory quantity to enable decisions makers to more proactively manage resources to improve readiness. This research is ongoing, and the investigators plan to develop several well-defined models forecasting support equipment to include the data preprocessing pipeline to facilitate downstream model adoption.

Premium content

Estimating Middle Tier Acquisition schedule risk

Next from NAML 2022

Measuring the impact of a ransomware attack

Similar lecture

Q-Functionals for Value-Based Continuous Control

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES