United States

Artificial Intelligence (AI) Safety is currently an area within the field of computer science with a strong emphasis on developing safe algorithms for autonomous vehicles such as self-driving cars and unmanned aircraft. As AI is integrated into more and more systems, the hazard analysis process has to be expanded to account for potential mishaps involving these new technologies. A better understanding of the capabilities of AI is needed to create safe systems. This work will give an overview and discuss some of the current development in AI Safety across the Department of Defense.

NAML 2022

Designing Safety into AI Enabled Systems

causal factors

hazards

risk

The Conference on Uncertainty in Artificial Intelligence (UAI) is one of the premier international conferences on research related to knowledge representation, learning, and reasoning in the presence of uncertainty. It is supported by the [Association for Uncertainty in Artificial Intelligence (AUAI).](https://www.auai.org/)

The 37th Conference on Uncertainty in Artificial Intelligence (UAI 2021) is held fully online. We have an exciting program consisting of keynote talks by [Susan Murphy](https://www.auai.org/uai2021/program?block=A), [Eric Horvitz](https://www.auai.org/uai2021/program?block=B), [Lenka Zdeborová](https://www.auai.org/uai2021/program?block=C), [Judea Pearl](https://www.auai.org/uai2021/program?block=D) and [Ankur Moitra](https://www.auai.org/uai2021/program?block=E), a town hall meeting, peer-reviewed papers, and three workshops. The main conference is single-track and each paper is show-cased with a live component (either a live discussion and Q&A, or a live lightning talk). All papers are also featured in video presentations and in poster sessions in gather.town. The three parallel workshops on Friday allow for smaller meetings on current topics.

The walkthrough video below shows how you can navigate this site and use the different possibilities for interaction: UAIsland2021 on gather.town for posters and social interactions, the mementor platform for virtual mentoring sessions, and the chat and Q&A systems for discussions about talks and posters.

Have a great UAI 2021! 
<br>
<br>Cassio de Campos and Marloes Maathuis
<br>UAI 2021 Program Chairs
<br>uai2021programchairs@gmail.com

<figure class="video_container">
  <iframe src="https://screencast-o-matic.com/watch/cri0lsViAGK?v=6&ff=1&title=0&controls=1" width=526  height=226  frameborder="0" allowfullscreen="true"> </iframe>
</figure>


A ticket is required to access this conference. Please use the provided link to register for the event.

https://auai.org/uai2021/registration

UAI 2021

The Conference on Uncertainty in Artificial Intelligence (UAI) is one of the premier international conferences on research related to knowledge representation, learning, and reasoning in the presence of uncertainty. It is supported by the [Association for Uncertainty in Artificial Intelligence (AUAI).](https://www.auai.org/)

technical paper

##### For full functionality of Underline, please SHUT OFF your VPN, or use a personal device.


Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

To spread awareness of current machine learning research relevant to Naval applications

To connect machine learning researchers with experts in military requirements

To build and strengthen collaborations within the defense research community


<figure class="video_container">
  <iframe src="https://s3.amazonaws.com/pf-user-files-01/u-59356/uploads/2022-03-22/hk13mxo/NAML_ReceptionWithtitle.mp4" width=826  height=526  frameborder="0" allowfullscreen="true"> </iframe>
</figure>

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

To spread awareness of current machine learning research relevant to Naval applications

To connect machine learning researchers with experts in military requirements

To build and strengthen collaborations within the defense research community



MITRE ATLAS is a knowledge base of adversary tactics, techniques, and case studies for machine learning systems based on real-world observations, demonstrations from machine learning red teams and security groups, and the state of the possible from academic research.

ATLAS enables researchers to navigate the landscape of threats to machine learning systems and is modeled after the MITRE ATT&CK® framework. The tactics and techniques in ATLAS are complementary to those in ATT&CK.

ATLAS can be used to:
- Develop case studies for your internal red teaming exercises
- Build defensive threat assessments from ATLAS TTPs
- Inform your security analysts of emerging threats to AI systems
- Report real-world adversary use of AI/ML techniques
- Understand adversary behaviors and mitigate threats to AI systems 

In this talk, we'll describe the ATLAS framework, give real-world examples of it's use, and give recommendations for based on our experience for future discovery of vulnerabilities in AI systems and related projects.

ATLAS (Adversarial Threat Landscape for AI Systems)

Deep Learning (DL) systems provide greater accuracy than earlier Machine Learning (ML) methods, but at the expense of needing large amounts of training data and being prone to overfitting, as well as a misclassification “brittleness” to particular noise patterns and certain natural object composites. These limitations make DL systems particularly vulnerable to cyber-attack, and unsuitable in their current form in cyber warfare. An architecture and respective approach is described in this paper that provides improved resiliency to DL systems, as well as accelerated learning for limited training data environments. This is accomplished through several composite techniques: matched pairing of DL classifiers with other AI/ML methods, separation or isolation layers using containers executing on a microkernel and mitigating misclassifications by using a cognitive reasoner that envelopes heterogeneous classifiers. A composite architecture incorporating these methods should provide the necessary resilience for cyber warfare environments.

A Resilient ML Architecture Approach

The Navy and DoD are prioritizing the rapid adoption of AI across domains in warfare to keep the technological advantage in the United States’ favor. Machine learning, the basis of much of the recent advancement in AI, suffers from a persistent and inadequately addressed critical flaw: adversarial examples. Since the initial discovery of adversarial examples in DNN classifiers in 2013, many new forms of these attacks have been invented and many narrow and particular defenses have been proposed. None of the defenses has survived counter testing. Some researchers have suggested that such susceptibility may be an unavoidable characteristic of problem formations in the space. At this time, no one effective, computationally acceptable, general approach has been discovered
which can shore up DNNs against this and related generalization problems. It is our contention that ML model robustness and resistance to the broad class of all ML adversarial examples can be improved with approaches that seek to train models to declare the data-point-sparse latent-space between data-dense regions of a model’s classification space as a barrier class. Identifying a general approach to training neural networks such that adversarial examples are either eliminated or computationally prohibitive representsa "holy grail" in the machine learning community. While we do not claim to have discovered this holy grail, we are collecting growing evidence that our intuitions hold valuable insights into the problem space which we believe will eventually bear significant fruit in the form of a viable general defense against all evasion adversarial attacks.

VALIDnet: A General Defense against Adversarial Examples in Neural Networks

Deep learning models are pervasive for a multitude of tasks, but the complexity of these models can limit interpretation and inhibit trust in their estimates of confidence. For the classification task, we investigate the induced geometric relationships between the class conditioned data distributions with the deep learning models’ output weight vectors. We propose a simple statistic to characterize the ’confidence’ of the model given a new input, which we call Angular Margin. We compare and contrast the statistic to respective Softmax outputs. We demonstrate that angular margin provides a simple yet superior statistic for detecting minimum-perturbation adversarial attacks and/or misclassified images than standard softmax predictions.

On Dogs and Cats and Adversarial Attacks

Advances in computational technologies and Artificial Intelligence (AI) methods present new opportunities for developing automated tactical decision aids to support warfighters making weapons engagement decisions. Tactical decisions become increasingly complex and can overwhelm human decision-making as threats increase in number, speed, diversity, and lethality. The deployment of such AI-enabled decision aids must consider system safety. This study explored the potential safety risks and failure modes that may arise as automation and AI technologies are introduced and implemented to support human-machine weapons engagement decisions. The Naval Postgraduate School is studying safety risks and failure modes, root causes, and mitigation and engineering strategies to prevent, address, and recover from this new class of possible safety failures.

The study includes five research initiatives: (1) a holistic study of safety risks, root causes, and mitigation strategies related to the introduction of AI into warfare decision aids, (2) analysis of the decision risk involved in human-machine teaming for making weapon engagement decisions, (3) analysis of human-machine trust for AI-enabled weapon engagement decisions, (4) an evaluation of the safety risks in implementing AI as a decision aid for air and missile defense decisions, and (5) a study of military data gathering and management activities and how they relate to AI safety. This technical talk will discuss the five research initiatives and report on their findings to date.


Safety in Artificial Intelligence-Enabled Warfare Decision Aids

We are applying a combined approach that utilizes generative adversarial networks (GANs) and transformers towards a general solution for data deconflation. Data deconflation expands on classical blind source separation problems, such as that of the Cocktail Party Problem, to superposition of data objects not confined to signals. Separating the conflated data objects (e.g aligning voices to their speakers, separating information from multiple co-located sensors, etc) into their individual sources allows for meaningful analysis of the data. Although classical solutions have proven successful in a certain problem set, it is unclear whether these solutions can be generalized. Our current approach works with a simple illustrative example for subsequence separation from a conflated sequence. The GAN’s objective is to be able to generate each subsequence with a label identifying its source, while being able to differentiate between properly and improperly generated sequences. We will present on our current status and the full network architecture and discuss application to more real-world examples. 

[1] Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017). Improved Training of Wasserstein GANs. arXiv [cs.LG]. Opgehaal van http://arxiv.org/abs/1704.00028 

[2] Hallman, R., & Cybenko, G. (2021). The Data Deconflation Problem: Moving from Classical to Emerging Solutions. In Proceedings of the 6th International Conference on Internet of Things, Big Data and Security. Special Session on Artificial Intelligence for Emerging IoT Systems: Open Challenges and Novel Perspectives. SCITEPRESS - Science and Technology Publications. https://doi.org/10.5220/0010530403750380 

[3] Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to Sequence Learning with Neural Networks. arXiv [cs.CL]. Opgehaal van http://arxiv.org/abs/1409.3215

Machine Learning for Data Deconflation: Adopting GANs for Generalized Source Separation

Given a LiDAR point cloud structure, performing navigational tasks such as obstacle avoidance, can be computationally expensive and cumbersome due to massive nature of 3D point clouds. In this work, we leverage 2D Multi-agent Reinforcement Learning(MARL) and features extracted from the point cloud such as bullseye locations to aid our path planning process in a correspondence 2D floor plan. We change the environment dynamically with adversarial threats such as fire or leakage -- the RL agents can quickly find a new path given the learned environment. The method shows strong performances in both throughput and latency.

Navigation in 3D point cloud environment via multi-agent reinforcement learning in a correspondence 2D floorplan.

Automated spectrum spectrum management requires real-time feedback from sensors in the sensors in the networknetwork. This real-time feedback requires a large backhaul as the network scales. We employ machine learning classifiers at the edge to reduce the backhaul reduce the bandwidth requirements of the network.

Machine Learning and Radio Frequency Spectrum Management

Baselining Few-Shot Object Detection on Maritime Data

The explosion of the number of connected devices in the last 10 years has led to a considerable increase in the nature and scope of Artificial Intelligence at the Edge (EdgeAI). Furthermore, hardware, software, and mission use cases have come together to drive several advances in AI supporting Department of Defense applications. During this talk we will describe how Deloitte views EdgeAI as a domain, deployment considerations for EdgeAI technology, and how EdgeAI can impact the challenges faced by the Navy. 

Edge technologies are characterized by a combination of sensor and computation that are networked to achieve an effect. Adding Machine Learning as a software component can give these systems the ability to deliver insights in millisecond time scales. The combination of Edge technologies and deployed Machine Learning models forms an EdgeAI system. 

The Department of Defense has made considerable investment in Artificial Intelligence, and the Department of the Navy has contributed to that effort significantly. Myriad use cases in the core mission domain of the US Navy exist for EdgeAI systems. For example, computer vision models deployed to wearable technologies can provide immediate “walk-through” assistance to Sailors afloat as they work on shipboard weapons systems and power plants. Unmanned systems can similarly be equipped to find and highlight potential issues during maintenance inspections using an entire sensor array. 

Several considerations become important when conducting AI/ML development for Edge applications. Our team uses the latest compute stack for training and deploying models to Edge environments, and we will highlight some of those considerations during this talk. Finally, we will show work that our team has done to address use cases of interest to the Navy mission, demonstrating the art of the possible in the EdgeAI space.

Designing Safety into AI Enabled Systems

Downloads

Next from NAML 2022

ATLAS (Adversarial Threat Landscape for AI Systems)

Similar lecture

Statistically Robust Neural Network Classification

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES