One of the major issues surrounding the adoption of machine learning in applications where accuracy is critical is the lack of trust in such systems due to being persistently perceived as black boxes. In this presentation, we discuss the importance of independently measuring the competency of a machine learning system and how that builds user trust. We define competency as the expected performance of a ML system for a given set of environmental conditions. We describe how to measure competency based on environmental conditions, what those conditions are and how they can be defined. We give examples of user-friendly competency statements and define competency metrics. Given that maritime environmental conditions can change rapidly, we also discuss ways to decide how often to update the user on changes in competency. We then introduce BAE System's MindfuL™ software which produces competency statements. We show results for our defined competency metrics, namely that for a given set of environmental conditions considered in a competency statement, the actual performance of the system will fall within the competency statement’s confidence interval over 99% of the time. 

The second part of the presentation will focus on the reliability of a machine learning system in the context of competency and expands on how this helps build user trust. In other words, we consider the system’s ability to behave as expected as based on the stated competency out in the real world. If there is an expected performance and set of resulting actions taken by the system for a given set of conditions, reliability is the likelihood of achieving that expected performance and resulting actions. Finally we conclude with thoughts on additional ways to build user trust via MindfuL™ software.


Measuring the competency and reliability of ML Systems

Machine learning requires large amounts of data. Despite the extensive availability of mission data collected from joint missions, military doctrine regularly compartmentalizes data to prevent accidental disclosure of operational details. Current procedures to consolidate data from different sources are manually intensive and vulnerable to insider manipulation. We desire solutions for cooperative secure learning that maintain accuracy while both protecting sensitive data sources and containing insider threats.

In this presentation, we describe a layered approach to secure, multi-party learning, in which each step in the learning process adds more security, and demonstrate its use on sensitive Security Operation Center (SOC) data. We will describe the components and flow through the system that allow for this layered security, for both single-party and multi-party learning. Our approach first combines automated anonymization and synthetic data generation to protect sensitive data with minimal loss in accuracy. We then extract feature embeddings that resist reverse engineering for sharing with the learning system via three options for privacy preserving models, and describe the training and testing process. Finally we show both performance and privacy metrics to prove this method's success in maintaining security without sacrificing performance. In most test cases, similar or better accuracy is achieved, with overall labeling accuracies ranging between 85% and 99%.

Finally, we conclude with discussion on how this work can be applied outside of cyber data, and how it can be used to make machine learning more robust in general.

Secure, Privacy-Preserving Machine Learning

Mindful™ software provides human-understandable competency statements in the DARPA Competency Aware Machine Learning program. MindfuL™ software models environments across ML applications compactly with hierarchical Dirichlet processes allowing it to provide environment based-competency assessments through deep learning and deep explainer models as well as competency of similar historical data.

Jen Sierchio

3

5

SHORT BIO

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES