United States

Naval surface vessels, both manned and unmanned, are equipped with multi-
modal sensor systems that must process and make sense of complex operating
environments for optimal mission performance and survivability. Autonomous
sensor resource management increases the effectiveness of a group of sensors and
the efficiency of each individual sensor. In this research we focus on optimizing
autonomous sensor resource management for an individual radar system. Radar
resource management is the process by which a radar system will allocate its
resources between various tasks, viz., scanning in order to detect previously un-
detected targets, as well as tracking previously detected targets. A sub-problem
of the resource management problem is radar time allocation, where the focus
is on determining how much time is to be spent on scanning for new targets,
and how much time is to be spent on tracking each target that has already been
detected. Because of the sheer volume of parameters in a radar system, tra-
ditional methods of solving these problems often involve heuristic algorithms,
or they utilize simple techniques that may achieve unsatisfactory performance.
However, as recent works have shown, advances in reinforcement learning (RL)
have opened up solutions in this space that can create significant improvements
in performance. Our ongoing joint industry-academia collaboration between
Syracuse University and Saab, Inc., which is a supplier of U.S. Navy and U.S.
Marine Corps radar systems, demonstrates the capabilities of RL to address the
time allocation problem in both a multi-target tracking setting and a scan while
tracking instance.
In multi-target tracking, we must decide on how to allocate the dwell times
of a particular radar such that tracking performance is maximized across a set
of targets in varying modes of flight operation. In the “scan while tracking”
instance, the problem is to determine the correct time allotment such that a
radar system is capable of discovering all targets in a scene while continuing to
accurately track all detected targets. We explore different RL algorithms along
with carefully designed cost functions to train agents capable of achieving state-of-the-art results on these problems. Further, we present results to demonstrate
the potential of these agents to generalize to other related problems in this
space. Simulation software has been implemented, and simulations are carried
out in near-realistic simulation environments (e.g., with targets whose velocities
change in magnitude and direction), demonstrating the potential for real-world
applications of these RL agents. We compare the RL approach with static and
heuristic time allocation strategies, especially in the context of the possibility
of sudden changes in the environment, with several new targets emerging unex-
pectedly during the radar surveillance operation, necessitating dynamic changes
to the relative magnitudes of time slots allocated to scanning and tracking var-
ious targets.

NAML 2023

Reinforcement Learning based Resource Allocation for Radar Scanning and Multi-Target Tracking

multi-target tracking

resource allocation

reinforcement learning

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

NAML 2021

poster

### Welcome to the 7th annual workshop on Naval Applications of Machine Learning!

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

**General Chairs:**

Katie Rainey, Naval Information Warfare Center Pacific

Nicole Isoda, Naval Information Warfare Center Pacific<br><br>

<button name="button" onclick="window.open('https://sites.google.com/go.spawar.navy.mil/naml/','blank')">**NAML 2023 Website**</button>

<br><br><br>

To access the site please register [**here**](https://events.r20.constantcontact.com/register/eventReg?oeidk=a07ejmzkc5g71584a82&oseq=&c=&ch=). 

The 7th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


Internet has become a ubiquitous influence in today’s age; however, internet activities have become vulnerable and detrimental due to malicious actions if not secured properly. For example, large scale data breaches such as those indicted at Yahoo and Equifax, essentially exploited the weaknesses of the traditional username/password authentication mechanism. Therefore, it has become imperative for us to find additional ways to protect our data, which also needs to be low-cost and unobtrusive. Existing authentication mechanisms can be categorized into what you know (knowledge-based), what you have (possession of physical things) and what you are (biometrics). Biometrics-based authentication can be further categorized into physiological and behavioral biometrics. Physiological biometrics are based on physiological characteristics of a person such as fingerprints, iris, facial recognition and palmprints, which can cumbersome due to the cost and usability. On the other hand, behavioral biometrics use the behavior of a person interacting with a device such as mouse dynamics (i.e., different mouse movements on a graphical user interface (GUI)), widget interactions (i.e., mouse cursor hovering over different icons or buttons on a GUI), keystroke dynamics, swipe, motion, walking or gait, all of which are inexpensive and unobtrusive techniques that can be leveraged to strengthen existing authentication without interfering with day-to-day computer operations. The experimental work of this proposal includes two novel contributions: 1) a survey of the state-of-the-art literature on mouse dynamics and widget interactions and 2) designing and validating fusion of mouse dynamics and widget interactions’ modalities on a realistic and popular website (i.e., PayPal and Facebook) utilizing our novel datasets. We performed literature survey on mouse dynamics with widget interactions along the following dimensions: tasks and experimental settings for data collection, taxonomy of raw attributes, feature extractions and mathematical definitions, publicly available datasets, algorithms (statistical, machine learning, and deep learning), fusion and performance, and limitations. Secondly, we experimented with trajectory driven window mapping to fuse mouse dynamics and widget interactions modalities using a total of 10 users for PayPal with an average of 12 minutes of data per user and a total of 15 users for Facebook with an average of 2 hours of data per user. Our mean EER rates with a score-level fusion of mouse dynamics and widget interactions are 7.64\% (SVM-rbf) and 3.25\% (GBM), for PayPal, and 5.49\% (SVM-rbf) and 2.54\% (GBM), for Facebook. We further improve the EER rates by combining decision scores from multiple consecutive trajectories, which yields a 0\% mean EER after 11 decision scores across all the users for both PayPal and Facebook

Multi-Agent Cooperative Game with a Learned Message Passing Interface

There is growing interest in the use of machine learning (ML) techniques to extract and classify signals of interest from passive sonar data. As with all machine learning applications a large body of data is required to train the ML model. Whilst a large body of open source acoustic data is available from a variety of sources and for different applications this data is often unlabelled, or labelled only for the task at hand. Marine mammal data recordings for example may be labelled with a species of interest but other sounds, such as vessels and other marine life, may not be labelled. For an effective classification model to be developed all sources in the recording must be labelled.
One solution to this problem is to manually label each recording, however the time required to create sufficient labels using this method can be prohibitive. This work follows a different approach, developing an algorithm to auto label large datasets using semi-supervised and active learning approaches. These make use of a small labelled dataset and minimal expert user interaction to create a large pseudo-labelled training dataset. Testing of pseudo-labelling has shown that it is possible to use this technique to create large training sets, however, Accuracy remains low. The introduction of active learning increases this Accuracy by guiding the model training.



Automating the Labelling of Underwater Audio Recordings

"With the rapid progression of technology, it becomes difficult for individuals to navigate the amount of data and systems at their disposal. To lessen the burden, technology can be leveraged to create an ambient intelligent environment where humans can naturally utilize systems with little prior training.  In this work, we demonstrate various technical applications that contribute to the effort of creating an environment that supports users by providing systems that enhance their performance. The features we developed allow the user to interact and control systems via hand gestures, action recognition, and verbal cues.

The capabilities have been implemented using various machine learning models and frameworks. For example, hand gesturing has been implemented with the MediaPipe posing framework, which allows for key point estimation on live camera feed. These key points allow us to identify the position of an individual’s hand in a relative space and can use this data to create interactive actions between the user and the system. Action recognition data is generated via an action classification model. This model estimates confidence scores for relevant action classification labels based on the real-time camera footage it receives as input. This data provides our system with more context about the individuals in our command centers, allowing us to tailor system interactions to specific actions. Lastly the verbal cues are implemented using a live transcript utilizing an automatic speech recognition model to transcribe live audio. This is fed into a dialogue manager which then determines the intent of the speaker using an intent classification model, which will determine what actions the dialogue manager should take or how the conversation should continue. Our goal is to expand our Human Computer Interaction (HCI) capabilities in order to achieve a fully ambient intelligent environment, which would allow its users to easily interact with the system and provide a more effective interactive experience."

Developing Interactive Ambient Intelligent Environments

Keynote Talk: Dr. Kimberly Sablon

"Abstract for the 2023 NIWC Pacific 
Seventh Annual Workshop on Naval Applications of Machine Learning:

The Strategic and Technical Landscape in 2023


Our briefing will provide a summary of the strategic and technical landscape facing NIWC Pacific, our government, academic and industry partners, as well as the DoD and the DoN in 2023. The purpose of this briefing will be to provide a stage-setter and strategic, operational and technical grounding for the discussions throughout the 2023 NIWC Pacific Naval Applications of Machine Learning Workshop.

As an organizing high concept for this briefing, it is our belief as well as that of our colleagues in the military, industry, academia and the NRDE community that if we know who our warfighters will likely have to take on, and where they will fight them, and we have a sense of when and why this fight might begin, and understand how our warfighters will work to defeat an enemy, then we will know what C4ISR, cyber, and space platforms, systems, sensors and weapons NIWC Pacific needs to deliver to our warfighters today and tomorrow. Further, we will have a strong sense of the role big data, artificial intelligence and machine learning will play in providing our warfighters with an asymmetric advantage.

Our briefing will consist of three primary segments:

•        An overview of the strategic environment, especially the dramatic change in the United States strategic calculus as articulated in the National Security Strategy, the National Defense Strategy and the Nation’s Tri-lateral Maritime Strategy, Advantage at Sea. We will address peer competitors such as China and Russia, and other threats as articulated in the “2+3” national security construct.

•        A “burning platform” overview of the strides our principal adversaries are making in big data, artificial intelligence and machine learning. This will provide an objective review regarding how crucial it is for the DoD and the DoN to advance U.S. government efforts to fully leverage big data, artificial intelligence and machine learning for military applications. We will highlight the Report on the National Security Commission on Artificial Intelligence, as well as statements by SECDEF, DEPSECDEF and other senior officials highlighting the importance of these technologies.

•        Finally, we will address what big data, artificial intelligence and machine learning can do for U.S. warfighters. It is not, as some worry about, to launch armed autonomous weapons downrange without a human in the loop. Rather, as the JAIC Director, Chief of Naval Operations, and others have emphasized, the primary thing that big data, artificial intelligence and machine learning for military applications will do for the DoD and the DoN will to enhance speed of decisions. 

We have mined all relevant information regarding DoD and DoN plans to leverage big data, artificial intelligence and machine learning to support U.S. military platforms, systems, sensors and weapons, and especially the plans evolving under the stewardship of the DoD’s Joint Artificial Intelligence Center (JAIC).

Far from a dry recitation of threats, this briefing will energize NAML attendees and motivate them to accelerate their efforts in their day-to-day work with big data, artificial intelligence and machine learning. Since this briefing will be a stage-setter for the workshop, we recommend we brief early on day one of the event.



"

The Strategic, Operational and Technical Landscape in 2023

The future of Naval Aviation is unmanned and eventually autonomous. This presentation focuses on the generation of certification for a ML algorithm to complete a task without a human in or on the loop: The autonomous aerial refueling task. With the support and cooperation of the National Airworthiness Council Artificial Intelligence Working Group I have published multiple academic papers, white papers, and several presentations on this topic. This presentation highlights my previous work area, and details a possible path forward. Standards and methods of compliance do not exist for certification officials to allow ML algorithms to operate without human oversite. Until these can be established use of ML without a human in or on the loop will remain a demonstration vice a fleet fielded system.

Standards and Methods of Compliance for Autonomous Unmanned Aerial Refueling

"Baseline efforts in Performance in Noise indicated that tasks performed in a tactical situation break down when speech intelligibility decreases. Speech intelligibility decrements can be caused by excessive background noise or cluttered audio channel communications. The objective of this effort is to investigate the development of a communication aid using combined benefits of speech recognition, speech analysis, and Machine Learning (ML) tools for enhancing and filtering communications to achieve high speech intelligibility even when communications are poor due to the addition of noise and cluttered comms. The benefit to this technology is not only improved communications in hard to hear situations, but also allows for the appropriate use of hearing protection without sacrificing ability to hear verbal communications. 
The purpose of this effort is to investigate the development of assistive technology through speech recognition and Machine Learning (ML) tools to enhance and filter communications for Navy specific environments and test the technology on the ability of Sailors to successfully complete tasks in low speech intelligibility conditions. Traditional commercial off the shelf (COTS) speech recognition algorithms are not effective with the typical sentence structure of military communications. Our current efforts are building upon previous successes to train models capable of parsing and understanding our Warfighters to provide a useful assistive technology for improving their ability to communicate in lower speech intelligibility environments.
"

Performance in Noise, Automated Speech Recognition Communications Aid

"Concerns of trustworthiness, unintended social
harms, and unacceptable social and ethical viola-
tions undermine the promise of ML advancements.
Moreover, such risks in complex ML-driven sys-
tems are often difficult to foresee, as many only
arise over periods of time, across populations, and
at scale. These risks emerge through interactions
amongst ML development choices, the context of
model use, environmental factors, and the effects
of a model on its target. Systems safety engi-
neering is an established discipline with a proven
track record of identifying and managing such
risks in high-complexity sociotechnical systems.
In this work, we apply a state-of-the-art systems
safety approach to concrete ML applications with
notable social and ethical risks to demonstrate a
systematic means to argue for safe and trustwor-
thy ML in sociotechnical systems."

Applying Real Safety to Machine Learning Problems

The task of parsing, reading and analyzing the textual and structured information contained in scanned/digital-born documents, namely Visual Document Understanding (VDU) enabled by Optical Character Recognition (OCR), is an incredibly challenging task due to the diversity of layouts and formats, the complexity of template structures, and lack of support for low resource languages, dialects, and scripts. Traditionally, methods for VDU/OCR have relied on rule-based or conventional machine learning approaches, which does not only produce inferior results but also is extremely hard to generalize and fine-tune the solution for documents of interest. To address these shortcomings, our team proposes a VDU/OCR solution enabled by a unified text-image multimodal transformer to learn cross-modal representations. Our approach leverages pre-trained model families such as Language-Independent Layout Transformer (LiLT) a for both text-based document understanding using language models, and image-based document understanding using Visual Transformers, and fine-tune them for the languages, scripts, and document formats of interest. 

Deep Learning for Visual Document Understanding in Low Resource Languages

While the recent boom in Artificial Intelligence (AI) has given rise to the technology’s use and popularity across many domains, the same boom has exposed vulnerabilities of the technology to many threats that could cause the next “AI winter." AI is no stranger to “winters," or drops in funding and interest in the technology and its applications. Many in the field consider the early 1970’s as the first AI winter with another proceeding in the late 1990’s and early 2000’s. There is some consensus that another AI winter is all but inevitable in some shape or form, however, current thoughts on the next winter do not consider secure and robust AI and the implications of the success or failure of these areas. The emergence of AI as an operational technology introduces potential vulnerabilities to AI’s longevity. The National Security Commission on AI (NSCAI) report outlines recommendations for building secure and robust AI, particularly in government and Department of Defense (DoD) applications. However, are they enough to help us fully secure AI systems and prevent the next “AI winter”? An approaching “AI Winter” would have a tremendous impact in DoD systems as well as those of our adversaries. Understanding and analyzing the potential of this event would better prepare us for such an outcome as well as help us understand the tools needed to counter and prevent this “winter” by securing and robustifying our AI systems. In this talk, Harguess will introduce the following four pillars of AI assurance, that if implemented, will help us to avoid the next AI winter: security, fairness, trust, and resilience.

Premium content

Reinforcement Learning based Resource Allocation for Radar Scanning and Multi-Target Tracking

Next from NAML 2023

Multi-Agent Cooperative Game with a Learned Message Passing Interface

Similar lecture

FARLAND (Framework for Advanced Reinforcement Learning for Autonomous Network Defense)

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES