Autonomous systems have an array of specialized sensors to observe their surroundings. An autonomous platform must be able to combine many disparate streams of information in order to leverage all of the available information while creating a comprehensive model of its environment. Deep learning-based object detectors typically need to be trained on swaths of labeled data to achieve good performance. Unfortunately, collecting and labeling multimodal data in the real world is exceedingly labor-intensive. The use of video game graphics engines in the production of images and video has emerged as a relatively cheap and effective way to create new datasets. A unique aspect of using gaming engines to generate data is the ability to introduce domain randomization which randomizes certain parameters of the game engine and generation scheme in order to improve generalization to real-life data. In this paper, we outline the creation of a multi-modal synthetic dataset using domain randomization. Our dataset will focus on the two most popular sensors in autonomous vehicles, LiDAR and RGB cameras. We will discuss the effect of different parameters on each modality and perform baseline testing of an object detector using a data-fusion deep learning architecture on both our synthetic dataset and the KITTI dataset for comparison.

Multi-modal Dataset Generation using Domain Randomization for Object
Detection

Labeled real world data is often difficult to obtain and especially scarce in Navy-related domains. Currently, relevant annotated data that does exist is frequently limited to large or medium sized bounding boxes, making it difficult to train computer vision algorithms to recognize smaller objects of interest. In this work, we present a naval-specific video dataset of helicopter operations performed at sea. This dataset contains videos from multiple camera sensors to incorporate variations in lens distortions and camera noise. It consists of videos ranging from one to three minutes each recorded during Littoral Combat Ship (LCS) exercises off the California coast in the fall and winter. Special consideration was taken to emphasize small instances of helicopters relative to the field of view and therefore provides a more even ratio of small-, medium-, and large-sized bounding boxes for training more robust detectors and trackers. Following the conventions of the field, we define small, medium, and large objects as objects with bounding boxes sized: less than 32, 32-96, and greater than 96 pixels squared respectively. We benchmark these videos on object detection with special consideration given to small-object mean average precision.


Helicopter Video Dataset for Detecting and Tracking

This is the second of four poster sessions

**Date & Time:**  Wednesday March 24th,  12-1:30pm PDT

**Location:** Poster Session Room 1B

The live poster sessions will happen in our Gather.Town lounge at the designated time. Each session has their own room. Once you click the gathertown button below, you will need to navigate to the Poster Session Room for each session located on the right side of the lounge. Each speaker will be by their poster where you can communicate with them and view their poster by clicking X on your keyboard.

[![](https://assets.underline.io/uploads/markdown_image/1/image/737a34c989ecf3ae95182670d99c072c.png)](https://gather.town/app/5IAViMZXC9ZF4E4l/NAML)

Poster Session 1B

poster

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

NAML 2021

**Moderator:** [Jeff Byers](https://www.underline.io/speakers/80146-jeff-byers)

**Speakers:** 

[Kyle Rice](https://www.underline.io/speakers/77567-kyle-rice)

[Tegan Emerson](https://www.underline.io/speakers/79700-tegan-emerson)

[Neil Bomberger](https://www.underline.io/speakers/80435-neil-bomberger)

[Lena Nans](https://www.underline.io/speakers/78913-lena-nans)


Diego Marez

2

1

SHORT BIO

Presentations

Multi-modal Dataset Generation using Domain Randomization for Object Detection

Helicopter Video Dataset for Detecting and Tracking

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES