Running machine learning analytics over geographically distributed datasets is a rapidly arising problem in the world of data management policies ensuring privacy and data security. Visualizing high dimensional data using tools such as t-distributed Stochastic Neighbor Embedding (tSNE) and Uniform Manifold Approximation and Projection (UMAP) became common practice for data scientists. Both tools scale poorly in time and memory.
While recent optimizations showed successful handling of 10,000 data points, scaling beyond million points is still challenging. We introduce a novel framework: Sketch and Scale (SnS). 
It leverages a Count Sketch data structure to compress the data on the edge nodes, aggregates the reduced size sketches on the master node, and runs vanilla tSNE or UMAP on the summary, representing the densest areas, extracted from the aggregated sketch. 


TSNE & UMAP: Sketch & Scale

This is the 3rd of four poster sessions

**Date & Time:**  Thursday March 25th,  8-9:30am PDT

**Location:** Poster Session Room 2A

The live poster sessions will happen in our Gather.Town lounge at the designated time. Each session has their own room. Once you click the gathertown button below, you will need to navigate to the Poster Session Room for each session located on the right side of the lounge. Each speaker will be by their poster where you can communicate with them and view their poster by clicking X on your keyboard.

[![](https://assets.underline.io/uploads/markdown_image/1/image/737a34c989ecf3ae95182670d99c072c.png)](https://gather.town/app/5IAViMZXC9ZF4E4l/NAML)

Poster Session 2A

poster

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

Viska Wei

1

SHORT BIO

Presentations

TSNE & UMAP: Sketch & Scale

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES