United States

Energy-Efficient Deep Neural Network Design: From Time-based Circuits to Layer-wise Pruning

https://underline.io/speakers/59497-chris-kim

NSF Workshop on Micro/Nano Circuits and Systems Design

Energy-Efficient Deep Neural Network Design: From Time-based Circuits to Layer-wise Pruning

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible. The experience for on-site participants would closely approximate a normal pre-COVID *ACL conference, with 5-6 thematically organized parallel sessions and live Q/A and interactive discussion immediately after the talks. Presentations by virtual participants will be equitably interleaved with those of on-site participants, projected on the auditorium screens as if on-site, and also followed immediately by live Q/A and interactive discussion at a time during reasonable waking hours for the virtual presenter. For all participants, on-site and virtual, who are unable to attend a session due to either time-zone issues or because they are participating in another session live, talk recordings and slides will be available online at a minimum after the live presentation (and in many cases before as well), and questions may be submitted in advance on session-specific discussion boards and answered live in session with the usual visual aids if desired.

<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/ceby/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>

Please Note: The EMNLP registration system is not currently connected to the underline site as we are still in the process of building out EMNLP 2021. You will receive access instructions from underline the week of November 1st. 

Access is given only to EMNLP upon registration, if you have not registered please do so [here](https://2021.emnlp.org/registration).

Registered attendees will receive access the week of November 1st.

EMNLP 2021

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible.

poster

 Electronic design and design automation (EDA) is a key enabling technology in the advancement of information processing that we are enjoying today. EDA has provided fundamental design principles and tools empowering the exponential growth of the number of transistors on an integrated circuit chip over the past 50 years. Without EDA, it would not have been possible for billion-transistor chips to be a reality! 

After five decades of growth, the field of EDA now faces some unique challenges. On one hand, we are approaching the physical limit of transistor scaling, and the golden days of exponential growth in transistor density are over. The exorbitant cost of integrated circuit fabrication and the excessive complexity of EDA tools make it difficult to bring design automation innovations to practice. On the other hand, the recent rise of machine learning and artificial intelligence applications and emerging architectures and technologies calls for more customized solutions to designing electronic systems. 

## **Objectives of the Workshop:**

This workshop, funded by the National Science Foundation (NSF), will assemble a diverse group of leading researchers from both academia and industry to identify future research directions in design and design automation of electronic systems and architectures in response to the challenges and opportunities. It will feature plenary talks and panels as well as roundtable discussions. Collectively, the workshop attendees will explore answers to overarching questions such as 

•	what are the high-risk and high-return research topics, 

•	what other fields adjacent to EDA research should be aggressively pursued for highly interdisciplinary collaboration, and

•	where the research funding should come from and how it should be distributed to encourage more transformative research. 

The workshop will also include a satellite event with a roundtable panel discussion on strategies to improve foundry access to US-based EDA researchers. A final report, based on the deliberations of the workshop, will be produced to the government recommending future research directions.


The Information Processing Factory (IPF) project aims to exploit self-awareness principles, together with lessons learned from
large-scale factories to contain complexity, achieve predictability and manage robust system design in the context of
autonomous Multi-Processor System-on-Chip (MPSoC) platforms in Cyber-Physical Systems (CPS) and the Internet of
Things (IoT). The goals of the project are to holistically couple key notions of self-awareness, self-organization, and large
scale factory automation to manage the multi-scale and multi-dimensional complexity of emerging MPSoC platforms that
execute dynamic workloads under a multitude of hard and soft constraints. Sensory information at multiple scales -- on-chip,
run-time systems, applications -- will need to be monitored, fused, and interpreted intelligently to assess health status,
mitigate adverse effects, and optimize system goals using a novel hierarchy of self-aware control loops that can both manage
complexity, as well as achieve guarantees. Additionally, the project aims to deploy a unique scalable distributed run-time
Major Activities:
verification (NUVA) architecture that should enable hardware-supported cross-layer monitoring of parametric specifications,
allowing both incorporation of trace-driven historical evidence for model building, as well as run-time execution for model
enhancement. The overall goals of the approach are to synergistically cover the multi-tiered abstraction hierarchy from low level
VLSI circuits to runtime OS and dynamic application software. The project will engage in a unique parallel collaboration
with DFG-funded German partners (at TU Braunschweig and TU Munich), with a dual focus on investigating self-aware IPF
principles for system-on-chip exemplars (US team), as well as for autonomous driving (German team). By leveraging the
German research partners' autonomous driving exemplar, the goal is to demonstrate self-aware IPF principles at multiple,
collaborative scales of autonomy and system design.

https://underline.io/speakers/6768-nikil-dutt

The Information Processing Factory – A Paradigm for Life Cycle Management of Dependable Systems

As Moore's law stalls, novel computing solutions are urgently needed to meet the ever-increasing computing demands from emerging applications such as modern machine learning tasks. With digital techniques running out of steam and emerging technologies pending material revolution, mixed-signal computing which combines circuit techniques in digital, analog, and time domains bring significant opportunities in the near future to dramatically improve computing efficiency.  The benefits of mixed-signal computing include high energy efficiency, compatibility with existing CMOS technology and EDA tools, and strong ties to physical world in a near-sensor or IoT environment.  In this poster, we will briefly discuss the concepts and examples of leveraging the dimensions of “time” in mixed-signal design to accelerate the computing tasks in emerging applications. 

https://underline.io/speakers/59498-jie-gu

Mixed-signal Computing for Emerging Applications

In-memory computing (IMC) SRAM has been demonstrated as a promising technique to significantly improve energy-efficiency for deep neural network (DNN) hardware accelerators. However, designing one involves setting many design variables such as the number of parallel rows to simultaneously activate, analog-to-digital converter (ADC) resolution at the periphery of a memory sub-array, precisions of various parts of a DNN model, etc., which affect energy-efficiency, accuracy, and area. Prior works have not investigated this multi-dimensional design optimization. To fill this knowledge gap, we present an SRAM-based IMC hardware modeling and optimization framework.

https://underline.io/speakers/60043-jae-sun-seo

SRAM-based In-Memory Computing for Energy-Efficient Deep Neural Networks

Bidirectional Brain Computer Interfaces play a key role in our effort to explore the human brain and hold the key to transformative changes in medicine and computing. however, significant barriers remain unaddressed by traditional, circuit-driven approaches. This research applies computational techniques to traditionally analog, mixed-signal and integrated power electronics to advance BBCI technology.

https://underline.io/speakers/59500-visvesh-sathe

Computationally Enhanced Bidirectional Brain Computer Interfaces

We discuss three key ideas for energy efficient nonvolatile memory and multistate nonvolatile synapses for neuromorphic computing :
1. Strain induced switching of magnetization in nanomagnets and MTJs.
2. VCMA control of topological state of magnetic skyrmions.
3. Strain control of domain wall position in nanoscale racetracks clocked by SOT.

https://underline.io/speakers/29802-jayasimha-atulasimha

Voltage control of nanoscale magnetism: For energy efficient memory and neuromorphic computing

This poster presents our work on rapid simulation and automated design of silicon photonics devices funded by the NSF under a CRII grant. Silicon Photonics is an exciting emerging technology platform which leverages existing CMOS Silicon-on-Insulator foundry infrastructure to guide and manipulate light on the nanoscale. Despite having great potential, silicon photonic devices are challenging to design and simulate due to the lack of analytical solutions, small wavelengths, and large structures involved. Simulations today rely on the Finite-Difference Time-Domain (FDTD) method. Unfortunately, FDTD suffers from slow convergence, poor accuracy, long simulation times, and significant numerical dispersion. The goals of this project are to couple a fast simulation method with optimization techniques to achieve rapid automated design of silicon photonics devices. To achieve this, we propose an alternative simulation approach which is based on boundary integral equations (BIE) in the frequency-domain. Our method is dispersion-free and only requires discretization of the material interfaces which reduces the number of unknowns significantly. The method is high-order accurate, achieving close to spectral accuracy, and only requires meshing at a few points per wavelength. Our poster introduces our BIE algorithms and implementations in both two and three dimensions and demonstrates recent results using our framework to rapidly simulate silicon photonic devices with high accuracy. We also demonstrate fast automated design of new, high-performance silicon photonic devices in both 2D and 3D by applying the adjoint method for computing the gradient of the objective function used for optimization. We have fabricated several of the devices optimized via our methodology and are currently in the process of measuring them to compare against our predicted simulation results. 

https://underline.io/speakers/59501-constantine-sideris




CRII: SHF: Ultra-fast Simulation and Automated Design of Silicon Photonics Devices

Associative Processing (AP) is a promising alternative to the Von Neumann model as it addresses the memory wall problem through its inherent in-memory computations. However, because of the countless design parameter choices, comparisons between implementations of two so radically different models are challenging for simulation-based methods. To tackle these challenges, we develop an alternative analytical approach based on a new concept called architecturally-determined complexity. Using this method, we asymptotically evaluate the runtime/storage/energy bounds of the two models, i.e., AP and Von Neumann. We further apply the method to gain more insights into the performance bottlenecks of traditional AP and develop a new machine model named Two Dimensional AP to address these limitations.

https://underline.io/speakers/59502-jing-li

CAREER: Associative In-Memory Graph Processing Paradigm: Towards Tera-TEPS Graph Traversal In a Box

With the advent of Internet-of-Things and the necessity to enable intelligence in embedded devices like mobile phones, smart watches etc., low-power and secure hardware implementation of neural networks is vital. Despite achieving super-human performance and classification accuracies on a variety of perception tasks, Deep Neural Networks (DNNs) have been shown to be adversarially vulnerable.  A DNN can be easily fooled into misclassifying an input with slight changes of pixel intensities. This vulnerability severely limits the deployment and potential safe-use of DNNs for real world applications such as self-driving cars, malware detection, healthcare monitoring systems etc. This poster investigates hardware aware optimization techniques to potentially resolve or resist software vulnerabilities (specifically, adversarial attacks). It aims to explore the design space of energy-accuracy-robustness trade-ff cohesively with algorithm-hardware co-design to create truly functional intelligent systems. This poster demonstrates robustness aware algorithms broadly applicable to the energy-efficient and secure implementation of DNN engines on both current complementary metal oxide semiconductor (CMOS) accelerator platforms and emerging memory technologies (with analog crossbar dot-product engines).

https://underline.io/speakers/59503-priyadarshini-panda

Enabling Energy-Efficient and Robust Machine Intelligence with Algorithm-Hardware Co-Design

The poster presents thermal and power delivery modeling and co-design considerations for emerging 2.5D/3D ICs.

https://underline.io/speakers/59504-muhannad-bakir

Electrical-Thermal Co-Design of 2.5D/3D ICs

While the proliferation of electronics has been driven by computing and consumer applications for a long time, integrated circuits (ICs) presently undergo accelerated integration into healthcare, transportation, robotics, and autonomous systems. In addition to provision of prescribed functionalities of sensing, computing, and processing, these ICs must meet stringent reliability specifications in order to safeguard performance and safety of the whole mission-critical system where deployed. Circuits designed to be fail-safe by design exhibit low occurrences of failure. However, having a sign of no failure under typical verification and test procedures yields no guarantee for meeting a given near-zero or extremely-low failure specification. On the other hand, exhaustiveness may never be achieved by brute-force failure detection, which results in an unacceptably high cost in simulation and testing. This project will develop efficient machine-learning techniques for extremely-rare circuit-failure detection without needing large amounts of expensive simulation or test data. The proposed techniques will enable cost-effective verification and test of reliability-critical ICs and mission-critical systems in general. The research undertaken will also enable the two groups at UC Santa Barbara and UT Dallas to educate and train undergraduate and graduate students, including women and underrepresented groups, thus expanding the and contributing to the much needed US technological workforce.

It is believed that extracting critical failure information via machine learning within practical limits of available measurement or simulation data can go a long way towards extremely rare failure detection. This project centers on developing an active-learning framework that intelligently samples in the high-dimensional space of complex interacting design parameters, manufacturing variations, and operating conditions, achieving the goal of data-efficient detection of rare circuit failures. The targeted active-learning framework will be supported by the development of machine-learning model foundations and robust learning methods that can scale to high-dimensional parameter spaces. The key objective of this project is to make extremely-rare failure discovery and identification of the underlying failure mechanisms practically viable by extracting the maximum amount of useful information possible from a small amount of available data. The proposed extremely-rare failure discovery work will be broadly applicable to verification and failure analysis of analog, mixed-signal, radio-frequency, and memory circuits with stringent failure specifications and many other types of mission-critical systems.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

https://underline.io/speakers/59522-hanbin-hu

Downloads

Next from NSF Workshop on Micro/Nano Circuits and Systems Design

The Information Processing Factory – A Paradigm for Life Cycle Management of Dependable Systems

Similar lecture

Layer-wise Model Pruning based on Mutual Information

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES