Content not yet available

This lecture has no active video or poster.

technical paper

AAAI 2024

February 23, 2024

Vancouver , Canada

Operator-Learning-Inspired Modeling of Neural Ordinary Differential Equations

keywords:

deep neural architectures

deep neural network algorithms

deep learning theory

Neural ordinary differential equations (NODEs), one of the most influential works of the differential equation-based deep learning, are to continuously generalize residual networks and opened a new field. They are currently utilized for various downstream tasks, e.g., image classification, time series classification, image generation, etc. Its key part is how to model the time-derivative of the hidden state, denoted dh(t)/dt. People have habitually used conventional neural network architectures, e.g., fully-connected layers followed by non-linear activations. In this paper, however, we present a neural operator-based method to define the time-derivative term. Neural operators were initially proposed to model the differential operator of partial differential equations (PDEs). Since the time-derivative of NODEs can be understood as a special type of the differential operator, our proposed method, called branched Fourier neural operator (BFNO), makes sense. In our experiments with general downstream tasks, our method significantly outperforms existing methods.

Downloads

SlidesPaper

Next from AAAI 2024

TaskLAMA: Probing the Complex Task Understanding of Language Models | VIDEO
technical paper

TaskLAMA: Probing the Complex Task Understanding of Language Models | VIDEO

AAAI 2024

+3Quan Yuan
Quan Yuan and 5 other authors

23 February 2024

Stay up to date with the latest Underline news!

Select topic of interest (you can select more than one)

PRESENTATIONS

  • All Lectures
  • For Librarians
  • Resource Center
  • Free Trial
Underline Science, Inc.
1216 Broadway, 2nd Floor, New York, NY 10001, USA

© 2023 Underline - All rights reserved