FROMANNUAL REVIEWS

Efficient Few-Shot Neural Architecture Search by Counting the Number of Nonlinear Functions

Content not yet available

This lecture has no active video or poster.

AAAI 2025

•

February 27, 2025

•

Philadelphia, United States

keywords:

auto ml and hyperparameter tuning

ml

Neural architecture search (NAS) enables finding the best-performing architecture from a search space automatically. Most NAS methods exploit an over-parameterized network (i.e., a supernet) containing all possible architectures (i.e., subnets) in the search space. However, the subnets that share the same set of parameters are likely to have different characteristics, interfering with each other during training. To address this, few-shot NAS methods have been proposed that divide the space into a few subspaces and employ a separate supernet for each subspace to limit the extent of weight sharing. They achieve state-of-the-art performance, but the computational cost increases accordingly. We introduce in this paper a novel few-shot NAS method that exploits the number of nonlinear functions to split the search space. To be specific, our method divides the space such that each subspace consists of subnets with the same number of nonlinear functions. Our splitting criterion is efficient, since it does not require comparing gradients of a supernet to split the space. In addition, we have found that dividing the space allows us to reduce the channel dimensions required for each supernet, which enables training multiple supernets in an efficient manner. We also introduce a supernet-balanced sampling (SBS) technique, sampling several subnets at each training step, to train different supernets evenly within a limited number of training steps. Extensive experiments on standard NAS benchmarks demonstrate the effectiveness of our approach. We will make our code and models publicly available online.

Downloads

SlidesPaper

Next from AAAI 2025

More Text, Less Point: Towards 3D Data-Efficient Point-Language Understanding
poster

More Text, Less Point: Towards 3D Data-Efficient Point-Language Understanding

AAAI 2025

Min Chen
Yixue Hao
+5
Min Chen and 7 other authors

27 February 2025

Similar lecture

Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts
poster

Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts

ACL 2024

Ganesh Jawahar
Muhammad Abdul-Mageed
+10
Ganesh Jawahar and 12 other authors

14 August 2024

Stay up to date with the latest Underline news!

Select topic of interest (you can select more than one)

PRESENTATIONS

  • All Presentations
  • For Librarians
  • Resource Center
  • Free Trial
Underline Science, Inc.
1216 Broadway, 2nd Floor, New York, NY 10001, USA

© 2026 Underline - All rights reserved

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.