Italy

Quantized low-precision neural networks are very popular because they require less computational resources for inference and can provide high performance, which is vital for real-time and embedded recognition systems. However, their advantages are apparent for FPGA and ASIC devices, while general-purpose processor architectures are not always able to perform low-bit integer computations efficiently. 
The most frequently used low-precision neural network model for mobile central processors is an 8-bit quantized network. However, in a number of cases, it is possible to use fewer bits for weights and activations, and the only problem is the difficulty of efficient implementation.
We introduce an efficient implementation of 4-bit matrix multiplication for quantized neural networks and perform time measurements on a mobile ARM processor. It shows 2.9 times speedup compared to standard floating-point multiplication and is 1.5 times faster than 8-bit quantized one. We also demonstrate a 4-bit quantized neural network for OCR recognition on the MIDV-500 dataset. 4-bit quantization gives 95.0% accuracy and 48% overall inference speedup, while an 8-bit quantized network gives 95.4% accuracy and 39% speedup. The results show that 4-bit quantization perfectly suits mobile devices, yielding good enough accuracy and low inference time.

ICPR 2020

Fast Implementation of 4-bit Convolutional Neural Networks for Mobile Devices

quantized neural networks

convolutional neural networks

mobile devices

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster


This paper studies the problem of Person Re-Identification (ReID)
for large-scale applications. Recent research efforts have been devoted
to building complicated part models, which introduce considerably
high computational cost and memory consumption, inhibiting its practicability
in large-scale applications. This paper aims to develop a novel learning
strategy to find efficient feature embeddings while maintaining the
balance of accuracy and model complexity. More specifically, we find
by enhancing the classical triplet loss together with cross-entropy
loss, our method can explore the hard examples and build a discriminant
feature embedding yet compact enough for large-scale applications.
Our method is carried out progressively using Bayesian optimization,
and we call it the Progressive Learning Algorithm (PLA). Extensive
experiments on three large-scale datasets show that our PLA is comparable
or better than the state-of-the-arts. Especially, on the challenging
Market-1501 dataset, we achieve Rank-1=94.7%/mAP=89.4% while saving
at least 30\% parameters than strong part models.

Deep Transfer Learning for Alzheimer's disease detection

Audio-Visual Predictive Coding for Self-Supervised Visual Representation Learning

Active learning aims to optimize the dataset annotation process when resources are constrained.
Most existing methods are designed for balanced datasets.
Their practical applicability is limited by the fact that a majority of real-life datasets are actually imbalanced.
Here, we introduce a new active learning method which is designed for imbalanced datasets.
It favors samples likely to be in minority classes so as to reduce the imbalance of the labeled subset and create a better representation for these classes. 
We also compare two training schemes for active learning: (1) the one commonly deployed in deep active learning using model fine tuning for each iteration and (2) a scheme which is inspired by transfer learning and exploits generic pre-trained models and train shallow classifiers for each iteration.
Evaluation is run with three imbalanced datasets.
Results show that the proposed active learning method outperforms competitive baselines.
Equally interesting, they also indicate that the transfer learning training scheme outperforms model fine tuning if features are transferable from the generic dataset to the unlabeled one. 
This last result is surprising and should encourage the community to explore the design of deep active learning methods.

Minority Class Oriented Active Learning for Imbalanced Datasets

Deploying deep learning models with millions, even billions, of parameters is challenging given real world memory, power and compute constraints. In an effort to make these models more practical, in this paper, we propose a novel model compression approach that exploits linear dependence between the activations in a layer to eliminate entire structural units (neurons/convolutional filters). Our approach also adjusts the weights of the layer in a manner that is provably lossless while training if the removed neuron was perfectly predictable. We combine this approach with an annealing algorithm that may be applied during training, or even on a trained model, and demonstrate, using popular datasets, that our technique can reduce the parameters of VGG and AlexNet by more than 97% on CIFAR-10, 85% on Caltech-256, and 19% on ImageNet at less than 2% loss in accuracy. Furthermore, we provide theoretical results showing that in overparametrized, locally linear (ReLU) neural networks where redundant features exist, and with
correct hyperparameter selection, our method is indeed able to capture
and suppress those dependencies.


Exploiting Non-Linear Redundancy for Neural Model Compression

Recently, many multi-stream gaze estimation methods have been proposed. They estimate gaze from eye and face appearances and achieve reasonable accuracy. However, most of the methods simply concatenate the features extracted from eye and face appearance. The feature fusion process has been ignored. Thus, we propose a novel Adaptive Feature Fusion Network (AFF-Net), which performs gaze tracking task in mobile tablets. We stack two-eye feature maps and utilize Squeeze-and-Excitation layers to adaptively fuse two-eye features according to their similarity on appearance. Meanwhile, we also propose Adaptive Group Normalization to recalibrate eye features with the guidance of facial feature. Extensive experiments on both GazeCapture and MPIIFaceGaze datasets demonstrate consistently superior performance of the proposed method.


Adaptive Feature Fusion Network for Gaze Tracking in Mobile Tablets

Topological Data Analysis (TDA) is a rising field in machine learning. TDA seeks to extract information about the shape of a data set by examining its topological and geometric properties. Persistence diagrams, one of main tools in TDA, store topological information about the data. Persistence curves, a recently developed framework, provides a canonical and flexible way to encode the information presented in persistence diagrams into vectors. Based on persistence curves, the main contributions of this work are to (1) provide new sets of features for time series (2) propose a hybrid metric that takes both geometric and topological information of the time series into account, and (3) we apply these metrics to all the datasets in the UCR Time Series Classification Archive. Empirical results show that the proposed metrics perform better than the relevant benchmark in most cases.

A Hybrid Metric Based on Persistent Homology and Its Application to Signal Classification

In this work, we propose a novel method to generate compact image representations which uses local spatial information through an attention mechanism, named Generalized Local Attention Pooling (GLAP). This method, instead of being placed at the end layer of the backbone, is connected at an intermediate level, resulting in lower memory requirements. 
Experiments showcase that our proposed mechanism outperforms other pooling methods when compared with current state-of-the-art losses for deep metric learning.

Generalized Local Attention Pooling for Deep Metric Learning

This paper is to provide an overview of how object features from images influence face detection performance, and how to select synthetic faces to address specific features. To this end, we investigate the effects of occlusion, scale, viewpoint, background, and noise by using a novel synthetic image generator based on 3DU Face Dataset. To examine the effects of different features, we selected three detectors (Faster RCNN, HR, SSH) as representative of various face detection methodologies. Comparing different configurations of synthetic data on face detection systems, it showed that our synthetic dataset could complement face detectors to become more robust against features in the real world. Our analysis also demonstrated that a variety of data augmentation is necessary to address nuanced differences in performance.

Object features and face detection performance: Analyses with 3D-rendered synthetic data


A reliable detection of the R-peaks in an electrocardiogram
(ECG) is a fundamental step for further heart
rate variability (HRV) analysis, biometric recognition techniques
and additional ECG waveform based analysis. In this paper, a
novel real-time and low-complexity R-peak detection algorithm is
presented for single lead ECG signals. The detection algorithm
is divided in two stages. In the first pre-processing stage, the
QRS complex is enhanced by taking the double derivative,
squaring and moving window integration. In the second, the
detection of the R-peak is achieved based on a finite state machine
(FSM) approach. The detection threshold is dynamically adapted
and follows an exponential decay after each detection, making
it suitable for R-peak detection under fast heart rate (HR)
and R-wave amplitude changes without additional search back.
The proposed algorithm was evaluated in a private single lead
ECG database acquired using a FieldWiz wearable device. The
database comprises five recordings from four different subjects,
recorded during dynamic conditions, namely: running, trail
running and weightlifting. The raw ECG signals were annotated
for the R-peak and the proposed method benchmarked against
common QRS detectors. The combined acquisition setup and
presented approach resulted in R-peak detection Sensitivity (Se)
of 99.77% and Positive Predictive Value of (PPV) of 99.18%,
comparable to state of the art real time QRS detectors. Due to its
low computational complexity, this method can be implemented
in embedded wearable systems, suited for cardiovascular tracking
devices in dynamic use cases and R-peak detection.

A Low-Complexity R-peak Detection Algorithm with Adaptive Thresholding for Wearable Devices

The characteristics of training data is a fundamental consideration when constructing any supervised classifier. Class mislabelling and imbalance are major training data issues that often adversely affect machine learning algorithms, including ensembles. This work proposes extended bagging algorithms to better handle noisy and multi-class imbalanced classification tasks. These algorithms upgrade the sampling procedure by taking benefit of the confidence in ensemble classification outcome. The underlying idea is that a bagging ensemble learning algorithm can achieve greater performance if it is allowed to choose the data from which it learns. The effectiveness of the proposed methods is demonstrated in performing classification on 10 various data sets.

Premium content

Fast Implementation of 4-bit Convolutional Neural Networks for Mobile Devices

Downloads

Next from ICPR 2020

Deep Transfer Learning for Alzheimer's disease detection

Similar lecture

Operation and Topology Aware Differentiable Architecture Search

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES