Knowledge distillation, which aims to train model under the supervision from another large model (teacher model) to the original model (student model), has achieved remarkable results in supervised learning. However, there are two major
problems with existing knowledge distillation methods. One is the teacher’s supervision is sometimes misleading, and the other is the student’s prediction is not accurate enough. To address the first issue, instead of learning a combination of
both teachers and ground truth, we apply knowledge adjustment to correct teachers’ supervision using ground truth. For the second problem, we use the selective classification framework to train the student model. In particular, the deep gambler loss is adopted to predict with reservation by explicitly introducing
the (m + 1)-th class. We consider two settings of knowledge distillation: (1) distillation across different network structures (AlexNet, ResNet), and (2) distillation across networks with different depths (ResNet18, ResNet50) to evaluate the effectiveness of our method. The experimental results on benchmark datasets
(i.e., Fashion-MNIST, SVHN, CIFAR10, CIFAR100) are reported with higher prediction accuracies and lower coverage errors.

Knowledge Distillation with a Precise Teacher and Prediction with Abstention

**Session Chair**: prof. Aleksandr Ermolov

PS T1.10: Neural Architectures for Pattern Analysis 

poster

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


Yi Xu

1

SHORT BIO

Presentations

Knowledge Distillation with a Precise Teacher and Prediction with Abstention

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES