
This paper strives for action recognition and detection in video modalities like RGB, depth maps or 3D-skeleton sequences when only limited modality-specific labeled examples
are available. For the RGB, and derived optical-flow, modality
many large-scale labeled datasets have been made available. They
have become the de facto pre-training choice when recognizing
or detecting new actions from RGB datasets that have limited
amounts of labeled examples available. Unfortunately, large-scale
labeled action datasets for other modalities are unavailable for
pre-training. In this paper, our goal is to recognize actions from
limited examples in non-RGB video modalities, by learning from
large-scale labeled RGB data. To this end, we propose a two-step
training process: (i) we extract action representation knowledge
from an RGB-trained teacher network and adapt it to a nonRGB student network. (ii) we then fine-tune the transfer model
with available labeled examples of the target modality. For the
knowledge transfer we introduce feature-supervision strategies,
which rely on unlabeled pairs of two modalities (the RGB and
the target modality) to transfer feature level representations from
the teacher to the student network. Ablations and generalizations
with two RGB source datasets and two non-RGB target datasets
demonstrate that an optical-flow teacher provides better action
transfer features than RGB for both depth maps and 3Dskeletons, even when evaluated on a different target domain, or
for a different task. Compared to alternative cross-modal action
transfer methods we show a good improvement in performance
especially when labeled non-RGB examples to learn from are
scarce.

Feature-Supervised Action Modality Transfer

PS T3.3: Computer Vision 2D, 3D and Tracking

poster

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


Fida Mohammad Thoker

1

SHORT BIO

Presentations

Feature-Supervised Action Modality Transfer

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES