Italy

A suitable combination of data in a multimodal emotion recognition model allows conveying and combining each channel&#39;s information to achieve a better recognition of the encoded emotion than would be possible using only a single modality and channel. In this paper, we focus on combining speech and physiological signals to predict the arousal and valence levels of the emotional states of a person. We designed a neural network that can use the information from raw audio signals, electrocardiograms, heart rate variability, electro-dermal activity, and skin conductance levels, to predict emotional states. The proposed deep neural network architecture works as an end-to-end process, which means, neither any pre-processing of the input data nor post-processing of the prediction of the network was applied. Using the data of the modalities available in the publicly accessible part of the RECOLA database, we achieved results comparable to other state-of-the-art approaches.

ICPR 2020

MMultimodal Emotion Recognition Based on Speech and Physiological Signals Using Deep Neural Networks

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


A suitable combination of data in a multimodal emotion recognition model allows conveying and combining each channel's information to achieve a better recognition of the encoded emotion than would be possible using only a single modality and channel. In this paper, we focus on combining speech and physiological signals to predict the arousal and valence levels of the emotional states of a person. We designed a neural network that can use the information from raw audio signals, electrocardiograms, heart rate variability, electro-dermal activity, and skin conductance levels, to predict emotional states. The proposed deep neural network architecture works as an end-to-end process, which means, neither any pre-processing of the input data nor post-processing of the prediction of the network was applied. Using the data of the modalities available in the publicly accessible part of the RECOLA database, we achieved results comparable to other state-of-the-art approaches.

technical paper

Semi-supervised learning exploits unlabeled data to improve generalization ability with insufficient annotations. In recent years, Mean Teacher method(MT) obtained impressive performance using prediction consistency as regularization. However, severe ambiguity in medical images makes the targets in the teacher model highly unreliable in obscure regions, thereby limits the model capability. To address this problem, we propose a novel multi-task learning semi-supervised framework to gain hierarchical consistency through training process. Specifically, we introduce region and shape predictions as subtasks to obtain the coarse-grained location and fine-grained boundary information. Then we predict pixel-level segmentation by fusing the hierarchical feature. Since calculating consistency loss in more loose regions typically alleviates the degradation caused by learning from unreliable targets, our teacher model generate guidance from each of the subtasks. Moreover, we focus on the geometrical correlations in different tasks and proposed the constraint method to refine the segmentation for accurate guidance. Experiments on the left atrium segmentation dataset show our algorithm achieves state-of-the-art performance comparing with other semi-supervised methods.

Hierarchical Consistency and Refinement for Semi-supervised Medical Segmentation

Satellite imagery is becoming increasingly available due to a large number of commercial satellite companies. Many fields use satellite images, including meteorology, forestry, natural disaster analysis, and agriculture. These images can be changed or tampered with image manipulation tools causing issues in applications using these images. Manipulation detection techniques designed for images captured by “consumer cameras” tend to fail when used on satellite images. In this paper we propose a supervised method, known as Nested Attention U-Net, to detect spliced areas in the satellite images. We introduce three datasets of manipulated satellite images that contain objects generated by a generative adversarial network (GAN). We test our approach and compare it to existing supervised splicing detection and segmentation techniques and show that our proposed approach performs well in detection and localization.

Nested Attention U-Net (NAU-N): A Splicing Detection Method for Satellite Images

We investigate the negative effects of rain streaks over the performance of a neural network for real time semantic segmentation of street scenes. This is done by synthetically augmenting the CityScapes dataset with artificial rain. We then define and train a generative adversarial network for rain removal, and quantify the benefits of its application as a pre-processing step to both rainy and "clean" images. Finally, we show that by retraining the semantic segmentation network on images processed for rain removal, it is possible to gain even more accuracy, with a model that produces stable results in all analyzed atmospheric conditions. For our experiments, we present a per-class analysis in order to provide deeper insights over the impact of rain on semantic segmentation.

On the impact of rain over semantic segmentation of street scenes

We propose a novel method for training Convolution Neural Network, named CNN-FQ, which takes a face image and outputs a scalar summary of the image quality. The CNN-FQ is trained from triplets of faces that are automatically labeled based on responses of a pre-trained face matcher. The quality scores extracted by the CNN-FQ are directly linked to the probability that the face matcher incorrectly ranks a randomly selected triplet of faces. We applied the proposed CNN-FQ, trained on CASIA database, for selection of the best quality image from a collection of face images capturing the same identity. The quality of the single face representation was evaluated on 1:1 Verification and 1:N Identification tasks defined by the challenging IJB-B protocol. We show that the recognition performance obtained when using faces selected based on the CNN-FQ scores is significantly higher than what can be achieved by competing state-of-the-art image quality extractors.

CNN Based Predictor of Face Image Quality

Low-light is a challenging environment for both human and computer vision to perform tasks such as object classification and detection. Recent works have shown potential in employing enhancements algorithms to support and improve such tasks in low-light, however there has not been any focused analysis to understand the direct effects that low-light enhancement have on an object detector. This work aims to quantify and visualize such effects on the multi-level abstractions involved in network-based object detection. First, low-light image enhancement algorithms are employed to enhance real low-light images, and then followed by deploying an object detection network on the low-light as well as the enhanced counterparts. A comparison of the activations in different layers, representing the detection features, are used to generate statistics in order to quantify the enhancements' contribution to detection. Finally, this framework was used to analyze several low-light image enhancement algorithms and identify their impact on the detection
model and task. This framework can also be easily generalized to any convolutional neural network-based models for the analysis of different enhancements algorithms and tasks.

Exploring the Contributions of Low-light Image Enhancement to Network-based Object Detection

This paper presents a deep learning-based method
to estimate gait attributes (e.g., stately, cool, relax, etc.). Similarly
to the existing studies on relative attribute, human perception-
based annotations on the gait attributes are given to pairs of
gait videos (i.e., the ﬁrst one is better, tie, and the second one
is better), and the relative annotations are utilized to train a
ranking model of the gait attribute. More speciﬁcally, we design
a Siamese (i.e., two-stream) network which takes a pair of
gait inputs and output gait attribute score for each. We then
introduce a suitable loss function called a signed contrastive loss
to train the network parameters with the relative annotation.
Unlike the existing loss functions for learning to rank does
not inherit a nice property of a quadratic contrastive loss, the
proposed signed quadratic contrastive loss function inherits the
nice property. The quantitative evaluation results reveal that
the proposed method shows better or comparable accuracies of
relative attribute prediction against the baseline methods.

Deep Gait Relative Attribute using a Signed Quadratic Contrastive Loss

Generating as-is 3D Models is constantly explored for various construction management applications. The industry has been dependent on either manual or semi-automated workflows for the Scan-to-BIM process, which is laborious as well as time taking. Recently machine learning has opened avenues to recognize geometrical elements from point clouds but has not been much used because of the insufficient labeled dataset. This study aims to set up a semi-automated workflow to create labeled data sets which can be used to train ML algorithms for element identification purpose. The study proposes an interactive user interface using a gaming engine within a mixed reality environment. A workflow for fusing as-is spatial information with the AR/VR based information is presented in Unity 3D. A user friendly UI is then developed and integrated with the VR environment to help the user to choose the category of the component by visualization. This results in the generation of an accurate as-is 3D Model, which does not require much computation or time. The intention is to propose a smooth workflow to generate datasets for learning-based methodologies in a streamlined Scan-to-BIM Process. However, this process requires user domain knowledge and input. The dataset can be continuously increased and improved to get automated results later.


Mixed Reality-Based Dataset Generation for Learning Based Scan-to-BIM

Remote working and collaboration is important towards helping workplaces to become flexible and productive. The significance of remote working has also been highlighted in the COVID-19 pandemic period in which mobility restrictions were enforced. This paper presents the development of an augmented reality platform, aiming to assist workers in remote collaboration and training. The platform consists of two communicating apps intended to be used by a remote supervisor (located e.g., at home) and an on-site worker, and uses intuitive digital annotations that enrich the physical environment of the workplace, thereby facilitating the execution of on-site tasks. The proposed platform was used and evaluated in user trials, demonstrating its usefulness and virtue by assessing its performance, worker satisfaction and task completion time.

An Augmented Reality-based Remote Collaboration Platform for Worker Assistance

The future civilian, and professional autonomous vehicles to be realised into the market should apprehend and interpret the road in a manner similar to the human drivers. In structured urban environments where signs, road lanes and markers are well defined and ordered, landmark-based road tracking and localisation has significantly progressed during the last decade with many autonomous vehicles to make their debut into the market. However, in semi-structured and rural environments where traffic infrastructure is deficient, the autonomous driving is hindered by significant challenges. The paper at hand presents a Lidar-based method for road boundaries detection suitable for a service robot operation in rural and semi-structured environments. Organised Lidar data undergo a spatial distribution processing method to isolate road limits in a forward looking horizon ahead of the robot. Stereo SLAM is performed to register subsequent road limits and RANSAC is applied to identify edges that correspond to road segments. In addition, the robot traversable path is estimated and progressively merged with Bézier curves to create a smooth trajectory that respects vehicle kinematics. Experiments have been conducted on data collected from our robot on a semi-structured urban environment, while the method has also been evaluated on KITTI dataset exhibiting remarkable performance.

PRRoad tracking in semi-structured environments using spatial distribution of Lidar data

Breathing rate together with heart rate, blood pressure,
and body temperature, are considered as essential and vital
signs of human body. In this demo, we present a novel system to
estimate the respiratory rate using a low-cost, non-intrusive, and
passive infrared thermopile sensor array. The thermopile sensor
array can detect the thermal infrared energy and generate a
low-resolution video from subjects within its viewing range. We
collected low resolution 8 8-pixel thermal video clips from 40
subjects at resting position from a distance of one-meter between
the thermopile sensor array and the subjects. We processed
the image frames using Haar-like filters and generated a one-dimensional
feature signal from each video. The signal exhibits
an almost periodic behavior following the respiratory movements.
A peak detection algorithm is used to detect each breath. The
proposed method is computationally efficient because it is based
on multiplierless Haar-type feature extraction and peak detection.
We achieved a high positive correlation rate of 97.77% between
the actual respiratory rate that was measured by monitoring
the chest movements during inhalation and exhalation and the
estimated rate in our dataset. We will demonstrate this IR array
based respiration rate detection system during the demo session.

Premium content

MMultimodal Emotion Recognition Based on Speech and Physiological Signals Using Deep Neural Networks

Downloads

Next from ICPR 2020

Hierarchical Consistency and Refinement for Semi-supervised Medical Segmentation

Similar lecture

Visual Style Extraction from Chart Images for Chart Restyling

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES