United States

In this paper we present early results of a novel approach for human activity recognition, focusing on activities of daily living. Our approach is multimodal, i.e., it combines RGB, depth and skeletal data measurements. Moreover, it is based on a novel hybrid deep neural network architecture that combines a convolutional neural network, a long short term memory network and it is evaluated using a publicly available dataset.

PETRA 2021

A Deep Hybrid Architecture for Human Activity Recognitionhttps://dl.acm.org/doi/10.1145/3453892.3461006

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible. The experience for on-site participants would closely approximate a normal pre-COVID *ACL conference, with 5-6 thematically organized parallel sessions and live Q/A and interactive discussion immediately after the talks. Presentations by virtual participants will be equitably interleaved with those of on-site participants, projected on the auditorium screens as if on-site, and also followed immediately by live Q/A and interactive discussion at a time during reasonable waking hours for the virtual presenter. For all participants, on-site and virtual, who are unable to attend a session due to either time-zone issues or because they are participating in another session live, talk recordings and slides will be available online at a minimum after the live presentation (and in many cases before as well), and questions may be submitted in advance on session-specific discussion boards and answered live in session with the usual visual aids if desired.

<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/ceby/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>

Please Note: The EMNLP registration system is not currently connected to the underline site as we are still in the process of building out EMNLP 2021. You will receive access instructions from underline the week of November 1st. 

Access is given only to EMNLP upon registration, if you have not registered please do so [here](https://2021.emnlp.org/registration).

Registered attendees will receive access the week of November 1st.

EMNLP 2021

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible.

poster

**About**
<br>The PErvasive Technologies Related to Assistive Environments (PETRA) conference is a highly interdisciplinary conference that focuses on computational and engineering approaches to improve the quality of life and enhance human performance in a wide range of settings, in the workplace, at home, in public spaces, urban environments, and other. Outcomes of this conference have a broad impact in application areas that include, manufacturing, transportation, healthcare, energy systems, security and safety, robotics, biomedicine, environment and conservation, and many others. PETRA brings together very different types of technologies to also address important social and healthcare issues for sensitive populations, such as the elderly, persons suffering from chronic conditions such as Alzheimer’s, Parkinson’s, Cerebral Palsy, and other disabilities or traumas.

<br>PETRA (which means “stone” in Greek) addresses important human living needs. From a technological perspective, PETRA motivates and inspires new scientific thrusts, innovations and discoveries to improve living environments, a particularly important issue, in an increasingly crowded and complex world where the need for inclusiveness and connectivity with the rest of the world is key. For the elderly in particular, PETRA addresses the fact that, as people grow older; they will increasingly rely on technology to be able to prolong staying in their homes and enjoy an independent life.

<br>The PETRA 2021 proceedings will be published in the ACM Digital Library as part of the ACM ICPS program. The ISBN number assigned to PETRA '21 is 978-1-4503-8792-7
<br>
<br>

The PErvasive Technologies Related to Assistive Environments (PETRA) conference is a highly interdisciplinary conference that focuses on computational and engineering approaches to improve the quality of life and enhance human performance in a wide range of settings, in the workplace, at home, in public spaces, urban environments, and other. Outcomes of this conference have a broad impact in application areas that include, manufacturing, transportation, healthcare, energy systems, security and safety, robotics, biomedicine, environment and conservation, and many others. PETRA brings together very different types of technologies to also address important social and healthcare issues for sensitive populations, such as the elderly, persons suffering from chronic conditions such as Alzheimer’s, Parkinson’s, Cerebral Palsy, and other disabilities or traumas.

n this paper, we present a novel method to learn end-to-end vi-suomotor policies for robotic manipulators. The method computesstate-action mappings in a supervised learning manner from videodemonstrations and robot trajectories. We show that the robotlearns to perform different tasks by associating image features withthe corresponding movement primitives of different grasp poses.To evaluate the effectiveness of the proposed learning method, weconduct experiments with a PR2 robot in a simulation environment.The purpose of these experiments is to evaluate the system’s abilityto perform manipulation tasks

Learning Visuomotor Policies with Deep Movement Primitives

We present a perspective of the role of participatory design in Human-Robot Collaboration (HRC) for People with Motor Disabilities (PWMD). We have been involved in several multiyear projects related to HRC for PWMD, where we encountered different challenges related to planning and participation, preferences of stakeholders, using certain participatory design techniques, technology exposure, as well as, ethical legal and social implications. These challenges helped us provide five lessons learned that could guide researchers when using participatory design with vulnerable groups. In particular, early-career researchers who start exploring research in HRC for people with disabilities.

Reflecting upon Participatory Design in Human-Robot Collaboration for People with Motor Disabilities: Challenges and Lessons Learned from Three Multiyear Projects

This paper presents a design concept for affordable exoskeletons without
renouncing to advanced interaction control modalities based on force
control technologies. Affordability is reached by (1) lowering the
motor requirements, thanks to a smart mechanical design, (2) lowering
the sensing requirements, thanks to extensive use of microprocessor
DMA, and (3) using low-cost computational platforms. A main finding
of this work is that by lowering the motor requirements the exoskeleton
improves its inherent force controllability.

Toward Personal Affordable Exoskeletons With Force Control Capabilities

This work presents the development of a child-robot interaction(CRI) system for edutainment scenarios, aiming to provide a framework for their design and to simplify access to social robots by educators with non-specialized technical knowledge in this challenging area. Our framework incorporates powerful robotic perception modules for action and emotion recognition of the interacting child, allowing the robot to exhibit empathy and be informed of the child’s activity. Both developed modules are evaluated on respective datasets, outperforming the current state-of-the-art by a significant margin, while retaining low computational cost. The modules are complemented by off-the-shelf automatic speech recognition and synthesis components to further enable and enrich edutainmment-focused CRI. Moreover, the developed framework allows custom CRI scenario-building via a suitable graphical user interface, providing a valuable asset to educators wishing to utilize social robots in the classroom.

A robotic edutainment framework for designing child-robot interaction scenarios

As mobile technologies have become ubiquitous in recent years, computer-based cognitive tests have become more popular and efficient. In this work, we focus on assessing motor function in children by analyzing their gait movements. Although there has been a lot of research on designing automated assessment systems for gait analysis, most of these efforts use obtrusive wearable sensors for measuring body movements. We have devised a computer vision-based assessment system that only requires a camera which makes it easier to employ in school or home environments. A dataset has been created with 27 children performing the test. Furthermore in order to improve the accuracy of the system, a deep learning based model was pre-trained on NTU-RGB+D 120 dataset and then it was fine-tuned on our gait dataset. The results highlight the efficacy of proposed work for automating the assessment of children's performances by achieving 76.61% classification accuracy.

Automated system to measure Tandem Gait to assess executive functions in children

This article proposes a novel approach for augmenting generative adversarial network (GAN) with a self-supervised task in order to improve its ability for encoding video representations that are useful in downstream tasks such as human activity recognition. In the proposed method, input video frames are randomly transformed by different spatial transformations, such as rotation, translation and shearing or temporal transformations such as shuffling temporal order of frames. Then discriminator is encouraged to predict the applied transformation by introducing an auxiliary loss. Subsequently, results prove superiority of the proposed method over baseline methods for providing a useful representation of videos used in human activity recognition performed on datasets such as KTH, UCF101 and Ball-Drop. Ball-Drop dataset is a specifically designed dataset for measuring executive functions in children through physically and cognitively demanding tasks. Using features from proposed method instead of baseline methods caused the top-1 classification accuracy to increase by more then 4%. Moreover, ablation study was performed to investigate the contribution of different transformations on downstream task.

Self-Supervised Human Activity Recognition by Augmenting Generative Adversarial Networks

A Novel Interactive 9-Peg Board for Cognitive and Physical Assessment

Hand pose estimation is getting a lot of attention in many areas such
as Human-Computer Interaction and Sign Language Recognition.
A fundamental step to accurately estimate the hand pose involves
detecting and localizing fingertips in an image. Despite the progress
of 2-D hand pose estimation in recent studies, accurate and robust
detection and localization of fingertips still remains a challenging
task due to low resolution of a fingertip in images and varying
lightning condition.
Inspired by the progress of the Generative Adversarial Network
(GAN) and image-style transfer, we propose a two-stage pipeline to
accurately localize the fingertip position even in varying lighting
and severe self occlusion on depth images. The idea is to use a
Cycle-consistent Generative Adversarial Network (Cycle-GAN) to
apply unpaired image-to-image translation and generate a depth
image with colored predictions on the fingertips, wrist, and palm
given a real depth image. The model is trained in a semi-supervised
manner using a collection of images from source and target domains
that do not need to be related in anyway. Then, by applying color
segmentation techniques, we localize the center of each colored area
which results in finding the location of each fingertip along with
center of the wrist and the palm. The proposed method achieves
visually promising results on noisy depth images captured using the
Microsoft Kinect. Experiments on the challenging NYU hand dataset
have demonstrated that our approach not only generates plausible
samples, but also outperforms state-of-the-art approaches on 2-D
fingertip estimation by a significant margin even in the presence
of severe self-occlusion and varying lighting conditions. Moreover,
fingertips would be detected irrespective of user orientation using
this method.

A pipeline for Hand 2-D Keypoint Localization using Unpaired Image to Image Translation

The ageing population leads to an increase of people requiring long-term care. Assisting people when getting out of bed and fast reactions to falls can help to reduce costs and the risk of injury. We describe the possibility of detecting getting up behavior from a bed using different deep learning models and depth data as a proof of concept. The hereby used computer vision approach uses unobtrusive depth data in order to protect people’s privacy. We gather data from different subjects, postures, views and rooms, with which we then train the network. Both classification and object detection methods are able to reliably detect getting up behavior. Situations that could not be correctly classified were when a person was changing from lying to sitting or when the legs of the person were covered with a blanket. Our results show that using pretrained networks is the key contributor in training. We also demonstrate that convolutional neural networks are capable of extracting highlevel task-dependent features from depth data which can be utilized
in developing ambient intelligent systems. The practicality of the network can be adapted from getting up from a bed to sitting and walking inside the room, based on the purpose of the real-life application.

Privacy preserving getup detection

Teletherapeutic devices enable new possibilities to track the therapy process by gathering and visualizing training statistics. For the therapist this can mean a quicker but also a deeper look at the therapy results. For the patient, the right presentation of data can foster the motivation and adherence to the therapy. In an iterative user centered design process, requirements of data visualization were gathered and the developed solution was validated with health professionals. For the therapists, the most important part is to have a comprehensive overview of conducted exercises and training duration. For the patient, the visualization should be simple and motivative. Further validation, especially with patients is needed.

A Deep Hybrid Architecture for Human Activity Recognitionhttps://dl.acm.org/doi/10.1145/3453892.3461006

Downloads

Next from PETRA 2021

Learning Visuomotor Policies with Deep Movement Primitives

Similar lecture

Mind the Context: The Impact of Contextualization in Neural Module Networks for Grounding Visual Referring Expressions

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

A Deep Hybrid Architecture for Human Activity Recognitionhttps://dl.acm.org/doi/10.1145/3453892.3461006

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from PETRA 2021

Learning Visuomotor Policies with Deep Movement Primitives

Similar lecture

Mind the Context: The Impact of Contextualization in Neural Module Networks for Grounding Visual Referring Expressions

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads