United Kingdom

This paper is concerned with designing purpose in autonomous robots for open-ended learning settings. Unconstrained human robot interaction situations and robotic systems that must operate in dynamic multi-robot scenarios are paradigmatic examples of open-endedness. An approach to the appropriate design and engineering of motivational structures to endow robots with a particular purpose is proposed and tested. This approach focuses on the drive structure and how it can be made to autonomously adapt to changing circumstances. Specifically, a simple evolutionary strategy for the autonomous regulation of multiple drives in order to optimize long-term operation is defined. The experimental results have been obtained on a Baxter robot facing changing situations in real setups.

AAMAS 2021

An Autonomous Drive Balancing Strategy for the Design of Purpose in Open-ended Learning Robots

**31st IEEE International Conference on Robot & Human Interactive Communication**
 This conference is a leading forum where state-of-the-art innovative results, the latest developments as well as future perspectives relating to robot and human interactive communication are presented and discussed.
 
 The conference covers a wide range of topics related to Robot and Human Interactive Communication, involving theories, methodologies, technologies, empirical and experimental studies. Papers related to the study of robotic technology, psychology, cognitive science, artificial intelligence, human factors, ethics and policies, interaction-based robot design and other topics related to human-robot interaction are welcome.
 
 **CONFERENCE THEME:**
 Social, asocial, and antisocial robots
 
 Robots are starting to populate our lives: at work, in our homes, hotels, shops. However, not everybody is comfortable with robots finding a place in society. For example, people may be afraid that robots will replace humans in their jobs. This belief is exacerbated by the media that, on one side, raise expectations of robots’ capabilities, while amplifying fears and worries on the other side.
 
 In the next years, due to the increasing request for interaction with people in social environments, academic research and market analyses predict that robots will become more and more “social”. But, at the same time, robots may incur the risk of being perceived as “asocial” as their pervasiveness in everyday environments may become an obstacle to social interaction between humans. Or they may even be rejected as “antisocial” if researchers do not take care to develop robots that behave according to the customs and habits of the communities where they are expected to operate.
 
 To prevent this from happening, robotic researchers can no longer take the luxury of being mere technicians. They must be capable of addressing all the challenges raised by future robot-populated communities in a multidisciplinary way. They need to share their knowledge with the public to make people aware of what robots really are, what expectations we can have from them, the risks we may incur, and the possible mitigation plans.
 
 While being open to any kind of contribution related to Human-Robot Interactive Communication, the RO-MAN 2022 conference will emphasize these aspects of robotic research. How can we create robots capable of being welcomed as our assistants and companions in society while, at the same time, not being perceived as “asocial” or “antisocial?” How can we create robots perceived as facilitators of the natural development of human beings and communities, rather than obstacles?
 
 Contributions from different disciplines, not exclusively related to technological development, will be welcomed as crucial to pave the way to global acceptance of future robots with an informed attitude.
 
 [![](https://assets.underline.io/uploads/markdown_image/1/image/af259abe105b3791c498ae1552d75710.png)](https://drive.google.com/file/d/18w73BZPB_JVOgcrcdZDqYpEupwVt4CnU/view?usp=sharing)

A registration is required to access this event page. If you have not already done so, please use the below link to register for the event. 

[Register Here](http://www.smile.unina.it/ro-man2022/welcome-to-ieee-ro-man-2022/)

Registration Is Required

IEEE RO-MAN 2022

31st IEEE International Conference on Robot & Human Interactive Communication
This conference is a leading forum where state-of-the-art innovative results, the latest developments as well as future perspectives relating to robot and human interactive communication are presented and discussed.

poster

AAMAS is the largest and most influential conference in the area of agents and multiagent systems, bringing together researchers and practitioners in all areas of agent technology and providing and internationally renowned high-profile forum for publishing and finding out about the latest developments in the field. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).  

![](https://assets.underline.io/uploads/markdown_image/1/image/ce4604bd54468a006164d16e080204fb.jpg)

Please click on the button below to register for this event.

If you are registered and see this page please check the account email you are logged in with. It must be the same as for the registration.

If you still have trouble accessing the contact please contact support at aamas2021@underline.io 

Automated care systems are becoming more tangible than ever: recent breakthroughs in robotics and machine learning can be used to address the need for automated care created by the increasing aging population. However, such systems require overcoming several technological, ethical, and social challenges. One inspirational manifestation of these challenges can be observed in the training of seeing-eye dogs for visually impaired people. A seeing-eye dog is not just trained to obey its owner, but also to "intelligently disobey": if it is given an unsafe command from its handler, it is taught to disobey it or even insist on a different course of action. This paper proposes the challenge of building a seeing-eye robot, as a thought-provoking use-case that helps identify the challenges to be faced when creating behaviors for robot assistants in general. Through this challenge, this paper delineates the prerequisites that an automated care system will need to have in order to perform intelligent disobedience and to serve as a true agent for its handler.

The Seeing-Eye Robot Grand Challenge: Rethinking Automated Care (Blue Sky Ideas Track)

Policy gradient methods have become one of the most popular classes of algorithms for multi-agent reinforcement learning. A key challenge, however, that is not addressed by many of these methods is multi-agent credit assignment: assessing an agent's contribution to the overall performance, which is crucial for learning good policies. We propose a novel algorithm called Dr.Reinforce that explicitly tackles this by combining difference rewards with policy gradients to allow for learning decentralized policies when the reward function is known. By differencing the reward function directly, Dr.Reinforce avoids difficulties associated with learning the 
-function as done by Counterfactual Multiagent Policy Gradients (COMA), a state-of-the-art difference rewards method. For applications where the reward function is unknown, we show the effectiveness of a version of Dr.Reinforce that learns a reward network that is used to estimate the difference rewards.

Difference Rewards Policy Gradients

Recently, there is growing attention on applying deep reinforcement learning (DRL) to solve the 3D bin packing problem (3D BPP), given its favorable generalization and independence of ground-truth label. However, due to the relatively less informative yet computationally heavy encoder, and considerably large action space inherent to the 3D BPP, existing methods are only able to handle up to 50 boxes. In this paper, we propose to alleviate this issue via an end-to-end multimodal DRL agent, which sequentially addresses three sub-tasks of sequence, orientation and position, respectively. The resulting architecture enables the agent to solve large-scale instances of 100 boxes or more. Experiments show that the agent could learn highly efficient policies that deliver superior performance against all the baselines on instances of various scales.

Solving 3D Bin Packing Problem via Multimodal Deep Reinforcement Learning

We propose a generic reward shaping approach for improving the rate of convergence in reinforcement learning (RL), called Self Improvement Based REwards, or SIBRE. The approach is designed for use in conjunction with any existing RL algorithm, and consists of rewarding improvement over the agent's own past performance. We prove that SIBRE converges in expectation under the same conditions as the original RL algorithm. The reshaped rewards help discriminate between policies when the original rewards are weakly discriminated or sparse. Experiments on several well-known benchmark environments with different RL algorithms show that SIBRE converges to the optimal policy faster and more stably. We also perform sensitivity analysis with respect to hyper-parameters, in comparison with baseline RL algorithms.

SIBRE: Self Improvement Based REwards for Adaptive Feedback in Reinforcement Learning

Distributional reinforcement learning (RL) provides beneficial impacts for the single-agent domain. However, distributional RL methods are not directly compatible with value function factorization methods for multi-agent reinforcement learning. This work provides a distributional perspective on value function factorization, offering a solution for bridging the gap between distributional RL and value function factorization methods.

A Distributional Perspective on Value Function Factorization Methods for Multi-Agent Reinforcement Learning

Transportation systems of the future can be best modeled as multi-agent systems. A number of coordination protocols such as autonomous intersection management (AIM), adaptive cooperative traffic light control (TLC), cooperative adaptive cruise control (CACC), among others have been developed with the goal of improving the safety and efficiency of such systems. The overall goal in these systems is to provide behavioral guarantees under the assumption that the participating agents work in concert with a centralized (or distributed) coordinator. While there is work on analyzing such systems from a security perspective, we argue that there is limited work on quantifying trustworthiness of individual agents in a multi-agent system. We propose a framework that uses an epistemic logic to quantify trustworthiness of agents, and embed the use of quantitative trustworthiness values into control and coordination policies. Our modified control policies can help the multi-agent system improve its safety in the presence of untrustworthy agents (and under certain assumptions, including malicious agents). We empirically show the effectiveness of our proposed trust framework by embedding it into AIM, TLC, and CACC platooning algorithms. In our experiments, our trust framework accurately detects attackers in CACC platoons; mitigates the effect of untrustworthy agents in AIM; and trust-aware TLC and AIM reduce collisions in all cases compared to the vanilla versions of these algorithms.

A General Trust Framework for Multi-Agent Systems

In this paper we investigate the effect of an underlying social network over agents in a well-known multi-agent resource allocation problem; the housing market. We first show that, when a housing market takes place over a social network with more than two agents and these agents have an option to avoid forwarding information about it to their followers, there does not exist an exchange mechanism that simultaneously satisfies strategy-proofness, Pareto efficiency, and individual rationality. It is also impossible to find a strategy-proof exchange mechanism that always chooses an outcome in a weakened core. These results highlight the difficulty of taking into account the agents' incentive of information diffusion in the resource allocation. To overcome these negative results, we consider two different ways of restricting the problem; limiting the domain of preferences and the structure of social networks.

Mechanism Design for Housing Markets over Social Networks

Reasoning about observed effects and their causes is important in many applications. For instance, understanding why a plan failed can aid the task of replanning by allowing the agent to tailor a better plan. But under incomplete information, an agent may be unable to determine which actions/events caused an effect. To overcome this, the agent may be able to perform some sensing actions that allow him to figure out what caused the effect. This becomes even more important in multiagent contexts, where an agent may want to identify which agents caused some effect, or possibly prevent other agents from determining who caused something. The effects involved may even be epistemic effects, such as an agent coming to know the PIN of a bank card, and the causes may be sensing actions. Reasoning about such causes is a key part of "theory of mind" and understanding other agents' behaviour. While there has been much work on causality from an objective standpoint, causality from the point of view of individual agents has received much less attention. In this paper, we develop a formalization of knowledge about actual causes in the situation calculus, and how it is affected by actions including sensing. We show that the proposed framework has some intuitive properties and study the conditions under which an agent can be expected to come to know the causes of an effect.

Knowing Why - On the Dynamics of Knowledge about Actual Causes in the Situation Calculus

Several resource allocation problems involve multiple types of resources, with a different agency being responsible for "locally" allocating the resources of each type, while a central planner wishes to provide a guarantee on the properties of the final allocation given agents' preferences. We study the relationship between properties of the local mechanisms, each responsible for assigning all of the resources of a designated type, and the properties of a sequential mechanism which is composed of these local mechanisms, one for each type, applied sequentially, under lexicographic preferences, a well studied model of preferences over multiple types of resources in artificial intelligence and economics. We show that when preferences are O-legal, meaning that agents share a common importance order on the types, sequential mechanisms satisfy the desirable properties of anonymity, neutrality, non-bossiness, or Pareto-optimality if and only if every local mechanism also satisfies the same property, and they are applied sequentially according to the order O. Our main results are that under O-legal lexicographic preferences, every mechanism satisfying strategyproofness and a combination of these properties must be a sequential composition of local mechanisms that are also strategyproof, and satisfy the same combinations of properties.

Sequential Mechanisms for Multi-type Resource Allocation

To help facilitate play and learning, game-based educational activities often feature a computational agent as a co-player. Personalizing this agent's behavior to the student player is an active area of research, and prior work has demonstrated the benefits of personalized educational interaction across a variety of domains. A critical research challenge for personalized educational agents is real-time student modeling. Most student models are designed for and trained on only a single task, which limits the variety, flexibility, and efficiency of student player model learning.    In this paper we present a research project applying transfer learning methods to student player models over different educational tasks, studying the effects of an algorithmic "multi-task personalization" approach on the accuracy and data efficiency of student model learning. We describe a unified robotic game system for studying multi-task personalization over two different educational games, each emphasizing early language and literacy skills such as rhyming and spelling. We present a flexible Gaussian Process-based approach for rapidly learning student models from interactive play in each game, and a method for transferring each game's learned student model to the other via a novel instance-weighting protocol based on task similarity. We present results from a simulation-based investigation of the impact of multi-task personalization, establishing the core viability and benefits of transferrable student models and outlining new questions for future in-person research.

An Autonomous Drive Balancing Strategy for the Design of Purpose in Open-ended Learning Robots

Next from AAMAS 2021

The Seeing-Eye Robot Grand Challenge: Rethinking Automated Care (Blue Sky Ideas Track)

Similar lecture

Open-VICO: An Open-Source Gazebo Toolkit for Multi-Camera-based Skeleton Tracking in Human-Robot Collaboration

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES