United Kingdom

Green Security Games (GSGs) have been successfully used in the protection of valuable resources such as fisheries, forests, and wildlife. Real-world deployment involves both resource allocation and subsequent coordinated patrolling with communication in the presence real-time, uncertain information. Previous game models do not address both of these stages simultaneously. Furthermore, adopting existing solution strategies is difficult since they do not scale well for larger, more complex variants of the game models. We propose a novel GSG model to address these challenges. We also present a novel algorithm, CombSGPO, to compute a defender strategy for this  game model. CombSGPO performs policy search over a multidimensional, discrete action space to compute an allocation strategy that is best suited to a best-response patrolling strategy for the defender,  learnt by training a multi-agent Deep Q-Network. We show via experiments that CombSGPO converges to better strategies and is more scalable than comparable approaches. From a detailed analysis of the coordination and signaling behavior learnt by CombSGPO, we find that strategic signaling emerges in the final learnt strategy.

AAMAS 2021

Reinforcement Learning for Unified Allocation and Patrolling in Signaling Games with Uncertainty

AAMAS is the largest and most influential conference in the area of agents and multiagent systems, bringing together researchers and practitioners in all areas of agent technology and providing and internationally renowned high-profile forum for publishing and finding out about the latest developments in the field. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).  

![](https://assets.underline.io/uploads/markdown_image/1/image/ce4604bd54468a006164d16e080204fb.jpg)

Please click on the button below to register for this event.

If you are registered and see this page please check the account email you are logged in with. It must be the same as for the registration.

If you still have trouble accessing the contact please contact support at aamas2021@underline.io 

technical paper

Modeling vessel movement in a maritime environment is an extremely challenging task given the complex nature of vessel behavior. Several existing multiagent maritime decision making frameworks require access to an accurate traffic simulator. We develop a system using electronic navigation charts to generate realistic and high fidelity vessel traffic data using Generative Adversarial Networks (GANs). Our proposed Ship-GAN uses a conditional Wasserstein GAN to model a vessel's behavior. The generator can simulate the travel time of vessels across different maritime zones conditioned on vessels' speeds and traffic intensity. Furthermore, it can be used as an accurate simulator for prior decision making approaches for maritime traffic coordination, which used less accurate model than our approach. Experiments performed on the historical data from heavily trafficked Singapore strait show that our Ship-GAN system generates data whose statistical distribution is close to the real data distribution, and better fit than prior methods.

Ship-GAN: Generative Modeling Based Maritime Traffic Simulator

In this paper, we demonstrate a Symbolic Reinforcement Learning (SRL) architecture for safe control in Radio Access Network (RAN) applications. In our automated tool, a user can select a high-level safety specifications expressed in Linear Temporal Logic (LTL) to shield an RL agent running in a given cellular network with aim of optimizing network performance, as measured through certain Key Performance Indicators (KPIs). In the proposed architecture, network safety shielding is ensured through model-checking techniques over combined discrete system models (automata) that are abstracted through reinforcement learning. We demonstrate the user interface (UI) helping the user set intent specifications to the architecture and inspect the difference in allowed and blocked actions.

Symbolic Reinforcement Learning for Safe RAN Control

In this study, we leverage multi-objective reinforcement learning to create tunable agents, i.e., agents that can adopt a range of different behaviours according to the designer's preferences, without the need for retraining. We apply this technique to sequential social dilemmas, settings where there is inherent tension between individual and collective rationality. Learning a single fixed policy in such settings leaves one at a significant disadvantage if the opponents' strategies change after learning is complete. In our work, we demonstrate empirically that the tunable agents framework allows easy adaption between cooperative and competitive behaviours in sequential social dilemmas without the need for retraining, allowing a single trained agent model to be adjusted to cater for a wide range of behaviours and opponent strategies.

Tunable Behaviours in Sequential Social Dilemmas using Multi-Objective Reinforcement Learning

In this paper we consider a particular class of problems called
 (MAGB) which constitutes a modified version of the Bernoulli MAB problem where two new elements must be taken into account:the
 and the
 The agent has an initial budget that evolves in time following the received rewards, which can be either +1 after a
 or -1 after a
 The problem can also be seen as a MAB version of the classic
 game. The contribution of this paper is a preliminary analysis on the probability of being ruined given the current budget and observations,   and the proposition of an alternative regret formulation, combining the classic regret notion with the expected loss due to the probability of being ruined. Finally, standard state-of-the-art methods are experimentally compared using the proposed metric.

Gambler Bandits and the Regret of Being Ruined

Agents that interact with other agents often do not know a priori what the other agents' strategies are, but have to maximise their own online return while interacting with and learning about others. The optimal adaptive behaviour under uncertainty over the other agents' strategies w.r.t. some prior can in principle be computed using the Interactive Bayesian Reinforcement Learning framework. Unfortunately, doing so is intractable in most settings, and existing approximation methods are restricted to small tasks. To overcome this, we propose to meta-learn (alongside the policy) approximate belief inference by combining sequential and hierarchical VAEs. We show empirically that our approach can learn a factorised belief model that separates the other agent's permanent and temporal structure, and outperforms methods that sample from the approximate posterior or do not have this hierarchical structure. A full version of this work can be found in Zintgraf et al. (2021).

Deep Interactive Bayesian Reinforcement Learning via Meta-Learning

Touch is an essential sense to the social development and well-being of individuals, acting as a communicative channel for emotions and empathy. Our overall objective is to enhance the ability of embodied conversational agents (ECAs) to bond with humans. To reach such an objective, we have endowed an ECA with the capacity to touch and be touched in a social interaction with a human in an immersive environment. By drawing inspiration from literature on human-human social touch and related works on human-agent tactile interactions, we have developed a framework for a touching ECA able to perceive when and how a user touches it and decide when and how to respond accordingly. This paper focuses on going beyond a simple bi-directional touch interaction through a decision model that actually adapts its behaviour to the content of the interaction, the level of rapport and the human's touch avoidance sensibility. This enables an actual interactive loop between the agent and the human.

Decision Model for a Virtual Agent that can Touch and be Touched

The envy-free, Pareto-efficient allocation of indivisible goods leads to computationally hard problems. There is a big variety of modeling issues, such as agent-specific utility functions or (high numbers of) different types of goods. In recent work, Bredereck et al. [ACM EC~2019] addressed this topic by showing (theoretical) fixed-parameter tractability results for "high-multiplicity fair allocation", exploiting parameters such as number of agents or maximum absolute   utility values. To this end, they used a number of tools from (theoretical) integer linear programming. We "engineer" their work towards practical usefulness, thereby being able to solve all real-world instances from the state-of-art online platform "spliddit.org for provably fair solutions". Besides providing the foundations for a fast tool for fair allocations, we also offer a flexible framework with the possibility to relax fairness or efficiency demands so to, e.g., allow tradeoffs between fairness and social welfare. Moreover, our framework provides ways to interpret and explain "solution paths" which makes it possible to perform further explorations in cases when no envy-free and efficient allocations exist.

High-Multiplicity Fair Allocation Made More Practical

In this work, we define the consensus-prevention problem, which examines the canonical swarm robotic consensus problem from an adversarial point of view: how (if at all) is it possible to lead a swarm into a disagreement, that is, prevent them from reaching an agreement. We focus on consensus-prevention in physically grounded tasks, concentrating on influencing the direction of movement of a flocking swarm and guaranteeing that the swarm will never converge to the same direction by the use of external, predefined agents, referred to as diverting agents.

Spatial Consensus-Prevention in Robotic Swarms

The pursuit of values drives human behavior and promotes cooperation. Existing research is focused on general (e.g., Schwartz) values that transcend contexts. However, context-specific values are necessary to (1) understand human decisions, and (2) engineer intelligent agents that can elicit human values and take value-aligned actions.

Axies: Identifying and Evaluating Context-Specific Values

Multi-agent reinforcement learning (MARL) has been increasingly explored to learn the cooperative policy towards maximizing a certain global reward. Many existing studies take advantage of graph neural networks (GNN) in MARL to propagate critical collaborative information over the interaction graph, built upon inter-connected agents. Nevertheless, the vanilla GNN approach yields substantial defects in dealing with complex real-world scenarios since the generic message passing mechanism is ineffective between heterogeneous vertices and, moreover, simple message aggregation functions are incapable of accurately modeling the combinational interactions from multiple neighbors. While adopting complex GNN models with more informative message passing and aggregation mechanisms can obviously benefit heterogeneous vertex representations and cooperative policy learning, it could, on the other hand, increase the training difficulty of MARL and demand more intense and direct reward signals compared to the original global reward. To address these challenges, we propose a new cooperative learning framework with pre-trained heterogeneous observation representations. Particularly, we employ an encoder-decoder based graph attention to learn the intricate interactions and heterogeneous representations that can be more easily leveraged by MARL. Moreover, we design a pre-training with local actor-critic algorithm to ease the difficulty in cooperative policy learning. Extensive experiments over real-world scenarios demonstrate that our new approach can significantly outperform existing MARL baselines as well as operational research solutions that are widely-used in industry.

Reinforcement Learning for Unified Allocation and Patrolling in Signaling Games with Uncertainty

Downloads

Next from AAMAS 2021

Ship-GAN: Generative Modeling Based Maritime Traffic Simulator

Similar lecture

Classifying the Complexity of the Possible Winner Problem on Partial Chains

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Reinforcement Learning for Unified Allocation and Patrolling in Signaling Games with Uncertainty

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2021

Ship-GAN: Generative Modeling Based Maritime Traffic Simulator

Similar lecture

Classifying the Complexity of the Possible Winner Problem on Partial Chains

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads