New Zealand


Learning effective policies for real-world problems is still an open challenge for the field of reinforcement learning (RL). The main limitation being the amount of data needed and the pace at which that data can be obtained. In this paper, we study how to build lightweight simulators of complicated systems that can run sufficiently
fast for deep RL to be applicable. We focus on domains where agents interact with a reduced portion of a larger environment while still being affected by the global dynamics. Our method combines the use of local simulators with learned models that mimic the influence of the global system. The experiments reveal that incorporating this idea into the deep RL workflow can considerably accelerate the training process and presents several opportunities for the future.

AAMAS 2022

Speeding up Deep Reinforcement Learning through Influence-Augmented Local Simulators

influence.

simulation

reinforcement learning

[![](https://assets.underline.io/uploads/markdown_image/1/image/21d13e03491d7e9acf6b8c46c7979c25.png)](https://docs.google.com/spreadsheets/d/1SCDDfkQk7es4J6RULKHVx1RzbB0Mutla--Sle5SVfb0/edit?usp=sharing)

**RoboCup 2021 will be Worldwide!**

We are excited to announce that RoboCup 2021 is a fully remote event with RoboCup competitions and activities taking place all over the world. Everyone is invited to participate in, or observe, the [competition leagues](https://2021.robocup.org/leagues) and the [symposium](https://2021.robocup.org/symposium)!

**RoboCup Symposium**

The 24th RoboCup International Symposium will be held in conjunction with RoboCup 2021 in a purely online setting. Scientific papers will be presented reporting innovative, original research with relevance to robotics and artificial intelligence. Within the described scope of topics we also encouraged submissions of high-quality overview articles, papers describing real-world research, and papers reporting theoretical results. In addition to the main track with regular research papers, there will be a special track focused on systems, data, and benchmarks featuring papers describing novel, open-source hardware and software systems, as well as datasets and benchmarks relevant to the community. In addition to research paper presentations, the RoboCup Symposium 2021 includes keynote talks by Dieter Fox, Jean-Paul Laumond, and Stefanie Tellex, and several interactive sessions.

**RoboCup Introduction Video**
<iframe width="995" height="516" src="https://www.youtube.com/embed/Cnh43cCsDKU" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>


Registration is required for entrance into RoboCup 2021. <br><br>
**Fees:** 
* Major Faculty or Symposium Faculty - US$50
* Major Student, Junior Mentor, Junior Student, or Symposium Student - US$20 

***Members of qualified teams:** Please contact your league for your registration password and proceed to register with your team. *<br>
***All non-team members:** Please register as "Symposium Only".*

You need to be registered to explore this event!

RoboCup 2021

The 24th RoboCup International Symposium will be held in conjunction with RoboCup 2021 in a purely online setting. Scientific papers will be presented reporting innovative, original research with relevance to robotics and artificial intelligence. Within the described scope of topics we also encouraged submissions of high-quality overview articles, papers describing real-world research, and papers reporting theoretical results. In addition to the main track with regular research papers, there will be a special track focused on systems, data, and benchmarks featuring papers describing novel, open-source hardware and software systems, as well as datasets and benchmarks relevant to the community. In addition to research paper presentations, the RoboCup Symposium 2021 includes keynote talks by Dieter Fox, Jean-Paul Laumond, and Stefanie Tellex, and several interactive sessions.

poster

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

The AAMAS conference series was initiated in 2002 in Bologna, Italy as a joint event comprising the 6th International Conference on Autonomous Agents (AA), the 5th International Conference on Multiagent Systems (ICMAS), and the 9th International Workshop on Agent Theories, Architectures, and Languages (ATAL).

Subsequent AAMAS conferences have been held in Melbourne, Australia (July 2003), New York City, NY, USA (July 2004), Utrecht, The Netherlands (July 2005), Hakodate, Japan (May 2006), Honolulu, Hawaii, USA (May 2007), Estoril, Portugal (May 2008), Budapest, Hungary (May 2009), Toronto, Canada (May 2010), Taipei, Taiwan (May 2011), Valencia, Spain (June 2012), Minnesota, USA (May 2013), Paris, France (May 2014), Istanbul, Turkey (May 2015), Singapore (May 2016), São Paulo (2017) and Stockholm, Sweden (2018), Montreal (May 2019), Auckland (May 2020, Virtual), London (May 2021, Virtual).
<br>
<br>



A ticket is required to attend this event, please register using the link below:

https://aamas2022-conference.auckland.ac.nz/attending/registration/

Registration Is Required

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).


Following Zhang and Grossi~(AAAI 2021), we study in more depth a variant of weighted voting games in which agents' weights are induced by a transitive support structure. 
This class of simple games is notably well suited to study the relative importance of agents in the liquid democracy framework. We first propose a pseudo-polynomial time algorithm to compute the Banzhaf and Shapley-Shubik indices for this class of game. Then, we study a bribery problem, in which one tries to maximize/minimize the voting power/weight of a given agent by changing the support structure under a budget constraint. We show that these problems are computationally hard and provide several parameterized complexity results.

Computation and Bribery of Voting Power in Delegative Simple Games

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1613.pdf

**Abstract:**



Proof-of-Work as a Stigmergic Consensus Algorithm


We propose the notion of deep reinforcement learning-based strategy templates for multi-issue bilateral negotiation. Each strategy template consists of a set of interpretable parameterized tactics that are used to decide an optimal action at any time. This contrasts with existing work that only estimates the threshold utility for those tactics that require it. As a result, we build automated agents for multi-issue negotiations that can adapt to different negotiation domains without the need to be pre-programmed.

Deep Learnable Strategy Templates for Multi-Issue Bilateral Negotiation


A financial system is represented by a network, where nodes cor- respond to banks, and directed labeled edges correspond to debt contracts between banks. Once a payment schedule has been de- fined, where we assume that a bank cannot refuse a payment to- wards one of its lenders if it has sufficient funds, the liquidity of the system is defined as the sum of total payments made in the network. Maximizing systemic liquidity is a natural objective of any financial authority, so, we study the setting where the financial authority offers bailout money to some bank(s) or forgives the debts of others in order to maximize liquidity, and examine efficient ways to achieve this. We investigate the approximation ratio provided by the greedy bailout policy compared to the optimal one, and we study the computational hardness of finding the optimal debt-removal and budget-constrained optimal bailout policy, respectively.
We also study financial systems from a game-theoretic stand- point. We observe that the removal of some incoming debt might be in the best interest of a bank, if that helps one of its borrowers remain solvent and avoid costs related to default. Assuming that a bank’s well-being (i.e., utility) is aligned with the incoming pay- ments they receive from the network, we define and analyze a game among banks who want to maximize their utility by strategically giving up some incoming payments. In addition, we extend the previous game by considering bailout payments. After formally defining the above games, we prove results about the existence and quality of pure Nash equilibria, as well as the computational complexity of finding such equilibria.

Forgiving Debt in Financial Network Games


Multi-agent Markov Decision Process (MMDP) has been an effective way of modelling sequential decision making algorithms for multi-agent cooperative environments. However, challenges such as exponential size of action space and dynamic changes limit the efficacy of proposed solutions. This paper propose a scalable and robust algorithm that can effectively solve MMDPs in real time. Simulation, pruning, and prediction are the three key components of the algorithm. The simulation component enables real time solutions by using a novel iterative pruning technique which in turn makes use of the prediction component trained with self play data. The algorithm is self-sustained as it generates new training data from simulation and gradually becomes better. Furthermore, we show empirical results demonstrating the capabilities of the algorithm and compare them with existing MMDP solvers.

A Simulation Based Online Planning Algorithm for Multi-Agent Cooperative Environments


Multi-agent reinforcement learning (MARL) enables us to create adaptive agents in challenging environments, even when the agents have limited observation. Modern MARL methods have focused on finding factorized value functions. While successful, the resulting methods have convoluted network structures. We take a radically different approach and build on the structure of independent Q-learners. Our algorithm LAN leverages a dueling architecture to represent decentralized policies as separate individual advantage functions w.r.t. a centralized critic that is cast aside after training. The critic works as a stabilizer that coordinates the learning and to formulate DQN targets. This enables LAN to keep the number of parameters of its centralized network independent in the number of agents, without imposing additional constraints like monotonic value functions. 
When evaluated on the SMAC, LAN shows SOTA performance overall and scores more than 80% wins in two super-hard maps where even QPLEX does not obtain almost any wins. Moreover, when the number of agents becomes large, LAN uses significantly fewer parameters than QPLEX or even QMIX. We thus show that LAN's structure forms a key improvement that helps MARL methods remain scalable.

Local Advantage Networks for Cooperative Multi-Agent Reinforcement Learning

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1648.pdf

**Abstract:**



REFORM: Reputation Based Fair and Temporal Reward Framework for Crowdsourcing


Online bipartite-matching platforms are ubiquitous and find applications in important areas such as crowdsourcing and ridesharing. In the most general form, the platform consists of three entities: two sides to be matched and a platform operator that decides the matching. The design of algorithms for such platforms has traditionally focused on the operator’s (expected) profit. Recent reports have shown that certain demographic groups may receive less favorable treatment under pure profit maximization. As a result, a collection of online matching algorithms have been developed that give a fair treatment guarantee for one side of the market at the expense of a drop in the operator’s profit. In this paper, we generalize the existing work to offer fair treatment guarantees to both sides of the market simultaneously, at a calculated worst case drop to operator profit. We consider group and individual Rawlsian fairness criteria. Moreover, our algorithms have theoretical guarantees and have adjustable parameters that can be tuned as desired to balance the trade-off between the utilities of the three sides. We also derive hardness results that give clear upper bounds over the performance of any algorithm.

Rawlsian Fairness in Online Bipartite Matching: Two-sided, Group, and Individual


Many scientific conferences employ a two-phase paper review process, where some papers are assigned additional reviewers after the initial reviews are submitted. Many conferences also design and run experiments on their paper review process, where some papers are assigned reviewers who provide reviews under an experimental condition. In this paper, we consider the question: how should reviewers be divided between phases or conditions in order to maximize total assignment similarity? We make several contributions towards answering this question. First, we prove that when the set of papers requiring additional review is unknown, a simplified variant of this problem is NP-hard. Second, we empirically show that across several datasets pertaining to real conference data, dividing reviewers between phases/conditions uniformly at random allows an assignment that is nearly as good as the oracle optimal assignment. This uniformly random choice is practical for both the two-phase and conference experiment design settings. Third, we provide explanations of this phenomenon by providing theoretical bounds on the suboptimality of this random strategy under certain natural conditions. From these easily-interpretable conditions, we provide actionable insights to conference program chairs about whether a random reviewer split is suitable for their conference.

Near-Optimal Reviewer Splitting in Two-Phase Paper Reviewing and Conference Experiment Design

**Read paper on the following link:**

https://ifaamas.org/Proceedings/aamas2022/pdfs/p1738.pdf

**Abstract:**



Speeding up Deep Reinforcement Learning through Influence-Augmented Local Simulators

Downloads

Next from AAMAS 2022

Computation and Bribery of Voting Power in Delegative Simple Games

Similar lecture

rSoccer: A Framework for Studying Reinforcement Learning in Small and VerySmall Size Robot Soccer

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Speeding up Deep Reinforcement Learning through Influence-Augmented Local Simulators

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2022

Computation and Bribery of Voting Power in Delegative Simple Games

Similar lecture

rSoccer: A Framework for Studying Reinforcement Learning in Small and VerySmall Size Robot Soccer

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads