Kunikazu Kobayashi, Koji Nakano, Takashi Kuremoto and Masanao Obayashi (February 1st 2010). Objective-based Reinforcement Learning System for Cooperative Behavior Acquisition, Application of Machine Learning, Yagang Zhang, IntechOpen, DOI: 10.5772/8615. 

Ling Cen, Minghui Dong, Haizhou Li Zhu Liang Yu and Paul Chan (February 1st 2010). Machine Learning Methods in the Application of Speech Emotion Recognition, Application of Machine Learning, Yagang Zhang, IntechOpen, DOI: 10.5772/8613.

Malin Bjornsdotter (February 1st 2010). Machine Learning for Functional Brain Mapping, Application of Machine Learning, Yagang Zhang, IntechOpen, DOI: 10.5772/8605.

New Zealand

We propose a novel method for analyzing and visualizing the complexity of standard reinforcement learning (RL) benchmarks based on score distributions. A large number of policy networks are generated by randomly guessing their parameters, and then evaluated on the benchmark task; the study of their aggregated results provide insights into the benchmark complexity. Our method guarantees objectivity of evaluation by sidestepping learning altogether: the policy network parameters are generated using Random Weight Guessing (RWG), making our method agnostic to (i) the classic RL setup, (ii) any learning algorithm, and (iii) hyperparameter tuning. We show that this approach isolates the environment complexity, highlights specific types of challenges, and provides a proper foundation for the statistical analysis of the task&#39;s difficulty. We test our approach on a variety of classic control benchmarks from the OpenAI Gym, where we show that small untrained networks can provide a robust baseline for a variety of tasks. The networks generated often show good performance even without gradual learning, incidentally highlighting the triviality of a few popular benchmarks.

AAMAS 2020

Analyzing Reinforcement Learning Benchmarks using Random Weight Guessing

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


AAAI 2023

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

We propose a novel method for analyzing and visualizing the complexity of standard reinforcement learning (RL) benchmarks based on score distributions. A large number of policy networks are generated by randomly guessing their parameters, and then evaluated on the benchmark task; the study of their aggregated results provide insights into the benchmark complexity. Our method guarantees objectivity of evaluation by sidestepping learning altogether: the policy network parameters are generated using Random Weight Guessing (RWG), making our method agnostic to (i) the classic RL setup, (ii) any learning algorithm, and (iii) hyperparameter tuning. We show that this approach isolates the environment complexity, highlights specific types of challenges, and provides a proper foundation for the statistical analysis of the task's difficulty. We test our approach on a variety of classic control benchmarks from the OpenAI Gym, where we show that small untrained networks can provide a robust baseline for a variety of tasks. The networks generated often show good performance even without gradual learning, incidentally highlighting the triviality of a few popular benchmarks.

technical paper

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

Optimal Swarm Strategy for Dynamic Target Search and Tracking

"We present VerSecTis -- a new experimental tool for the verification of timed security protocols' (TSP) modelled by Timed Interpreted Systems (TIS).
In addition to the TSP's time-independent properties, our tool can also examine the time dependencies of the TSP's executions on which their security depends.
The verification method consists of a new TSPs' modelling method and
a translation of the reachability problem for TIS to the Satisfiability Modulo Theories problem. We also deliver nineteen TSPs to verify, and we plan to expand the tool with further protocols."

VerSecTis - An Agent Based Model Checker for Security Protocols (Demo)

**Please click on the button bellow to see this lecture on SlidesLive:**

[![](https://assets.underline.io/uploads/markdown_image/1/image/08e10ad349922f32e7322b77b8df9019.png)](https://slideslive.com/38946802/boston-dynamics)

**Abstract:**

Hurricanes are devastating natural disasters. In deciding how to respond to a hurricane, in particular whether and when to evacuate, a decision-maker must weigh often highly uncertain and contradictory information about the future path and intensity of the storm. To effectively plan to help people during a hurricane, it is crucial to be able to predict and understand this evacuation decision. To this end, we propose a computational model of human sequential decision-making in response to a hurricane based on a Partial Observable Markov Decision Process (POMDP) that models concerns, uncertain beliefs about the hurricane, and future information. We evaluate the model in two ways. First, hurricane data from 2018 was used to evaluate the model's predictive ability on real data. Second, a simulation study was conducted to qualitatively evaluate the sequential aspect of the model. The evaluation with hurricane data shows that our proposed features are significant predictors and they can predict the data well, within and across distinct hurricane datasets. The simulation results show that, across different setups, our model generates predictions on the sequential decisions making aspect that align with expectations qualitatively and suggests the importance of modeling information.

A Computational Model of Human Hurricane Evacuation Decision

We propose a new perspective on adversarial attacks against deep reinforcement learning agents. Our main contribution is CopyCAT, a targeted attack able to consistently lure an agent into following an outsider's policy. It is pre-computed, therefore fast inferred, and could thus be usable in a real-time scenario. We show its effectiveness on Atari 2600 games in the novel read-only setting. In this setting, the adversary cannot directly modify the agent's state - its representation of the environment - but can only attack the agent's observation - its perception of the environment. Directly modifying the agent's state would require a write-access to the agent's inner workings and we argue that this assumption is too strong in realistic settings.

CopyCAT: Taking Control of Neural Policies with Constant Attacks

Multiagent systems can use commitments as the core of a general coordination infrastructure, supporting both cooperative and non-cooperative interactions. For agents whose objectives are aligned, and where one agent can help another achieve greater reward by sacrificing some of its own reward, their cooperative commitment should be chosen to maximize their joint reward. We present a solution to the problem of how cooperative agents can efficiently find an (approximately) optimal commitment by querying about carefully selected commitment choices. We prove structural properties of the agents' values as functions of the parameters of the commitment specification, and develop a greedy method for composing a query with provable approximation bounds, that we empirically show can find nearly optimal commitments in a fraction of the time that methods that lack our insights require.

Efficient Querying for Cooperative Commitments

In scenarios that require teamwork, we usually have at hand a variety of specific tasks, for which we need to form a team in order to carry out each one. Here we target the problem of matching teams with tasks within the context of education, and specifically in the context of forming teams of students and allocating them to internship programs. We provide a formalization of the problem, and show the computational hardness of solving it optimally.
Thereafter, we propose a heuristic anytime algorithm that generates an initial team allocation which later on attempts to improve in an iterative process.
Moreover, we present the results of our evaluation, and compare the results with a state-of-the-art optimizer.

TAIP: an anytime algorithm for allocating student teams to internship programs

Theory of Mind is commonly defined as the ability to attribute mental states (e.g., beliefs, goals) to oneself, and to others. A large body of previous work---from the social sciences to artificial intelligence---has observed that Theory of Mind capabilities are central to providing an explanation to another agent or when explaining that agent’s behaviour. In this paper, we build and expand upon previous work by providing an account of explanation in terms of the beliefs of agents and the mechanism by which agents revise their beliefs given possible explanations. Our account satisfies a number of important desiderata which we posit must be met by accounts of explanation that purport to employ Theory of Mind, in contrast to previous work which has addressed some but not all of these desiderata.

Towards the Role of Theory of Mind in Explanation

Maximizing Plan Legibility in Stochastic Environments (Extended Abstract)

In this work, the Distributed Bayesian (D-Bay) algorithm is applied to an autonomous search use case. Within the use case multiple unmanned aerial vehicles equipped with cameras cooperatively search an area and minimize the required time. The use case is mod- eled within the continuous Distributed Constraint Optimization Problem (DCOP) framework. This framework extends the (discrete) DCOP framework by allowing variables with continuous domains. Compared to similar DCOP solvers, the characteristics of the D- Bay algorithm are well suited for the use case and allow for the implementation on autonomous vehicles with limited resources (computational power, memory, and communication bandwidth). Experimental results are given and these results are used to vali- date a simulation environment. Within the simulation environment various scenarios are implemented. The D-Bay algorithm was able to find solutions within 3.5 % of the optimal solution with a limited amount of samples per agent.

The Distributed Bayesian algorithm: simulation and experimental results

Multirobot systems for covering environments are increasingly used in applications like cleaning, industrial inspection, patrolling, and precision agriculture. The problem of covering a given environment using multiple robots can be naturally formulated and studied as a multi-Traveling Salesperson Problem (mTSP). In a mTSP, the environment is represented as a graph and the goal is to find tours (starting and ending at the same depot) for the robots in order to visit all the vertices with minimum global cost, namely the length of the longest tour. The mTSP is an NP-hard problem for which several approximation algorithms have been proposed. These algorithms usually assume generic environments, but tighter approximation bounds can be reached focusing on specific environments. In this paper, we address the case of environments composed of sub-parts, called modules, that can be reached from each other only through some linking structures. Examples are multi-floor buildings, in which the modules are the floors and the linking structures are the staircases or the elevators, and floors of large hotels or hospitals, in which the modules are the rooms and the linking structures are the corridors. We focus on linear modular environments, with the modules organized sequentially, presenting an efficient (with polynomial worst-case time complexity) algorithm that finds a solution for the mTSP whose cost is within a bounded distance from the cost of the optimal solution. The main idea of our algorithm is to allocate disjoint "blocks" of adjacent modules to the robots, in such a way that each module is covered by only one robot. We experimentally compare our algorithm against some state-of-the-art algorithms for solving mTSPs in generic environments and show that it is able to provide solutions with lower makespan and spending a computing time several orders of magnitude shorter.

Analyzing Reinforcement Learning Benchmarks using Random Weight Guessing

Downloads

Next from AAMAS 2020

Optimal Swarm Strategy for Dynamic Target Search and Tracking

Similar lecture

Spectral Feature Augmentation for Graph Contrastive Learning and Beyond

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Analyzing Reinforcement Learning Benchmarks using Random Weight Guessing

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Optimal Swarm Strategy for Dynamic Target Search and Tracking

Similar lecture

Spectral Feature Augmentation for Graph Contrastive Learning and Beyond

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads