New Zealand

Athanasios A. Pantelous and Grigoris I. Kalogeropoulos (October 1st 2009). The Benefits of Schooling: A Human Capital Alocation into a Continuous Optimal Control Framework, Technology, Education and Development, Aleksandar Lazinica and Carlos Calafate, IntechOpen, DOI: 10.5772/7296. 

Sie Long Kek, Sy Yi Sim, Wah June Leong and Kok Lay Teo (November 22nd 2019). Conjugate Gradient Approach for Discrete Time Optimal Control Problems with Model-Reality Differences [Online First], IntechOpen, DOI: 10.5772/intechopen.89711. 

Maria Elena Lopez, Rafael Barea, Luis Miguel Bergasa, Manuel Ocana and Maria Soledad Escudero (February 1st 2007). Global Navigation of Assistant Robots using Partially Observable Markov Decision Processes, Mobile Robots: Perception & Navigation, Sascha Kolski, IntechOpen, DOI: 10.5772/4777. 

We live in a world full of complex social systems. Achieving opti- mal control in a complex social system is challenging due to the difficulty in modeling and optimization. To capture the complex social system dynamics accurately and succinctly, we model the decision-making problem as a partially observable discrete event decision process. To withstand the curse of dimensionality in high- dimensional belief state spaces and to optimize the problem in an amenable searching space, we investigate the connections between the value function of a partially observable decision process and that in the corresponding fully-observable scenario, and reduce the optimal control of a partially observable discrete event decision process to a policy optimization with a specially formed fully ob- servable decision process and a belief state estimation. When tested in real-world transportation scenarios, in comparison with other state-of-the-art approaches, our proposed algorithm leads to the least average time on-road, the largest number of vehicles at work during work hours and the fewest training epochs to converge to the highest total rewards per episode.


AAMAS 2020

Optimal Control in Partially Observable Complex Social Systems

complex social systems; optimal control; partially observable

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

technical paper

We introduce a model for agent-environment systems where the agents are implemented via feed-forward ReLU neural networks and the environment is non-deterministic. We study the verification problem of such systems against CTL properties. We introduce a bounded fragment of CTL and show its usefulness in identifying shallow bugs in the system. We present a novel parallel algorithm for MILP-based verification of agent-environment systems, present an implementation, and report the experimental results obtained against a variant of the VerticalCAS use-case.


Formal Verification of Neural Agents in Non-deterministic Environments

Coupon has been a major marketing tool that promotes sales for new and repeated buyers and proven effective in numerous realistic scenarios. In this paper, we mainly focus on the problem of designing coupons to maximize the revenue in second-price auction. 

Firstly, we derive the dominant strategies of bidders if they are provided with coupons in second-price auction and prove that the revenue optimization problem with coupons for all the bidders is NP-complete. Secondly, we cast the problem of designing coupons to maximize revenue into a learning framework. With well-designed loss functions, we perform theoretical analysis of its properties and propose corresponding algorithms to solve the problem. Finally, with both synthetic data and industrial data, extensive experiments are conducted to demonstrate their effectiveness.


Learning to Design Coupons in Online Advertising Markets

Advertising has become a dominant source of revenue generation on the Internet. Billions of advertisement slots are sold via auctions. And there are many pricing methods ,e.g., CPM (cost-per-mille), CPC (cost-per-click), CPA (cost-per-action), OCPM (optimized cost-per-mille) and so on. In this paper, we study the OCPM method (i.e., advertisers bid for conversions while pay per mille) under VCG auction. However, automatically bid in each view to maximize advertisers' conversions while still meet their target cost-per-conversion in feeds is difficult. To deal with these difficulties, we propose a reinforcement learning framework, i.e., RSDRL (ROI-sensitive distributional reinforcement learning). By making full use of the characteristics of auction rules which are missed by other methods, we design a reward function to surrogate conversion events and a bid generation method based on theoretical results. We also provide some theoretical results to guide hyperparameter tuning. Last, we validate RSDRL on a large industrial dataset with millions of auctions. Plenty of experiments (both online and offline) are used to evaluate the performance of our framework and RSDRL yields substantially better results than compared algorithms.


Optimized Cost per Mille in Feeds Advertising

The plan recognition task is to infer an actor’s plan and goal given observations about its behavior. We submit that in some cases, for plan recognition to be effective and complete, it must appeal to a notion of epistemics to i) recognize epistemic goals, where the actor is trying to achieve some state of knowledge or belief; and ii) model the observer, and its knowledge of the actor, as first class elements of the plan recognition process. To this end, we formalize the notion of Epistemic Plan Recognition, which builds on two growing areas of research: epistemic planning and plan recognition. Our epistemic plan recognition specification appeals to an epistemic logic framework to represent agent beliefs. To realize our specification, we cast the epistemic plan recognition problem as an epistemic planning problem, whose solutions can be generated using existing epistemic planning tools. Finally, we evaluate our approach by utilizing and comparing existing epistemic planners on a diverse set of epistemic plan recognition problems.


Epistemic Plan Recognition

This paper focuses on two-sided matching where one side (a hospital or firm) is matched to the other side (a doctor or worker) so as to maximize a cardinal objective under general feasibility constraints. In a standard model, even though multiple doctors can be matched to a single hospital, a hospital has a responsive preference and a maximum quota. However, in practical applications, a hospital has some complicated cardinal preference and constraints. With such preferences (e.g., submodular) and constraints (e.g., knapsack or matroid intersection), stable matchings may fail to exist. This paper first determines the complexity of checking and computing stable matchings based on preference class and constraint class. Second, we establish a framework to analyze this problem on packing problems, and the framework enables us to access the wealth of online packing algorithms so that we construct approximately stable algorithms as a variant of generalized deferred acceptance algorithm. We further provide some inapproximability results. 


Approximately Stable Matchings with General Constraints

Employing Models of Human Social Motor Behavior for Artifical Agent Trainers

We present a framework for verifying strategic behaviour in an unbounded multi-agent system. We introduce a novel probabilistic semantics for parameterised multi-agent systems and define the corresponding verification problem against two probabilistic variants of alternating-time temporal logic. We define a verification procedure using an abstract model construction. We show that the procedure is complete for one variant of our specification language, and partial for the other. We present an implementation and report experimental results. 


Parameterised Verification of Strategic Properties in Probabilistic Multi-Agent Systems

In computational social choice, shift bribery is the procedure of paying voters to shift the briber’s preferred candidate forward in their preferences so as to make this candidate an election winner; the more general swap bribery procedure also allows one to pay voters to swap other candidates in their preferences. The complexity of swap and shift bribery is well-understood for many voting rules; typically, finding a minimum-cost bribery is computationally hard. We have studied swap and shift bribery in the setting where voters’ preferences are known to be single-peaked or single-crossing. We obtain polynomial-time algorithms for several variants of these problems for classic voting rules, such as Plurality, Borda and Condorcet-consistent rules. In this talk we present a polynomial time algorithm for shift bribery in the setting where voters’ preferences are single-peaked. 
"


Algorithms for Swap and Shift Bribery in Structured Elections

Redistribution mechanisms have been proposed for more efficient resource allocation but not for profit. We consider redistribution mechanism design in a setting where participants are connected and the resource owner is only connected to some of them. In this setting, to make the resource allocation more efficient, the resource owner has to inform the others who are not her neighbours, but her neighbours do not want more participants to compete with them. Hence, the goal is to design a redistribution mechanism such that participants are incentivized to invite more participants and the resource owner does not earn or lose much money from the allocation. In this presentation, I will first show that existing redistribution mechanisms cannot be directly applied in the network setting. Then I will introduce a novel network-based redistribution mechanism such that all participants on the network are invited, the allocation is more efficient and the resource owner has no deficit.


Redistribution Mechanism on Networks

When planning for multi-robot navigation tasks under uncertainty, the presence of multiple robots in the same part of the environment simultaneously can have an adverse effect on navigation performance. We refer to this effect as congestion. Existing solutions fail to model congestion at planning time, which leads to inefficient behaviour. In this presentation, I will detail how to probabilistically model congestion, presenting a structure called the probabilistic reservation table that summarises the plans of robots using continuous-time Markov chains. I will then show how this structure can be used for planning by outlining an approach that, for each robot, sequentially builds and solves a Markov decision process where the transition probabilities are obtained by querying the probabilistic reservation table.


Downloads

Next from AAMAS 2020

Formal Verification of Neural Agents in Non-deterministic Environments

Similar lecture

Learning Complementary Representations of the Past using Auxiliary Tasks in Partially Observable Reinforcement Learning

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Formal Verification of Neural Agents in Non-deterministic Environments

Similar lecture

Learning Complementary Representations of the Past using Auxiliary Tasks in Partially Observable Reinforcement Learning

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads