New Zealand

The circadian rhythm is responsible for the daily variations in metabolism, and circadian rhythm disorders have direct implications for many diseases, such as obesity and mental disorders. The regulation of sleep time is the most common example of the importance of the circadian rhythm for the functioning of the human body. In this sense, this work aims to study a mathematical and computational model based on multiagent simulation that simulates the synchronization and desynchronization of the circadian rhythm in relation to the pain variables. The results from the multiagent simulation of circadian rhythms show that in relation to pain, sleep, especially its biological rhythms, is directly affected by pain. In this way, our mathematical model was able to show that pain causes changes in the circadian rhythm and it can contribute to the medical field analysis.

AAMAS 2020

Circadian Rhythm and Pain: Mathematical Model based on Multiagent Simulation

multiagent simulation

circadian rhythm; pain; multiagent simulation; mathematical model

circadian rhythm

mathematical model

pain

Welcome to the **largest national, multi-specialty research** event featuring research from more than 750 medical students, residents, fellows, and international medical graduates.

Use the buttons below or navigation menu to access the event.
 

[![](https://assets.underline.io/markdown_image/1/image/20e1e77aa954efeac7a15d84c85eb383.jpg)](https://underline.io/events/468/sessions?searchGroup=all)
 
[![](https://assets.underline.io/markdown_image/1/image/068522d81aca0b68a369787c7dfccd1b.jpg)](https://underline.io/events/468/posters?searchGroup=all)
 
[![](https://assets.underline.io/markdown_image/1/image/d3971ece79d5e5575b178bdc2c8bbfae.jpg)](https://underline.io/events/468/help-desk)
 

#### Event Details

The 2024 American Medical Association Research Challenge includes a poster symposium hall and a semifinals hall with the top scored research. **Log in at any time throughout the event to:**

* **View posters and presentations** on a variety of topics and specialties
* **Score posters in the semifinals** to help decide the top five finalists (AMA member exclusive opportunity)
* **Connect with participants** and provide feedback to help advance their research

At the conclusion of this event, the AMA will announce the top five finalists who will advance to the AMA Research Challenge finals, present their research to an elite panel of judges, and compete to win a $10,000 grand prize presented by Laurel Road. 

### Grand Prize Sponsor

AMA would like to thank Laurel Road for sponsoring a $10,000 grand prize for the winner of the AMA Research Challenge. Laurel Road is not involved in winner selection process. The winner will be announced on February 20, 2025 during the AMA Research Challenge finals. Payment of the grand prize will be directly from Laurel Road. See the AMA Research Challenge Official Rules for terms and conditions.

[![](https://assets.underline.io/uploads/markdown_image/1/image/611e331d4a0eba0920f240cfc04b4d77.png)](https://www.laurelroad.com/)

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

**There is no registration fee for this event! **

If you have used the AMA Research Challenge platform previously, please click the ‘Log-in’ button below. If you do not remember your password, please click “forgot password” and follow prompts to reset your password. After resetting your password, please refresh your webpage to access the platform.

Register to explore research in a variety of topics. AMA members get exclusive access to score posters in the semifinals. 

**Please use this link [here](https://www.ama-assn.org/member-benefits/events/ama-research-challenge) to register for free.**

Registration Is Required

AMA Research Challenge 2024 

Join the AMA Research Challenge poster symposium and semifinals to explore top research posters and score posters in the semifinals (an AMA member exclusive opportunity).

technical paper

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

We propose Stable Emergent Policy (STEP) approximation, a distributed policy iteration scheme to stably approximate decentralized policies for partially observable and cooperative multi-agent systems. STEP offers a novel training architecture, where function approximation is used to learn from action recommendations of a decentralized planning algorithm. Planning is enabled by exploiting a training simulator, which is assumed to be available during centralized learning, and further enhanced by reintegrating the learned policies.


A Distributed Policy Iteration Scheme for Cooperative Multi-Agent Policy Approximation

"Neural networks are effective function approximators, but hard to train in the reinforcement learning (RL) context mainly because samples are correlated. For years, scholars have got around this by employing experience replay or an asynchronous parallel-agent system. This paper proposes Discrete-to-Deep Supervised Policy Learning (D2D-SPL) for training neural networks in RL. D2D-SPL discretises the continuous state space into discrete states and uses actor-critic to learn a policy. It then selects from each discrete state an input value and the action with the highest numerical preference as an input/target pair. Finally it uses input/target pairs from all discrete states to train a classifier. D2D-SPL uses a single agent, needs no experience replay and learns much faster than state-of-the-art methods. We test our method with two RL environments, the Cartpole and an aircraft manoeuvring simulator."


Discrete-to-Deep Supervised Policy Learning

Predictive auxiliary tasks have been shown to improve performance in numerous reinforcement learning works, however, this effect is still not well understood. The primary purpose of the work presented here is to investigate the impact that an auxiliary task's prediction timescale has on the agent's policy performance. We consider auxiliary tasks which learn to make on-policy predictions using temporal difference learning. We test the impact of prediction timescale using a specific form of auxiliary task in which the input image is used as the prediction target, which we refer to as temporal difference autoencoders (TD-AE). We empirically evaluate the effect of TD-AE on the A2C algorithm in the VizDoom environment using different prediction timescales. While we do not observe a clear relationship between the prediction timescale on performance, we make the following observations: 1) using auxiliary tasks allows us to reduce the trajectory length of the A2C algorithm, 2) in some cases temporally extended TD-AE performs better than a straight autoencoder, 3) performance with auxiliary tasks is sensitive to the weight placed on the auxiliary loss, 4) despite this sensitivity, auxiliary tasks improved performance without extensive hyper-parameter tuning. Our overall conclusions are that TD-AE increases the robustness of the A2C algorithm to the trajectory length and while promising, further study is required to fully understand the relationship between auxiliary task prediction timescale and the agent's performance.


Work in Progress: Temporally Extended Auxiliary Tasks

Describing an accurate simulation model of the driving behavior of real-world vehicles is a laborious or even impossible task, because a driver reacts to a dynamically changing environment. As multiple external factors determine driving behavior, it is usually difficult to obtain an accurate model, owing to a lack of sensors or inability to collect data. In this paper, we propose a novel technique to combine driving behavior in vehicular Internet of Things (IoT) big data with a multi-agent system. This enables correct and scalable simulation without modeling the behavior of vehicular IoT devices or the environment. We develop an extensible simulation framework, called FlowSim, that demonstrates the application of our technique for a simulation of camera-image data collection from connected cars.


Simulating Vehicular IoT Applications by Combining a Multi-agent System and Big Data

"Patients are often required to follow a medical treatment after discharge, e.g., for a chronic condition, rehabilitation after surgery, or for cancer survivor therapies. The need to adapt to new lifestyles, medication, and treatment routines, can produce an individual burden to the patient, who is often at home without the full support of healthcare professionals. Although technological solutions -in the form of mobile apps and wearables- have been proposed to mitigate these issues, it is essential to consider individual characteristics, preferences, and the context of a patient in order to oer personalized and effective support. The specific events and circumstances linked to an individual prole can be abstracted as a patient trajectory, which can contribute to a better understanding of the patient, her needs, and the most appropriate personalized support. Although patient trajectories have been studied for different illnesses and conditions, it remains challenging to effectively use them as the basis for data analytics methodologies in decentralized eHealth systems. In this work, we present a novel approach based on the multi-agent paradigm, considering patient trajectories as the cornerstone of a methodology for modelling eHealth support systems. In this design, semantic representations of individual treatment pathways are used in order to exchange patient-relevant information, potentially fed to AI systems for prediction and classification tasks. This paper describes the major challenges in this scope, as well as the design principles of the proposed agent-based architecture, including an example of its
use through a case scenario for cancer survivors support."


Agent-based modeling for Ontology-driven analysis of patient trajectories

Multi-agent coordination is prevalent in many real-world applications. However, such coordination is challenging due to its combinatorial nature. An important observation in this regard is that agents in the real world often only directly affect a limited set of neighbouring agents. Leveraging such loose couplings among agents is key to making coordination in multi-agent systems feasible.
In this work, we focus on learning to coordinate.
Specifically, we consider the multi-agent multi-armed bandit framework, in which fully cooperative loosely-coupled agents must learn to coordinate their decisions to optimize a common objective. 
We propose multi-agent Thompson sampling (MATS), a new Bayesian exploration-exploitation algorithm that leverages loose couplings. We empirically show that MATS outperforms the state-of-the-art algorithm, MAUCE, on two synthetic benchmarks with Bernoulli-distributed rewards. Our results show that MATS improves significantly upon state-of-the-art coordination methods in terms of performance, demonstrating the value of using MATS in applications with sparse neighbourhood structures.


Thompson Sampling for Factored Multi-Agent Bandits

Reactive behaviour in BDI-based models and architectures adopted in Agent Programming is typically modelled in terms of reactive plans not bound to any specific goal. In this paper we present and discuss an extension of the plan model used in BDI in which goal-based plans encapsulate both proactive and reactive behaviour. This brings important benefits both in the practice of agent program- ming and in supporting agent reasoning at runtime. The paper first introduced the model formally and then its evaluation through two concrete implementations, based on two existing agent program- ming platforms i.e. Jason and ASTRA.

Encapsulating Reactive Behaviour in Goal-based Plans in BDI Agents Programming

Reinforcement Learning (RL) has been widely used to solve sequential decision-making problems. However, RL algorithms suffer from poor sample efficiency and require a long time to learn a suitable policy, especially when multiple agents are learning without prior knowledge. This problem can be alleviated through reusing knowledge from other agents during the learning process. One notable approach is advising actions based on a teacher-student relationship, where the decision of a student agent during learning is aided by an experienced teacher agent. A critical assumption in teacher-student paradigm is that the communication may be limited, so that a student may wait for a while and learn by itself before receiving the next advice. More importantly, in some noisy or stochastic environments, the student may not be able to mas- ter the advised actions when they are only performed once. We propose three methods for agents choosing between learning by exploration, asking for advice and reusing previous advice. The results show that our approaches significantly outperform existing advising methods without reusing advice.


Learning by Reusing Previous Advice in Teacher-Student Paradigm

"This paper evaluates the applicability of reinforcement learning (RL) to multi-product inventory management in supply chains. The novelty of this problem with respect to supply chain literature is (i) we consider concurrent inventory management of a large number (hundreds) of products under realistic constraints such as shared capacity, and (ii) the number of products (size of the problem) can change frequently, implying that the RL agent needs to work in this regime without retraining. We approach the problem as a special class of dynamical system control, and explain why the generic problem cannot be satisfactorily solved using classical optimisation techniques. Subsequently, we formulate the problem in a
reinforcement learning framework that can be used for parallelised decision-making, and use the advantage actor critic (A2C) and deep Q-network (DQN) algorithms with quantised action spaces to solve the problem. Experiments on scales between 100 and 220 products
show that these approaches perform better than other baseline algorithms. They are also able to transfer learning without retraining, when the number of products change."

Using Reinforcement Learning for a Large Variable-Dimensional Inventory Management Problem

Recent advances in artificial intelligence (AI) have drawn attention to the need for AI systems to be understandable to human users. The explainable AI (XAI) literature aims to enhance human understanding and human-AI team performance by providing users with necessary information about AI system behavior. Simultaneously, the human factors literature has long addressed important considerations that contribute to human performance, including how to determine human informational needs.
 Drawing from the human factors literature, we propose a three-level framework for the development and evaluation of explanations about AI system behavior. Our proposed levels of XAI are based on the informational needs of human users, which can be determined using the levels of situation awareness (SA) framework from the human factors literature. Based on our levels of XAI framework, we also propose a method for assessing the effectiveness of XAI systems.

Downloads

Next from AAMAS 2020

A Distributed Policy Iteration Scheme for Cooperative Multi-Agent Policy Approximation

Similar lecture

Relationship between Circadian Misalignment in Nightshift Workers and Metabolic Syndrome

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

A Distributed Policy Iteration Scheme for Cooperative Multi-Agent Policy Approximation

Similar lecture

Relationship between Circadian Misalignment in Nightshift Workers and Metabolic Syndrome

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads