**Abstract:**

Migration has played a pivotal role in shaping both cultural and genetic diversity throughout human history. Results from population genetics show that even low rates of migration between populations lead to the homogenization of their genetic compositions. On the cultural side, we know that conformity helps maintain differences between groups. However, equivalent insights into the effect of migration on cultural composition remain limited. To address this gap, I have developed a simulation model that explores the joint evolution of genetic and cultural traits, investigating how varying strengths of conformity and migration rates influence differentiation both within and between populations

**Speaker's social media:**

Twitter: @camilla_cenni

LIVE - Cultural drivers of stone-tool use in Balinese long-tailed macaques

Event extraction (EE), as a crucial information extraction (IE) task, aims to identify event triggers and their associated arguments from unstructured text, subsequently classifying them into pre-defined types and roles. In the biomedical domain, EE is widely used to extract complex structures representing biological events from literature. Due to the complicated semantics and specialized domain knowledge, it is challenging to construct biomedical event extraction datasets. Additionally, most existing biomedical EE datasets primarily focus on cell experiments or the overall experimental procedures. Therefore, we introduce AniEE, an event extraction dataset concentrated on the animal experiment stage. We establish a novel animal experiment customized entity and event scheme in collaboration with domain experts. We then create an expert-annotated high-quality dataset containing discontinuous entities and nested events and evaluate our dataset on the recent outstanding NER and EE models.

AniEE: A Dataset of Animal Experimental Literature for Event Extraction

Recently, improving the robustness of policies across different environments attracts increasing attention in the reinforcement learning (RL) community. Existing robust RL methods mostly aim to achieve the max-min robustness by optimizing the policy’s performance in the worst-case environment. However, in practice, a user that uses an RL policy may have different preferences over its performance across environments. Clearly, the aforementioned max-min robustness is oftentimes too conservative to satisfy user preference. Therefore, in this paper, we integrate user preference into policy learning in robust RL, and propose a novel User-Oriented Robust RL (UOR-RL) framework. Specifically, we define a new User-Oriented Robustness (UOR) metric for RL, which allocates different weights to the environments according to user preference and generalizes the max-min robustness metric. To optimize the UOR metric, we develop two different UOR-RL training algorithms for the scenarios with or without a priori known environment distribution, respectively. Theoretically, we prove that our UOR-RL training algorithms converge to near-optimal policies even with inaccurate or completely no knowledge about the environment distribution. Furthermore, we carry out extensive experimental evaluations in 6 MuJoCo tasks. The experimental results demonstrate that UOR-RL is comparable to the state-of-the-art baselines under the average-case and worst-case performance metrics, and more importantly establishes new state-of-the-art performance under the UOR metric.

User-Oriented Robust Reinforcement Learning

Safety comes first in many real-world applications involving autonomous agents. Despite a large number of reinforcement learning (RL) methods focusing on safety-critical tasks, there is still a lack of high-quality evaluation of those algorithms that adheres to safety constraints at each decision step under complex and unknown dynamics. In this paper, we revisit prior work in this scope from the perspective of state-wise safe RL and categorize them as projection-based, recovery-based, and optimization-based approaches, respectively.  Furthermore, we propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection. This novel technique explicitly enforces hard constraints via the deep unrolling architecture and enjoys structural advantages in navigating the trade-off between reward improvement and constraint satisfaction. To facilitate further research in this area,  we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit, a toolkit that provides off-the-shelf interfaces and evaluation utilities for safety-critical tasks. We then perform a comparative study of the involved algorithms on six benchmarks ranging from robotic control to autonomous driving. The empirical results provide an insight into their applicability and robustness in learning zero-cost-return policies without task-dependent handcrafting. The project page is available at \url{https://sites.google.com/view/saferlkit}.

Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks

Cooperative multi-agent reinforcement learning (c-MARL) is widely applied in safety-critical scenarios, thus the analysis of robustness for c-MARL models is profoundly important. However, robustness certification for c-MARLs has not yet been explored in the community. In this paper, we propose a novel certification method, which is the first work to leverage a scalable approach for c-MARLs to determine actions with guaranteed certified bounds. c-MARL certification poses two key challenges compared with single-agent systems: (i) the accumulated uncertainty as the number of agents increases; (ii) the potential lack of impact when changing the action of a single agent into a global team reward. These challenges prevent us from directly using existing algorithms. Hence, we employ the false discovery rate (FDR) controlling procedure considering the importance of each agent to certify per-state robustness and propose a tree-search-based algorithm to find a lower bound of the global reward under the minimal certified perturbation. As our method is general, it can also be applied in single-agent environments. We empirically show that our certification bounds are much tighter than those of the state-of-the-art RL certification solutions. We also run experiments on two popular c-MARL algorithms: QMIX and VDN, in two different environments, with two and four agents. The experimental results show that our method produces a meaningful guaranteed robustness for all models and environments. Our tool CertifyCMARL is available at https://github.com/TrustAI/CertifyCMARL.

Certified Policy Smoothing for Cooperative Multi-Agent Reinforcement Learning

As the complexity of pacemaker devices continues to grow, the importance of capturing its functional correctness requirement formally cannot be overestimated. The pacemaker system specification document by \emph{Boston Scientific} provides a widely accepted set of specifications for pacemakers. 
As these specifications are written in a natural language, they are not amenable for automated verification, synthesis, or reinforcement learning of pacemaker systems. This paper presents a formalization of these requirements for a dual-chamber pacemaker in \emph{duration calculus} (DC), a highly expressive real-time specification language.
The proposed formalization allows us to automatically translate pacemaker requirements into executable specifications as stopwatch automata, which can be used to enable simulation, monitoring, validation, verification and automatic synthesis of pacemaker systems. 
The cyclic nature of the pacemaker-heart closed-loop system results in DC requirements that compile to a decidable subclass of stopwatch automata. We present shield reinforcement learning (shield RL),  a shield synthesis based reinforcement learning algorithm, by automatically constructing safety envelopes from DC specifications. 


Correct-by-Construction Reinforcement Learning of Cardiac Pacemakers from Duration Calculus Requirements

**Author(s):** Chira, A; Kirby, K.; Epperlein, T.; Kuehr, F; Bräuer, J. Introduction: Dogs show an extraordinaire capacity to understand, communicate, and cooperate with us. However, what we know about the social-cognitive skills of dogs comes from observations and experiments performed in WEIRD societies. Around the world though, dogs are kept for various functions, they are perceived and treated differently, suggesting cross-cultural diversity in the strength of dog-human relationships, and perhaps departures from the ‘typical’ suite of skills dogs are thought to possess. Materials & Methods: Here we address these issues by collecting data on the function and perception of dogs in 124 globally distributed societies using the eHRAF cross-cultural database. We hypothesize that dog-human relationships will be closer in societies where dog functions require intense cooperation between dogs and humans, i.e., hunting and herding versus guarding, defense, and carry animals. We characterize close dog-human relationships as increased primary caregiving (or positive care), decreased negative treatment, and an increased extent to which dogs are viewed as friends or kin. Results & Discussion: The number of functions dogs perform associates with closer dog-human relationships. Further, the presence of herding (and to a lesser extent, guarding herds, and carry roles) increases the odds of positive care given to dogs, while using dogs for hunting is linked with higher odds of dogs being treated akin to a person. These findings support the idea of cross-cultural variation in dog-human relationships and show how the strength of our bond with dogs can be linked with the function dogs fill in societies. The results are a first step towards challenging the notion that all dogs are the same, and fuel questions about the universality of cognitive and social skills we commonly associate with our canine friends.

Beyond WEIRD dogs - Cultural Differences In Dog Human Interactions

 Irons (2001) and Sosis (2003) proposed that some actions are “hard” or “costly” to fake signals of altruism. If it is harder for non-altruists to perform an action than for altruists, associating with other signalers can solve cooperation dilemmas. This invites intuitive reference to costly signaling theory from animal behavior, but this theory describes signals of quality, not intent. Further, costly and hard are open to interpretation. It may take extra effort to perform the signal dishonestly. Deception may be unlikely to be innovated. Socially learning of deception may be inhibited as deceivers present as honest. Signal receivers may invest in false-signal detection. These all present distinct dynamics. Multiple interpretations of the verbal model of costly/hard to fake signals are mathematically formalized and implemented in social learning models. Analytic solutions for evolutionary equilibrium are derived for each. Under a range of conditions for some but not all interpretations, such signals and altruism spread through a population. Where deception entails some cost or is less likely to be learned hard to fake-signals can lead to group success through altruistic cooperation. A particularly interesting case is where deceptive parasitic free-riders paradoxically benefit the group by amplifying recruitment via effective false advertising: “Useful Liars”.

Credibility and Deception: modeling deceptive signals of intention to cooperate

 Conflict is a persistent feature of primate societies. Although sociability has valuable benefits, such as protection against predators or access to resources, maintaining peaceful relationships with other group members can be costly for individuals. Conflict episodes within a social group promote physical contact and changes in social dynamics. All social animals engage in episodes of conflict and aggression for dominance, control of resources, food, sexual partners, or territories. To restore social balance, primates display appeasement gestures that may repair relationships after fights. One way to restore harmony after is through sexual behavior. Non-conceptive sexual behaviors have been described in most primate species. These are explained by its communicative or sociosexual functions, which include tension regulation, dominance signaling, ties strengthening, and reconciliation among others. In this study, we systematically reviewed literature on primate conflict and resolution to determine how frequently sexual behavior has appeasing function. Results showed that closely related species like bonobos (Pan paniscus) and chimpanzees (P. troglodytes) routinely use sexual behavior for conflict resolution. However, it is still necessary to investigate reconciliatory sexual behavior in other primate species.

Make love, not war: Conflict and sexual reconciliation in primates

Introduction. This poster celebrates advances in the study of culture in non-human taxa indexed by articles appearing in the two highest profile science journals. The present century has witnessed a remarkable surge in the quality and quantity of these studies and their reception in such journals. Approach. Titles of articles on animal culture together with brief abstracts and key illustrations extracted from Nature and Science are displayed, to illustrate the range of taxa and cultural phenomena addressed. Findings. Twenty titles are available in the present century alone, a majority from just the last decade. Reports of culture cover an expanding range of species including primates, cetaceans, birds and insects. Behavioural domains are no less diverse, from mate choice to foraging techniques, vocal repertoires and migration routes. Evidence extends to gene-culture co-evolution. Implications for conservation are now under review by IUCN and UN. Conclusions. The studies collated here index the growing recognition that cultural phenomena are widespread in animals. They provide a robust and exciting platform from which to further explore the scope of culture in non-human species.

Animal Culture Studies Advance in the Pages of Nature and Science

Social primates exhibit a vast behavioural and cultural repertoire. Understanding the ontogeny of sociability in primates can bring important findings in comparative studies between human and non-human cultures. In this context, our objective was to analyse the development of the social personality of 12 wild capuchin monkeys (Sapajus libidinosus) and test if sex and monkey identity affected the social personality. We used videos recorded in the wild during the first 3 years of life of 6 males and 6 females and analysed the rate of emission of affiliative social behaviours, such as grooming, lipsmacking and others. We used GLMMs and tested the repeatability of behaviours for statistical analysis. The results showed low repeatability for monkey identity and sex as random effects The population exhibited high values of inter and intraindividual variance in the emission of behaviours, especially during the first year and a half of life, that is the infancy of S. libidinosus. The repeatability values stabilised in the juvenile period, but remained low. Our results show that social personality is not yet established in the first three years of life for these primates. Longitudinal studies following the individuals until adulthood are necessary to understand if and when personality is consolidated in capuchin monkeys.

The development of social personality in a wild neotropical primate

**Author(s):** Ottoni, E; Coutinho, P; Falótico, T We surveyed tool use by 3 groups of capuchin monkeys in S. das Confusões Nat. Park (Brazil), a savannah population equidistant from two long-term study sites - Faz. Boa Vista, where monkeys are expert nutcrackers, but do not use digging stones nor probes, and S. da Capivara, where all 3 are customary. We employed direct observation, camera traps, indirect observation of tool use sites, and two experiments, one on induced seeds' excavation, the other on probe use, based in a previous comparative study (FBV x PNSC). Nutcracking was observed as expected, but no probing. Stone-aided digging of arthropods and tubers was indirectly registered in the range of one group; one of the other groups (the only one to take part) did not use tools in the digging experiment. In the molasses-probing task, monkeys did not succeed, except by one innovator, but made inept attempts to hit the box with sticks or stones. The results on stone-aided digging are not conclusive but suggest resource-availability causes. The apparent absence of customary probe use, with no clear environmental causes, seems to point to a lower probability of the establishment of a tradition with less cultural niche construction.

Underline digital video library

LIVE - Cultural drivers of stone-tool use in Balinese long-tailed macaques

AniEE: A Dataset of Animal Experimental Literature for Event Extraction

User-Oriented Robust Reinforcement Learning

Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks

Certified Policy Smoothing for Cooperative Multi-Agent Reinforcement Learning

Correct-by-Construction Reinforcement Learning of Cardiac Pacemakers from Duration Calculus Requirements

Beyond WEIRD dogs - Cultural Differences In Dog Human Interactions

Credibility and Deception: modeling deceptive signals of intention to cooperate

Make love, not war: Conflict and sexual reconciliation in primates

Animal Culture Studies Advance in the Pages of Nature and Science

The development of social personality in a wild neotropical primate

Comparing the toolkits of three savannah populations of bearded capuchin monkeys (Sapajus libidinosus): environmental and traditional factors.

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES