Spain

The widespread use of experimental benchmarks in AIresearch  has  created  new  competition  and  collaboration  dynamics that are still poorly understood. In this paper we provide an innovative methodology to explore this dynamics and analyse the way different entrants in these competitions, from academia to tech giants,behave and react depending on their own or others’ achievements.We perform an analysis of over twenty popular benchmarks in AI,linking their underlying research papers. We identify links between researchers and institutions (i.e., communities) beyond the standard co-authorship relations, and we explore a series of hypotheses about their behaviour as well as some aggregated results in terms of activity, breakthroughs and efficiency. As a result, we detect and characterise the dynamics of research communities at different levels of abstraction, including organisation, affiliation, trajectories, results and activity.

ECAI 2020

The Scientometrics of AI Benchmarks: Unveiling the Underlying Mechanics of AI Research

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


AAAI 2023

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

technical paper

ECAI2020 featured a full programme of technical papers, as well as many AI-related events of interest for researchers, students and all attendants who are interested on contemporary AI, including:




**An extensive programme of workshops, tutorials, and invited speakers**

**The Prestigious Applications of Intelligent Systems conference (PAIS)**

**The Frontiers in AI track sessions**

**The Lab To Market event**

**The Women in AI time**

**The EU Challenges forum**

… and many more!

ECAI 2020 featured a full programme of technical papers, as well as many AI-related events of interest for researchers, students and all attendants who are interested on contemporary AI.

The performance of the machine learning algorithms can be empirically evaluated via test campaigns: conventional testing approaches allow to obtain, from annotated databases, a first estimate of the operating environment of artificial intelligence systems. However, as those data are hardly I .i.d, additional scores could be useful to offer complementary performance assessment typically for intelligent systems performing critical functions. This paper proposes an approach to generate, without annotation phase, realistic corner cases to complete the exploration of the operating environment. The results of the proof of concept provided for optical character recognition highlight the interest of test data generation and automatic scenario selection methods, including GAN and adversarial approaches, to address this evaluation need.

Generating corner cases for crash-testing deep networks

"The field of Artificial Intelligence (AI) and, in particular, the Machine Learning area, counts on a wide range of performance metrics and benchmark data sets to assess the problem-solving effec- tiveness of its solutions. However, the appearance of research centres, projects or institutions addressing AI solutions from a multidisci- plinary and multi-stakeholder perspective suggests a new approach to assessment comprising ethical guidelines, reports or tools and frame- works to help both academia and business to move towards a re- sponsible conceptualisation of AI. They all highlight the relevance of three key aspects: (i) enhancing cooperation among the different stakeholders involved in the design, deployment and use of AI; (ii) promoting multidisciplinary dialogue, including different domains of expertise in this process; and (iii) fostering public engagement to maximise a trusted relation with new technologies and practitioners. In this paper, we introduce the Observatory on Society and Artifi- cial Intelligence (OSAI), an initiative grew out of the project AI4EU aimed at stimulating reflection on a broad spectrum of issues of AI (ethical, legal, social, economic and cultural). In particular, we de- scribe our work in progress around OSAI and suggest how this and similar initiatives can promote a wider appraisal of progress in AI. This will give us the opportunity to present our vision and our modus operandi to enhance the implementation of these three fundamental dimensions.
"

Progressing Towards Responsible AI

This work presents an operational definition that facilitates a common understanding of AI and its measurability, paving the way for a transparent and comparable monitoring activity in the context of AI Watch, the European Commission’s knowledge service to monitor the development, uptake and impact of artificial intelligence (AI) for Europe. The operational definition is constituted by a taxonomy and a list of related keywords. The method that we propose is iterative, considering that AI is a dynamic field, so its definition should be updated over time to capture the rapid AI evolution. The method consists of the following steps: (i) qualitative analysis of AI definitions and subdomains emanating from reports with academic, industrial and policy perspectives, (ii) selection of definition as starting point, taxonomy formation, and identification of representative keywords in AI with a natural language processing method, and (iii) taxonomy and keywords validation. This results in a unique taxonomy that represents and interconnects all the AI subdomains from political, research and industrial perspective and enables the efficient mapping of the AI landscape of economic agents across different technological areas.

Setting the Boundaries of the AI Landscape: An Operational Definition for the European Commission’s AI Watch

This short paper aims at introducing some labelling concepts developed and used by the authors. Indeed, developing a conceptual framework for interdisciplinary research in history is a much-needed strategy in order to ensure that historians use all vestiges from the past regardless their origin or support for the construction of historical discourse. Fixing the semantics of historical knowledge is the first unavoidable step to build a new scenario in which NLP tasks enable more efficient data gathering and exploitation.

The Semantics of Historical Knowledge. Labelling Strategies for Interdisciplinary and Digital Research in History

Presentation is about our chatbot creation platform and its dialog management techniques. The framework enables creating chatbots without coding using intent-slot model. It contains intent detection and dialog act classifier modules to understand user utterances more accurately. Also the framework includes and analytics tool to monitor and cluster past conversation data to discover topics and training sentences.

Domain-Independent, Task-Oriented Chatbot Creation and Conversation Policy Management Framework

"Static word embeddings, like word2vec or GloVe, are often assessed when solving syntactic and semantic analogies. Among the latter, we are interested in relations that one would find in lexical-semantic knowledge bases like WordNet, also covered in analogy test sets for English. This paper describes the creation of a new test for assessing Portuguese word embeddings, dubbed TALES, with an exclusive focus on lexical-semantic relations, acquired from lexical resources in Portuguese.
It further reports on the performance of methods previously used for solving analogies, with pre-trained Portuguese word embeddings, when applied to the created dataset, an experiment that revealed that TALES is challenging to solve.
Results achieved are briefly discussed, with conclusions that may be useful for developing new approaches for this problem, possibly new embeddings, as well as future versions of TALES."

 TALES: Test Set of Portuguese Lexical-Semantic Relations for Assessing Word Embeddings

"In this presentation we will show the application of the Thematic Theory and the Thematic Progression Theory to automatically annotate discourse features, which enable the inference of the implicit conceptual schemas of texts.
"

A new approach for extracting the conceptual schema of texts based on the linguistic Thematic Progression theory

Open information extraction (OpenIE) is a task of extracting structured information from unstructured texts independently of the domain. Recent advances have applied Deep Learning for Natural Language tasks improving the state-of-the-art, even though those methods usually require a large and high-quality corpus. The construction of an OpenIE dataset is a tedious and error-prone task, and one technique employed concerns the extractions from rule-based techniques and manual validation of those extraction triples. As low-resource languages usually lack available datasets for the application of high-performance Deep Learning techniques, our intuition is that a low-resource model based-on multilingual information can learn generalizations across languages and benefits from cross-lingual data. Moreover, we would like to interpret the set of generalized information gathered from multilingual learning to increase the Open IE classification task. In this paper, we introduce TabOIEC, a multilingual classifier based on generic morpho-syntactic features. Our classifier carries a glass-box method which can provide interpretation about some of the classifier decisions. We evaluate our approach through a small corpus of Open IE extractions for the English, Spanish, and Portuguese languages. Our results consider that for all languages our approach improves F1 measures, particularly for monolinguality. Experiments on Zero-shot learning provide evidence that our TabOIEC generalizes the classifier on other languages than that trained, although there is a shy transfer learning among them. Experiments on multilinguality do reduce the cost of training, however, in our experiments were difficult to provide appropriate generalizations. 

Explainable Open IE Classifier with Morpho-syntactic Rules

Maintaining blood glucose in a target range is essential for people living with Type 1 diabetes in order to avoid excessive periods in hypoglycemia and hyperglycemia which can result in severe complications. Accurate blood glucose prediction can reduce this risk and enhance early interventions to improve diabetes management. However, due to the complex nature of glucose metabolism and the various lifestyle related factors which can disrupt this, diabetes management still remains challenging. In this work we propose a novel deep learning model to predict future BG levels based on the historical continuous glucose monitoring measurements, meal ingestion, and insulin delivery. We adopt a modified architecture of the generative adversarial network that comprises of a generator and a discriminator. The generator computes the BG predictions by a recurrent neural network with gated recurrent units, and the auxiliary discriminator employs a one-dimensional convolutional neural network to distinguish between the predictive and real BG values. Two modules are trained in an adversarial process with a combination of loss. The experiments were conducted using the OhioT1DM dataset that contains the data of six T1D contributors over 40 days. The proposed algorithm achieves an average root mean square error (RMSE) of 18.34 ± 0.17 mg/dL with a mean absolute error (MAE) of 13.37 ± 0.18 mg/dL for the 30-minute prediction horizon (PH) and an average RMSE of 32.31 ± 0.46 mg/dL with a MAE of 24.20 ± 0.42 for the 60-minute PH. The results are compared for clinical relevance using the Clarke error grid which confirms the promising performance of the proposed model.

Blood Glucose Prediction for Type 1 Diabetes Using Generative Adversarial Networks

We developed a system for predicting the prognosis for more than 2000 COVID-19 patients hospitalised at ASST Spedali Civili di Brescia, one of the hospitals with more COVID-19 patients in Italy. Our system is based on lab tests and X-Ray data and focuses on death risk at different times during the hospitalisation using randomized ensemble of decision trees.

The Scientometrics of AI Benchmarks: Unveiling the Underlying Mechanics of AI Research

Downloads

Next from ECAI 2020

Generating corner cases for crash-testing deep networks

Similar lecture

Balanced Column-Wise Block Pruning for Maximizing GPU Parallelism

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

The Scientometrics of AI Benchmarks: Unveiling the Underlying Mechanics of AI Research

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ECAI 2020

Generating corner cases for crash-testing deep networks

Similar lecture

Balanced Column-Wise Block Pruning for Maximizing GPU Parallelism

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads