Portugal

The evaluation of algorithms has been debated in computer science for decades. Recent years have seen an increasing use of different kinds of evaluations based on empirical research methods and sample data. Various applied areas of computing struggle with different concerns of validity, presumably because an overarching theoretical framework has been missing. In this keynote, we introduce a framework for researching algorithms that highlights several validity concerns. We discuss, in turn, research strategies to address these concerns. Our framework can be useful for any type of applied research in computer science that builds on the evaluation of algorithms.

IC3K 2021

Validity Concerns for Evaluating Algorithms

[![](https://assets.underline.io/uploads/markdown_image/1/image/9a88ae95fbde06e80f4bcf6d4f7d133a.png)](https://certificateaixia.paperform.co/)
The certificate will be provided by December 17. 
In case you need it urgently please contact: francesca.gasparini@unimib.it. 

 

The 20th International Conference of the Italian Association for Artificial Intelligence (**AIxIA 2021**) is organized by AIxIA (Associazione Italiana per l’Intelligenza Artificiale), which is a non-profit scientific society founded in 1988 and devoted to the promotion of Artificial Intelligence. The society aims to increase the public awareness of AI, encourage the teaching of it and promote research in the field.

 
 



**General Chair**

Stefania Bandini, University of Milano-Bicocca, Italyand RCAST - The University of Tokyo, Japan


**Program Chairs**

Viviana Mascardi, University of Genova, Italy; Matteo Palmonari, University of Milano-Bicocca, Italy; Giuseppe Vizzari, University of Milano-Bicocca, Italy

**Organizing Committee**

Francesca Gasparini, University of Milano-Bicocca, Italy (**Chair**); Stefano Ferilli, University of Bari, Italy; Marta Giltri, University of Milano-Bicocca, Italy; Daniela Briola, University of Insubria, Italy


**Publicity and Communication Committee**

Elisabetta Fersini, University of Milano-Bicocca, Italy (**Chair**); Angelo Ferrando, University of Genova, Italy; Francesca Alessandra Lisi, University of Bari, Italy; Giovanni Stilo, University of L'Aquila, Italy; Marco Viviani, University of Milano-Bicocca, Italy


**Workshop Chairs**

Gabriella Cortellessa, ISTC-CNR, Rome, Italy; Silvia Rossi, University of Naples Federico II, Italy; Fabio Stella, University of Milano-Bicocca, Italy


**PC Members**

Fulvio Mastrogiovanni, University of Genova, Italy; Roberto Basili, University of Rome Tor Vergata, Italy; Nicola Gatti, Polytechnic University of Milan, Italy; Stefano Bistarelli, University of Perugia, Italy; Piero Poccianti, Consorzio Operativo Gruppo MPS, Italy; Francesco Scarcello, University of Calabria, Italy; Flavio S. Correa Da Silva, University of São Paulo, Brazil; Elena Cabrio, Université Côte d'Azur, CNRS, Inria, I3S, France; Antonio Lieto, University of Turin, Italy; Diego Calvanese, Free University of Bozen-Bolzano, Italy; Alessio Micheli, University of Pisa, Italy; Francesca A. Lisi University of Bari Aldo Moro, Italy; Mauro Vallati, University of Huddersfield, United Kingdom; Bernardo Magnini, Fondazione Bruno Kessler (FBK), Italy; Luigia C. Aiello, Sapienza University of Rome, Italy; Rafael H. Bordini, Pontifical Catholic University of Rio Grande do Sul, Brazil; Chiara Ghidini, Fondazione Bruno Kessler (FBK), Italy; Federico Bergenti, University of Parma, Italy; Francesco Ricca, University of Calabria, Italy; Andrea Burattin, Technical University of Denmark, Denmark; Angelo Oddi, ISTC-CNR, Italian National Research Council, Italy; Farshad Badie, Aalborg University, Denmark; Emanuel Sallinger, University of Oxford, United Kingdom; Marin Lujak, University Rey Juan Carlos, Spain; Giancarlo Fortino, University of Calabria, Italy; Salvatore Gaglio, University of Palermo, Italy; Johannes K. Fichte, TU Dresden, Germany; Matteo Baldoni, University of Turin, Italy; Fabrizio Riguzzi, Univeristy of Ferrara, Italy; Ivana Dusparic, Trinity College Dublin, Ireland; Serena Villata, CNRS - Laboratoire d'Informatique, Signaux et Systèmes de Sophia-Antipolis, France; Paolo Rosso, Technical University of Valencia, Spain; Ana L. C. Bazzan, Federal University of Rio Grande do Sul, Brazil; Gianluigi Greco, University of Calabria, Italy; Stefano Cagnoni, University of Parma, Italy; Gian Luca Pozzato, University of Turin, Italy; Salvatore Ruggieri, University of Turin, Italy; Claudio Di Ciccio, Sapienza University of Rome, Italy; Eloisa Vargiu, Cetaqua, Water Technology Center, Spain; Marco Maratea, University of Genova, Italy; Andrea Roli, University of Bologna, Italy; Viviana Patti, University of Turin, Italy; Stefania Costantini, University of L'Aquila, Italy; Antonius Weinzierl, Vienna University of Technology, Austria; Silvia Rossi, University of Naples Federico II, Italy; Giovanni Semeraro, University of Bari, Italy; Luciano Serafini, Fondazione Bruno Kessler (FBK), Italy; Laura Giordano, University of Eastern Piedmont, Italy; Ernesto Jimenez-Ruiz, City, University of London, United Kingdom; Marco Gavanelli, University of Ferrara, Italy; Federico Chesani, University of Bologna, Italy

Please register with AIxIA to be able to enter the conference on Underline.io. If you have already registered, please login to Underline.io to be able to see full conference content. In case you do not have an Underline account, please create one. Thank you

Please login to your Underline account

AIxIA 2021

The 20th International Conference of the Italian Association for Artificial Intelligence (AIxIA 2021) is organized by AIxIA (Associazione Italiana per l’Intelligenza Artificiale), which is a non-profit scientific society founded in 1988 and devoted to the promotion of Artificial Intelligence. The society aims to increase the public awareness of AI, encourage the teaching of it and promote research in the field.

keynote

The purpose of the IC3K is to bring together researchers, engineers and practitioners on the areas of Knowledge Discovery, Knowledge Engineering and Knowledge Management.

Big Data, Data Science and Security are being integrated to solve many of the security and privacy challenges. For example, machine learning techniques are being applied to solve security problems such as malware analysis and insider threat detection. However, there is also a major concern that the machine learning techniques themselves could be attacked. Therefore, the machine learning techniques are being adapted to handle adversarial attacks. This area is known as adversarial machine learning. In addition, privacy of the individuals is also being violated through these machine learning techniques as it is now possible to gather and analyze vast amounts of data and therefore privacy enhanced data science techniques are being developed.To assess the developments on the integration of Big Data, Data Science and Security over the past decade and apply them to the Internet of Transportation, the presentation will focus on four aspects. First it will examine the developments on applying Data Science techniques for detecting cyber security problems such as insider threat detection as well as the advances in adversarial machine learning. Some developments on privacy aware and policy-based data management frameworks will also be discussed. Next it will discuss how cloud technologies may be used to securely and privately share the information for various Big Data applications such as the Internet of Things. Finally, it will describe ways in which Big Data, Data Science and Security could be incorporated into these applications.

Integrating Big Data, Data Science and Cyber Security with Applications in Cloud and Internet of Things

There is growing interest in large interdisciplinary data portals to support a wide range of research and research communities. In this talk, we will introduce emerging best practices in building large data and knowledge portals that aim to be long lived. We will provide examples in multiple disciplines of semantic knowledge portals that are collaboratively designed and maintained by interdisciplinary teams.

Designing Knowledge Enabled Systems for Interdisciplinary Settings

A data-driven computing paradigm has risen to rival the previously dominant knowledge-driven paradigm. As modern society becomes ever more digitalized, all kinds of human activities are increasingly mediated by a mix of data-driven and knowledge-driven software. The wondrous technical capabilities of software systems are evoking utopian dreams as well as dystopian nightmares. Users, regulators, and the general public are calling for responsible AI and transparency and accountability from software professionals.In software engineering, requirement analysis techniques have served to mediate between human needs and wants on one hand, and technical system design on the other. This talk will outline the role of social modeling in requirements engineering, as illustrated by the i* framework. In i*, actors depend on each other to achieve what they want. By applying i* modeling, one can explore the potential impacts of various system designs on stakeholders, in search of design options that would better meet the desires and aspirations of all concerned.The widespread adoption of data-driven computing in today’s highly digitalized world suggests that requirements analysis needs to engage with the full cognitive cycle – the cycle by which actors sense and interpret data, generate knowledge, execute actions, and in turn producing new data. Various software technologies, data-driven and knowledge-driven, enhance and complement specific aspects of the cognitive cycles of humans and organizations, often bringing about spectacular gains as well as vulnerabilities. Social modeling encompassing the full cognitive cycle can potentially help system designers and stakeholders jointly explore design options in order to derive the most benefits while avoiding negative consequences when developing today’s advanced software applications.

Premium content

Validity Concerns for Evaluating Algorithms

Downloads

Next from IC3K 2021

Integrating Big Data, Data Science and Cyber Security with Applications in Cloud and Internet of Things

Similar lecture

Fairness and Popularity Bias in Recommender Systems: an Empirical Evaluation

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES