United States

Artificial intelligence has emerged as a promising technique for helping with healthcare from various aspects such as disease diagnostics and prognostics, personalized treatment and management, therapeutics research and development, disease surveillance and public health policy making, etc. Because AI relies more on the data rather than human judgement, they hold the promise of suffering less from subjective bias and thus reduce healthcare disparities. At the same time, due to the various issue of health data and fragmented nature of our healthcare system, AI can introduce implicit additional algorithmic bias and worsen healthcare disparities. This makes AI a double-edged sword. In this talk, I will introduce existing research and examples from both aspects, and discuss important considerations on designing fair healthcare AI algorithms.

NAACL 2022

Towards Building Trustworthy Machine Learning Models in Medicine: Evaluation vs Explanation

disparities

fairness

The OWSD General Assembly and international conference will be held entirely online for the first time in OWSD's 28-year history, allowing as many of OWSD's 7600+ members to participate as possible.

The conference theme is 'Women, science and development', exploring three areas of relevance: 1) the importance of basic sciences for development; 2) the impact of applied sciences on development and 3) including sex and gender as variables in scientific research.


**Keynote speakers include:** 
* Fabiola Gianotti, Director-General of CERN; 
* Bina Agarwal, an internationally celebrated economist specializing in the political economy of gender, poverty and inequality; and 
* Angela Saini, award-winning author of Inferior: How Science Got Women Wrong and the New Research That’s Rewriting the Story, Superior: The Return of Race Science, and other books. 

OWSD members will present their own research in the three themes during poster sessions throughout the conference. In addition, members will also present their own scientific and personal responses to the COVID-19 pandemic. All members will also have the opportunity to take part in a workshop on increasing their visibility and impact as a scientist, with a session on using podcasts for storytelling by Joy Owango and Adelle Onyango of the Training Centre in Communication (TCC Africa), followed by a session on filmmaking for impact by Nicole Leghissa, OWSD video consultant.

The conference will include a handover of the previous OWSD Executive Board to the new members, elected in June-July 2021. On the last day of the conference (19 November), the OWSD Italy Country Network will be launched. **This will be an in-person event in Trieste.** Italian government representatives and donors are invited to attend. The Trieste event will be coordinated with similar in person events being held in OWSD countries throughout the developing world and streamed online. At least 2,000 OWSD members, women scientists from developing countries around the world, are expected to participate online in the General Assembly and conference.

 

**ABOUT THE PROGRAMME**

In order to approach the complex and wide-ranging challenges of the 2030 Agenda for Sustainable Development, societies around the world must access and harness all the talent and experience available to them. The concerns and perceptions of the greatest number of all possible actors and recipients must be considered in order that the most relevant issues and effective solutions are found. Women are considerably underrepresented as researchers in STEM subjects around the globe; according to UNESCO Institute of Statistics (UIS) data, in 2019 the global average percentage of women researchers was still below 30%. Yet women typically have first hand specialist knowledge of the day-to-day challenges diverse individuals face within communities, and can be key informers in the design and development of their solutions. Moreover, since women are most often the providers of informal community support, if they are not fully informed—and if solutions are developed without their consultation—it may be the case that products, services and solutions are not relevant or appropriate. Technological advances may mean improved machines on large farms, for example, potentially increasing food production, but since the key points of food insecurity in the world are on small-scale farms managed by women throughout the Global South, such capital investments will have little impact in the areas where food production is most critical—and where women’s knowledgeable implementation of appropriate innovative technologies could make the most difference.

 
 


**Keynote Speakers:**
* Fabiola Gianotti, Director-General of CERN; 
* Bina Agarwal, an internationally celebrated economist specializing in the political economy of gender, poverty and inequality; and 
* Angela Saini, award-winning author of Inferior: How Science Got Women Wrong and the New Research That’s Rewriting the Story, Superior: The Return of Race Science, and other books. 

**Eminent Guests:**
* Elena Bonetti, Italian Minister for Family and Equal Opportunities
* Maria Cristina Messa,	Italian Minister of Universities and Research
* Shamila Nair-Bedouelle,	Assistant Director General for Natural Sciences, UNESCO
* Maria Chiara Carrozza,	President, National Research Council of Italy
* Kerstin Jonsson Cisse,	Head of Unit for Globally Sustainable Economic Development, Sida - Sweden
* Naser Faruqui,	Director of Education and Science at IDRC - Canada
* Ylann Schemm,	Director, Elsevier Foundation
* Alberto Quadrio Curzio,	President Emeritus of the Italian Academy of Sciences and Humanities 
* Mohamed Hassan,	President, TWAS the World Academy of Sciences

**Chairs, panelists, interviewers and other speakers:**
Jennifer Thomson, Tonya Blowers, Rana Dajani, Shobhana Narasimhan, Joy Owango, Elizabeth Pollitzer, Kausila Timsina, Elizabeth Bandason, Susana Arrechea, Reinabelle Reyes, Dorine Odongo, Elizabeth Pollitzer, Adelle Onyango, Nicole Leghissa, Jana Rodriguez Hertz, Maryse Nkoua Ngavouka, Kate Shaw, Chao Mbogo, Prativa Pandey, Edna Patricia Matta-Camacho, Amira Shaheen, Wezi Nyambili, Ana Lucía Martinez Abarca, Mashura Shammi, Therese Umuhoza, Dionica Gamboa, Marian Nkansah, Babalola Olubukola Oluranti, Esi Awuah, Fortunate Farirai, Atya Kapley, Hasin Anupama Azhari, Huda Basaleem, Nashwa Eassa, Shymaa Enany, Kleinsy Bonilla, Patricia Castillo-Briceno, Fang Xin

Please register for the conference to be able to enter the event page on Underline.io. If you have already registered, please login to Underline.io to be able to see full conference content. In case you do not have an Underline account, please create one. Thank you

Please login to your Underline account

OWSD 6th GENERAL ASSEMBLY AND INTERNATIONAL CONFERENCE

The conference theme is 'Women, science and development', exploring three areas of relevance: the importance of basic sciences for development; the impact of applied sciences on development, and including sex and gender as variables in scientific research.

invited talk

## Welcome to NAACL 2022!
The Annual Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) is taking place July 10-15, 2022 as a hybrid event, in Seattle, WA and online. We are happy to welcome all of you to conference! 
 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, keynote presentations, a plenary panel on the place of linguistics and symbolic structures in NLP, and an Industrial Track panel on Careers in NLP.
 
Posters (including Findings of NAACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop and will have an oral session and a poster session.
 
The program also features 6 Tutorials and 26 Workshops.
 
 We wish you a wonderful conference!
 
[The NAACL 2022 Organizing Committee](https://2022.naacl.org/committees/organization/)

2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics

The bi-encoder design of dense passage retriever (DPR) is a key factor to its success in open-domain question answering (QA), yet it is unclear how DPR's question encoder and passage encoder individually contributes to overall performance, which we refer to as the \textit{encoder attribution} problem.
The problem is important as it helps us identify the factors that affect individual encoders to further improve overall performance.
In this paper, we formulate our analysis under a probabilistic framework called {\it encoder marginalization}, where 
we quantify the contribution of a single encoder by marginalizing other variables.
First, we find that the passage encoder contributes more than the question encoder to in-domain retrieval accuracy.
Second, we demonstrate how to find the affecting factors for each encoder, where we train DPR with different amounts of data and use encoder marginalization to analyze the results.
We find that positive passage overlap and corpus coverage of training data have big impacts on the passage encoder, while the question encoder is mainly affected by training sample complexity under this setting.
Based on this framework, we can devise data-efficient training regimes:\ for example, we manage to train a passage encoder on SQuAD using 60\% less training data without loss of accuracy.

An Encoder Attribution Analysis for Dense Passage Retriever in Open Domain Question Answering

The widespread use of Artificial Intelligence (AI) in consequential domains, such as
healthcare and parole decision making systems, has drawn intense scrutiny on the
fairness of these methods. However, ensuring fairness is often insufficient as the
rationale for a contentious decision needs to be audited, understood, and defended.
We propose that the attention mechanism can be used to ensure fair outcomes while simultaneously providing feature attributions to account for how a decision was made. Toward this goal, we design an attention-based model that can be leveraged as an attribution framework. It can identify features responsible for both performance and fairness of the model through attention interventions and attention weight manipulation. Using this attribution framework, we then design a post-processing bias mitigation strategy and compare it with a suite of baselines. We demonstrate the versatility of our approach by conducting experiments on two distinct data types, tabular and textual.

Attributing Fair Decisions with Attention Interventions

In an effort to guarantee that machine learning model outputs conform with human moral values, recent work has begun exploring the possibility of explicitly training models to learn the difference between right and wrong. This is typically done in a bottom-up fashion, by exposing the model to different scenarios, annotated with human moral judgements. One question, however, is whether the trained models actually learn any consistent, higher-level ethical principles from these datasets – and if so, what? Here, we probe the Allen AI Delphi model with a set of standardized morality questionnaires, and find that, despite some inconsistencies, Delphi tends to mirror the moral principles associated with the demographic groups involved in the annotation process. We question whether this is desirable and discuss how we might move forward with this knowledge.

Does Moral Code Have a Moral Code? Probing Delphi’s Moral Philosophy

Large-scale surveys are a widely used instrument to collect data from a target audience. Beyond the single individual, an appropriate analysis of the answers can reveal trends and patterns and thus generate new insights and knowledge for researchers. Current analysis practices employ shallow machine learning methods or rely on (biased) human judgment. This work investigates the usage of state-of-the-art NLP models such as BERT to automatically extract information from both open- and closed-ended questions. We also leverage explainability methods at different levels of granularity to further derive knowledge from the analysis model. Experiments on EMS---a survey-based study researching influencing factors affecting a student's career goals---show that the proposed approach can identify such factors both at the input- and higher concept-level.

Explaining Neural NLP Models for the Joint Analysis of Open- and Closed-Ended Survey Answers

Neural rationale models are popular for interpretable predictions of NLP tasks. In these, a selector extracts segments of the input text, called rationales, and passes these segments to a classifier for prediction. Since the rationale is the only information accessible to the classifier, it is plausibly defined as the explanation. Is such a characterization unconditionally correct? In this paper, we argue to the contrary, with both philosophical perspectives and empirical evidence suggesting that rationale models are, perhaps, less rational and interpretable than expected. We call for more rigorous evaluations of these models to ensure desired properties of interpretability are indeed achieved. The code for our experiments is at https://github.com/yimingz89/Neural-Rationale-Analysis.

The Irrationality of Neural Rationale Models

In this paper, we conduct an empirical study on a bias measure, log-likelihood Masked Language Model (MLM) scoring, on a benchmark dataset. Previous work evaluates whether MLMs are biased or not for certain protected attributes (e.g., race) by comparing the log-likelihood scores of sentences that contain stereotypical characteristics with one category (e.g., black) versus another (e.g., white). We
hypothesized that this approach might be too sensitive to the choice of contextual words than the meaning of the sentence. Therefore, we computed the same measure after paraphrasing the sentences with different words but with same meaning. Our results demonstrate that the log-likelihood scoring can be more sensitive to utterance of specific words than to meaning behind a given sentence. Our paper reveals a shortcoming of the current log-likelihood-based bias measures for MLMs and calls for new ways to improve the robustness of it.

An Empirical Study on Pseudo-log-likelihood Bias Measures for Masked Language Models Using Paraphrased Sentences

Conversational recommender systems offer a way for users to engage in multi-turn conversations to find items they enjoy. For users to trust an agent and give effective feedback, the recommender system must be able to explain its suggestions and rationales. We develop a two-part framework for training multi-turn conversational recommenders that provide recommendation rationales that users can effectively interact with to receive better recommendations. First, we train a recommender system to jointly suggest items and explain its reasoning via subjective rationales.
We then fine-tune this model to incorporate iterative user feedback via self-supervised bot-play. Experiments on three real-world datasets demonstrate that our system can be applied to different recommendation models across diverse domains to achieve state-of-the-art performance in multi-turn recommendation. Human studies show that systems trained with our framework provide more useful, helpful, and knowledgeable suggestions in warm- and cold-start settings. Our framework allows us to use only product reviews during training, avoiding the need for expensive dialog transcript datasets that limit the applicability of previous conversational recommender agents.

Self-Supervised Bot-Play for Transcript-Free Conversational Recommendation with Rationales

In this work, we propose a knowledge enhanced generative model to mitigate
these two issues. Our generative model is a unified framework to sequentially generate relational triplets under various relation extraction settings and explicitly utilizes relevant knowledge from Knowledge Graph (KG) to resolve ambiguities.

Enhanced Knowledge for Joint Entity and Relation Extraction

Structured Knowledge has recently emerged as an essential component to support finegrained Question Answering (QA). In general, QA systems query a Knowledge Base (KB) to detect and extract the raw answers as final prediction. However, as lacking of context, language generation can offer a much informative and complete response. In this paper, we propose to combine the power of transfer learning and the advantage of entity placeholders to produce high-quality verbalization of extracted answers from a structured KB. We claim that such approach is especially well-suited for answer generation. Our experiments show 44.25%, 3.26% and 29.10% relative gain in BLEU over the state-of-the- art on the VQuAnDA, ParaQA and VANiLLa datasets, respectively. We additionally provide minor hallucinations corrections in VANiLLa standing for 5% of each of the training and testing set. We witness a median absolute gain of 0.81 SacreBLEU. This strengthens the importance of data quality when using automated evaluation.

Transfer Learning and Masked Generation for Answer Verbalization

I will talk about our research on extracting issues and evaluations for
the administrative aspects of local governments from the analysis of
citizens' voices on social media. Specifically, I will explain our
method of collecting and analyzing citizens' voices from Twitter on a
city-by-city basis and analyze how the actions of local governments in
Japan changed citizens' voices. As an applied case study, I will also
present our work on understanding the mood of citizens' interest for the
region and our analysis of citizens' opinions on childcare and
restaurant take-out services in the COVID-19 disaster.

Downloads

Next from NAACL 2022

An Encoder Attribution Analysis for Dense Passage Retriever in Open Domain Question Answering

Similar lecture

Gender Disparities and Bias in Medicine and Healthcare

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from NAACL 2022

An Encoder Attribution Analysis for Dense Passage Retriever in Open Domain Question Answering

Similar lecture

Gender Disparities and Bias in Medicine and Healthcare

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads