Korea, Republic of

Pre-Training (PT) of text representations has been successfully applied to low-resource Neural Machine Translation (NMT). However, it usually fails to achieve notable gains (sometimes, even worse) on resource-rich NMT on par with its Random-Initialization (RI) counterpart. We take the first step to investigate the complementarity between PT and RI in resource-rich scenarios via two probing analyses, and find that: PT improves NOT the accuracy, but the generalization by achieving flatter loss landscapes than that of RI; PT improves NOT the confidence of lexical choice, but the negative diversity by assigning smoother lexical probability distributions than that of RI. Based on these insights, we propose to combine their complementarities with a model fusion algorithm that utilizes optimal transport to align neurons between PT and RI. Experiments on two resource-rich translation benchmarks, WMT’17 English-Chinese (20M) and WMT’19 English-German (36M), show that PT and RI could be nicely complementary to each other, achieving substantial improvements considering both translation accuracy, generalization, and negative diversity.

COLING 2022

On the Complementarity between Pre-Training and Random-Initialization for Resource-Rich Machine Translation

model fusion

random-initialization

resource-rich translation

pre-training

machine translation

## Welcome to NAACL 2022!
The Annual Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) is taking place July 10-15, 2022 as a hybrid event, in Seattle, WA and online. We are happy to welcome all of you to conference! 
 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, keynote presentations, a plenary panel on the place of linguistics and symbolic structures in NLP, and an Industrial Track panel on Careers in NLP.
 
Posters (including Findings of NAACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop and will have an oral session and a poster session.
 
The program also features 6 Tutorials and 26 Workshops.
 
 We wish you a wonderful conference!
 
[The NAACL 2022 Organizing Committee](https://2022.naacl.org/committees/organization/)

NAACL 2022

2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics

poster

[![](https://assets.underline.io/uploads/markdown_image/1/image/81a3c0317d24f663b49b996875024d45.png)](https://aclanthology.org/volumes/2022.coling-1/)



### THE CONFERENCE WE KNOW AND WE WANT.
**COLING**, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes but also in industrial research departments including tech-startups. COLING provides opportunities for all these communities to showcase their exciting discovery.

In fall of 2022, COLING will be held in Gyeongju in a hybrid format. All participants can either present at the venue site or join virtually. As more people get vaccinated, we are happy to provide safer environments for our colleagues. We believe that COLING 2022 will be one of the conferences, free from the pandemic. The hybrid format gives presenters and sponsors a valuable opportunity to promote their companies in both an online and in-person venue. For the first time in a long time, customers can interact with their sponsor's products first-hand. The online venue, too, gives sponsors the chance to network with those unable to attend the in-person session.

The hybrid format ultimately lends itself to greater exposure for our sponsors: COLING2022 will let you reach more potential partners and customers than ever before!

To gain access to this event page you are required to register. Find more information and pay the registration fee on event the organizer’s website **[https://coling2022.org/reg](https://coling2022.org/reg)**

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

Even though many recent semantic parsers are based on deep learning methods, we should not forget that rule-based alternatives might offer advantages over neural approaches with respect to transparency, portability, and explainability. Taking advantage of existing off-the-shelf Universal Dependency parsers, we present a method that maps a syntactic dependency tree to a formal meaning representation based on Discourse Representation Theory. Rather than using lambda calculus to manage variable bindings, our approach is novel in that it consists of using a series of graph transformations. The resulting UD semantic parser shows good performance for English, German, Italian and Dutch, with F-scores over 75%, outperforming a neural semantic parser for the lower-resourced languages. Unlike neural semantic parsers, our UD semantic parser requires little training data, is easy to port to other languages, and is completely transparent.

Transparent Semantic Parsing with Universal Dependencies using Graph Transformations

Distant supervision (DS) is a strong way to expand the datasets for enhancing relation extraction (RE) models but often suffers from high label noise. Current works based on attention, reinforcement learning, or GAN are black-box models so they neither provide meaningful interpretation of sample selection in DS nor stability on different domains. On the contrary, this work proposes a novel model-agnostic instance sampling method for DS by influence function (IF), namely REIF. Our method identifies favorable/unfavorable instances in the bag based on IF, then does dynamic instance sampling. We design a fast influence sampling algorithm that reduces the computational complexity from $\mathcal{O}(mn)$ to $\mathcal{O}(1)$, with analyzing its robustness on the selected sampling function. Experiments show that by simply sampling the favorable instances during training, REIF is able to win over a series of baselines which have complicated architectures. We also demonstrate that REIF can support interpretable instance selection.


Finding Influential Instances for Distantly Supervised Relation Extraction

Few-shot Named Entity Recognition with Entity-level Prototypical Network Enhanced by Dispersedly Distributed Prototypes

We propose, DocQueryNet, a value retrieval method with arbitrary queries for form-like documents to reduce human effort of processing forms. Unlike previous methods that only address a fixed set of field items, our method predicts target value for an arbitrary query based on the understanding of the layout and semantics of a form. To further boost model performance, we propose a simple document language modeling (SimpleDLM) strategy to improve document understanding on large-scale model pre-training. Experimental results show that DocQueryNet outperforms previous designs significantly and the SimpleDLM further improves our performance on value retrieval by around 17% F1 score compared with the state-of-the-art pre-training method. https://github.com/salesforce/QVR-SimpleDLM(Code is available here).

DocQueryNet: Value Retrieval with Arbitrary Queries for Form-like Documents

Existing fake news detection methods aim to classify a piece of news as true or false and provide veracity explanations, achieving remarkable performances. However, they often tailor automated solutions on manual fact-checked reports, suffering from limited news coverage and debunking delays. When a piece of news has not yet been fact-checked or debunked, certain amounts of relevant raw reports are usually disseminated on various media outlets, containing the wisdom of crowds to verify the news claim and explain its verdict. In this paper, we propose a novel Coarse-to-fine Cascaded Evidence-Distillation (CofCED) neural network for explainable fake news detection based on such raw reports, alleviating the dependency on fact-checked ones. Specifically, we first utilize a hierarchical encoder for web text representation, and then develop two cascaded selectors to select the most explainable sentences for verdicts on top of the selected top-K reports in a coarse-to-fine manner. Besides, we construct two explainable fake news datasets, which are publicly available. Experimental results demonstrate that our model significantly outperforms state-of-the-art detection baselines and generates high-quality explanations from diverse evaluation perspectives.

A Coarse-to-fine Cascaded Evidence-Distillation Neural Network for Explainable Fake News Detection

We describe a novel approach to transcribing morphologically complex, local, oral languages. The approach connects with local motivations for participating in language work which center on language learning, accessing the content of audio collections, and applying this knowledge in language revitalization and maintenance. We develop a constraint-based approach to interactive word completion, expressed using Optimality Theoretic constraints, implemented in a finite state transducer, and applied to an Indigenous language. We show that this approach suggests correct full word predictions on 57.9% of the test utterances, and correct partial word predictions on 67.5% of the test utterances. In total, 87% of the test utterances receive full or partial word suggestions which serve to guide the interactive transcription process.

A Finite State Aproach to Interactive Transcription

Explainability is a crucial component of natural language processing (NLP) systems. Explanation is communication, and research on explainability is expected to address notions such as communicative goals and common grounding. Automated Writing Evaluation (AWE) is an ideal field to contribute to such research. AWE refers to NLP tasks designed to support human learners by evaluating the quality of texts produced in educational contexts, such as written answers to questions, argumentative essays, etc., and providing constructive feedback. Since explanations are vital in educational assessment, pedagogical literature is rich with insights that can facilitate research on explainability in AWE systems. This talk will overview recent research trends, explore what style of explanations are pedagogically suitable and technologically feasible at each layer of language production, ranging from content planning to surface realization, and identify open research issues to encourage researchers to enter this emerging field.

Explainability in Automated Writing Evaluation

With the global population ageing rapidly, a key health concern lies in Neurocognitive Disorders (NCD) – a common form being Alzheimer’s Disease (AD). NCDs are particularly prominent in older adults, which has an insidious onset followed by gradual, irreversible deterioration in memory, communication, thinking and other cognitive functions. Thus the detection of Alzheimer’s disease is crucial for timely intervention to slow down disease progression. This talk presents the background that motivated our interdisciplinary team (with researchers in engineering, geriatrics, linguistics, neurology, psychology and public health) to embark on this meaningful research topic. We will also present our recent research and development of spoken language technologies to analyze English and Chinese recordings from neuropsychological assessments for discriminating between healthy controls and participants with NCD. Encouraging experimental results support the viability of fully automatic screening. We will also describe ongoing research on naturalistic data collection and spoken language interface development for older adult users.

AI for dementia detection

Deep nets are great, and they would be even greater if they could collaborate with each other through a flexible communication protocol. Since many networks come equipped with language model interfaces--that is, they can process and produce tokens in English (or other languages)--it makes sense to look into natural language as a universal neural network communication protocol. The questions I would like to discuss in my talk, then, are the following: 
* What are the challenges we face in developing a universal language interface among deep networks?
* What do we currently know about the way in which deep nets are using language to interact in machine-to-machine communication scenarios?
* Which characteristics of human linguistic communication do we expect or wish machine-to-machine communication to possess? Should it be essentially the same as human-to-human (and human-to-machine) communication?

Premium content

On the Complementarity between Pre-Training and Random-Initialization for Resource-Rich Machine Translation

Downloads

Next from COLING 2022

Transparent Semantic Parsing with Universal Dependencies using Graph Transformations

Similar lecture

Combining Translation and Domain Pre-Training for Cross-Lingual News Article Similarity

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES