Spain

We present a multi-level alignment pretraining method in a unified architecture for multi-lingual semantic parsing. In this architecture, we use an adversarial training method to align the space of different languages and use sentence level and word level parallel corpus as supervision information to align the semantic of different languages. Finally, we jointly train the multi-level alignment and semantic parsing tasks. We conduct experiments on a publicly avail-able multi-lingual semantic parsing dataset ATIS and a newly constructed dataset. Experimental results show that our model outperforms state-of-the-art methods on both datasets.


COLING 2020

Multi-level Alignment Pretraining for Multi-lingual Semantic Parsing

multi-lingual

pretraining

semantic parsing

stages 

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes, but also in industrial research departments and in technological startups. COLING conferences provide opportunities for all these communities to showcase their exciting developments.

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

poster

Open-domain reading comprehension is a challenging issue in terms of natural language understanding. In this paper, we proposes a miscellaneous-context-based
method to convert a sentence into a knowledge embedding in the form of a directed graph. Based on the idea of conceptual graphs, this graph representation method
abbreviated as AGG-CG can (1) accommodate the slot-filling challenges in typical question answering, (2) access to the sentence-level graph structure in order to explicitly capture the neighbouring connections of reference concept nodes, and (3) project an edge of a sentence-level graph to the space of semantic relevance with respect to the corresponding concept nodes, by cooperating with the semantics-measured module. In conclusion, our approach is the first towards the interpretable process of learning human-comprehensible graphical representations with the experimental evidence.

Automated Graph Generation at Sentence Level for Reading Comprehension Based on Conceptual Graphs

• Companies are racing to leverage digital technologies (AI, IoT,
Cloud, among others)
•Majority of companies report that their strategies did not deliver
anticipated business results
• We used NLP to understand the different clusters of digital
strategy patterns of Fortune 500 companies
• Results showed four approaches to digital strategy
• The results provided a baseline to improve our understanding of
digital strategy approaches

Deep Learning Framework for Measuring the Digital Strategy of Companies from Earnings Calls

Sparsity is regarded as a desirable property of representations, especially in terms of explanation. However, its usage has been limited due to the gap with dense representations. Most NLP re- search progresses in recent years are based on dense representations. Thus the desirable property of sparsity cannot be leveraged. Inspired by Fourier Transformation, in this paper, we propose a novel Semantic Transformation method to bridge the dense and sparse spaces, which can facilitate the NLP research to shift from dense space to sparse space or to jointly use both spaces. Experiments using classification tasks and natural language inference task show that the proposed Semantic Transformation is effective.


Transformation of Dense and Sparse Text Representations

Multimodal named entity recognition (MNER) for tweets has received increasing attention recently. Most of the multimodal methods used attention mechanisms to capture the text-related visual information. However, unrelated or weakly related text-image pairs account for a large proportion in tweets. Visual clues unrelated to the text would incur uncertain or even negative effects for multimodal model learning. In this paper, we propose a novel pre-trained multimodal model based on Relationship Inferring and Visual Attention (RIVA) for tweets. The RIVA model controls the attention-based visual clues with a gate regarding the role of image to the semantics of text. We use a teacher-student semi-supervised paradigm to leverage a large unlabeled multimodal tweet corpus with a labeled data set for text-image relation classification. In the multimodal NER task, the experimental results show the significance of text-related visual features for the visual-linguistic model and our approach achieves SOTA performance on the MNER datasets.

RIVA: A Pre-trained Tweet Multimodal Model Based on Text-image Relationship for Multimodal NER

Understanding neural models is a major topic of interest in the deep learning community. In this paper, we propose to interpret a general neural model comparatively. Specifically, we study the sequence-to-sequence (Seq2Seq) model in the contexts of two mainstream NLP tasks–machine translation and dialogue response generation–as they both use the Seq2Seq model. We investigate how the two tasks are different and how their task difference results in major differences in the behaviors of the resulting translation and dialogue generation systems. This study allows us to make several interesting observations and gain valuable insights, which can be used to help develop better translation and dialogue generation models. To our knowledge, no such comparative study has been done so far.

Translation vs. Dialogue: A Comparative Analysis of Sequence-to-Sequence Modeling


Annotating data is trivial in real-world situations. To solve this problem, Active Learning has been well-studied to decrease the needed amount of the annotating data and successfully applied to NLU on some pipeline model. However, no research has been done on investigating how the relation information between intents and slots can improve the efficiency of AL algorithms. Therefore, we propose a pool-based multitask active learning framework for NLU and an efficient computation for the entropy of a joint model. Experiment results show that our framework is efficient and has the potential for industrial use.

A Multitask Active Learning Framework for Natural Language Understanding

Visual storytelling aims to generate a narrative paragraph from a sequence of images automatically. Existing approaches construct text description independently for each image and roughly concatenate them as a story, which leads to the problem of generating semantically incoherent content. In this paper, we propose a new way for visual storytelling by introducing a topic description task to detect the global semantic context of an image stream. A story is then constructed with the guidance of the topic description. In order to combine the two generation tasks, we propose a multi-agent communication framework that regards the topic description generator and the story generator as two agents and learn them simultaneously via iterative updating mechanism. We validate our approach on VIST dataset, where quantitative results, ablations, and human evaluation demonstrate our method's good ability in generating stories with higher quality compared to state-of-the-art methods.

Keep it Consistent: Topic-Aware Storytelling from an Image Stream via Iterative Multi-agent Communication

A large body of research on gender-linked language has established foundations regarding cross-gender differences in lexical, emotional, and topical preferences, along with their sociological underpinnings. We compile a novel, large and diverse corpus of spontaneous linguistic productions annotated with speakers’ gender, and perform a first large-scale empirical study of distinctions in the usage of figurative language between male and female authors. Our analyses suggest that (1) idiomatic choices reflect gender-specific lexical and semantic preferences in general language, (2) men’s and women’s idiomatic usages express higher emotion than their literal language, with detectable, albeit more subtle, differences between male and female authors along the dimension of dominance compared to similar distinctions in their literal utterances, and (3) contextual analysis of idiomatic expressions reveals considerable differences, reflecting subtle divergences in usage environments, shaped by cross-gender communication styles and semantic biases.


Pick a Fight or Bite your Tongue: Investigation of Gender Differences in Idiomatic Language Usage

In natural languages, words are used in association to construct sentences. It is not words in isolation, but the appropriate combination of hierarchical structures that conveys the meaning of the whole sentence. Neural networks can capture expressive language features; however, insights into the link between words and sentences are difficult to acquire automatically. In this work, we design a deep neural network architecture that explicitly wires lower and higher linguistic components; we then evaluate its ability to perform the same task at different hierarchical levels. Settling on broad text classification tasks, we show that our model, MHAL, learns to simultaneously solve them at different levels of granularity by fluidly transferring knowledge between hierarchies. Using a multi-head attention mechanism to tie the representations between single words and full sentences, MHAL systematically outperforms equivalent models that are not incentivized towards developing compositional representations. Moreover, we demonstrate that, with the proposed architecture, the sentence information flows naturally to individual words, allowing the model to behave like a sequence labeller (which is a lower, word-level task) even without any word supervision, in a zero-shot fashion.

Seeing Both the Forest and the Trees: Multi-head Attention for Joint Classification on Different Compositional Levels

Sentence intention matching (SIM) is vital for natural language understanding, especially for Chinese sentence intention matching task. We propose a novel sentence intention matching model, named the intra-correlation encoding model (ICE). It combines character-granularity and word-granularity features to perform sentence intention matching and employs soft alignment attention to enhance the local information of sentences on the corresponding levels.

Multi-level Alignment Pretraining for Multi-lingual Semantic Parsing

Downloads

Next from COLING 2020

Automated Graph Generation at Sentence Level for Reading Comprehension Based on Conceptual Graphs

Similar lecture

Normalizing Compositional Structures Across Graphbanks

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Multi-level Alignment Pretraining for Multi-lingual Semantic Parsing

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from COLING 2020

Automated Graph Generation at Sentence Level for Reading Comprehension Based on Conceptual Graphs

Similar lecture

Normalizing Compositional Structures Across Graphbanks

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads