Spain

conversational reasoning

multimodal dialog

dialog state tracking

stages 

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes, but also in industrial research departments and in technological startups. COLING conferences provide opportunities for all these communities to showcase their exciting developments.

COLING 2020

Next generation virtual assistants are envisioned to handle multimodal inputs (e.g., vision, memories of previous interactions, and the user's utterances), and perform multimodal actions (e.g., displaying a route while generating the system's utterance).

We introduce Situated Interactive MultiModal Conversations (SIMMC) as a new direction aimed at training agents that take multimodal actions grounded in a co-evolving multimodal input context in addition to the dialog history.

We provide two SIMMC datasets totalling 13K human-human dialogs (169K utterances) collected using a multimodal Wizard-of-Oz (WoZ) setup, on two shopping domains: (a) furniture -- grounded in a shared virtual environment; and (b) fashion -- grounded in an evolving set of images. Datasets include multimodal context of the items appearing in each scene, and contextual NLU, NLG and coreference annotations using a novel and unified framework of SIMMC conversational acts for both user and assistant utterances.

Finally, we present several tasks within SIMMC as objective evaluation protocols, such as structural API prediction, response generation, and dialog state tracking. We benchmark a collection of existing models on these SIMMC tasks as strong baselines, and demonstrate rich multimodal conversational interactions.

Our data, annotations, and models are publicly available (https://github.com/facebookresearch/simmc).

Situated and Interactive Multimodal Conversations

technical paper

Event-Guided Denoising for Multilingual Relation Learning

Retrieving Skills from Job Description: A Language Model Based Extreme Multi-label Classification Framework

Analogy Models for Neural Word Inflection

IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP

RatE: Relation-Adaptive Translating Embedding for Knowledge Graph Completion

A Closer Look at Linguistic Knowledge in Masked Language Models: The Case of Relative Clauses in American English

Words are the Window to the Soul: Language-based User Representations for Fake News Detection

Native-like Expression Identification by Contrasting Native and Proficient Second Language Speakers

Multi-Word Lexical Simplification

Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translation

https://assets.underline.io/transcript/4605/subtitle/ca4c406b1433f99d2baeff0a67316f0e.vtt

English (automatic)

Situated and Interactive Multimodal Conversations

Downloads

Next from COLING 2020

Event-Guided Denoising for Multilingual Relation Learning

Similar lecture

Out-of-Task Training for Dialog State Tracking Models