Presidential Address

Test of Time Awards

A key contribution to being a successful researcher in natural language processing, as in any area, is having a clear overarching vision of what your body of research is trying to accomplish. Using my own 40-year career as an example, I will attempt to provide general advice on formulating and pursuing a coherent research vision. In particular, I will focus on formulating a unique, personal objective that exploits your specific talents, knowledge, and passions, and that is distinct from the current popular trends in the field. I will also focus on formulating a vision that bridges existing fields of study to produce an overarching agenda that unifies previously disparate ideas.

The Vision Thing: Finding and Pursuing your Research Passion

Large language models (LLMs) have shown increasing capacity at planning and executing a high-level goal in a live computer environment (e.g. MiniWoB++). To perform a task, recent works often require a model to learn from trace examples of the task via either supervised learning or few/many-shot prompting. Without these trace examples, it remains a challenge how an agent can autonomously learn and improve its control on a computer, which limits the ability of an agent to perform a new task. We approach this problem with a zero-shot agent that requires no given expert traces. Our agent plans for executable actions on a partially observed environment, and iteratively progresses a task by identifying and learning from its mistakes via self-reflection and structured thought management. On the easy tasks of MiniWoB++, we show that our zero-shot agent often outperforms recent SoTAs, with more efficient reasoning. For tasks with more complexity, our reflective agent performs on par with prior best models, even though previous works had the advantages of accessing expert traces or additional screen information.

A Zero-Shot Language Agent for Computer Control with Structured Reflection

One of the frequent points in the mainstream narrative about large language models is that they have “emergent properties” (sometimes even dangerous enough to be considered existential risk to mankind). However, there is much disagreement about even the very definition of such properties. If they are understood as a kind of generalization beyond training data - as something that a model does without being explicitly trained for it - I argue that we have not in fact established the existence of any such properties, and at the moment we do not even have the methodology for doing so.

A sanity check on emergent properties

Inverse Text Normalization (ITN) involves rewriting the verbalised form of text from spoken transcripts to its corresponding written form. The task inherently expects challenges in identifying ITN entries due to spelling variations in words arising out of dialects, transcription errors etc. Additionally, in Tamil, word boundaries between adjacent words in a sentence often get obscured due to Punarchi, i.e. phonetic transformation of these boundaries. Being morphologically rich, the words in Tamil show a high degree of agglutination due to inflection and clitics. The combination of such factors leads to a high degree of surfaceform variations, making scalability with pure rule-based approaches difficult. Instead, we experiment with fine-tuning three pre-trained neural LMs, consisting of a seq2seq model (s2s), a non-autoregressive text editor (NAR) and a sequence tagger + rules combination (tagger). While the tagger approach works best in a fully-supervised setting, s2s performs the best (98.05 F-Score) when augmented with additional data, via bootstrapping and data augmentation (DA&B). S2S reports a cumulative percentage improvement of 20.1 %, and statistically significant gains for all our models with DA&B. Compared to a fully supervised setup, bootstrapping alone reports a percentage improvement as high as 14.12 %, even with a small seed set of 324 ITN entries.

Scaling Neural ITN for Numbers and Temporal Expressions in Tamil: Findings for an Agglutinative Low-resource Language | VIDEO

The presentation titled "CLMSM: A Multi-Task Learning Framework for Pre-training on Procedural Text" introduces a novel framework designed to enhance the processing and understanding of procedural text. CLMSM stands out by its unique approach of utilizing metadata from procedures, integrating a Contrastive Learning strategy for global context comprehension, and employing Masked Step Modeling to capture local context nuances. This framework addresses the inherent challenge in procedural text processing – the need for a temporal understanding and tracking of entities across steps, a critical aspect that conventional pre-training methods like Masked Language Modeling (MLM) often overlook.

In this presentation, we demonstrate how CLMSM significantly outperforms various baseline models in entity tracking tasks, underscoring the importance of in-domain pre-training for accurately learning temporal and entity-based information in procedures. The results are particularly notable in scenarios requiring the tracking of entities with implicit locations, highlighting CLMSM’s advanced knowledge retention capabilities. Additionally, we present insights into CLMSM's adaptability to out-of-domain tasks, further establishing its versatility. This framework not only sets a new standard in procedural text understanding but also opens avenues for future research, particularly in extending its applicability to other domains with procedural text, potentially unveiling new challenges and opportunities.

CLMSM: A Multi-Task Learning Framework for Pre-training on Procedural Text

Transliteration is very important in the Indian
language context due to the usage of multi-
ple scripts and the widespread use of roman-
ized inputs. However, few training and evalu-
ation sets are publicly available. We introduce
Aksharantar1, the largest publicly available
transliteration dataset for Indian languages cre-
ated by mining from monolingual and paral-
lel corpora, as well as collecting data from hu-
man annotators. The dataset contains 26 mil-
lion transliteration pairs for 21 Indic lan-
guages from 3 language families using 12
scripts. Aksharantar is 21 times larger than ex-
isting datasets and is the first publicly available
dataset for 7 languages and 1 language fam-
ily. We also introduce the Aksharantar testset
comprising 103k word pairs spanning 19 lan-
guages that enables a fine-grained analysis of
transliteration models on native origin words,
foreign words, frequent words, and rare words.
Using the training set, we trained IndicXlit,
a multilingual transliteration model that im-
proves accuracy by 15% on the Dakshina test
set, and establishes strong baselines on the Ak-
sharantar testset introduced in this work. The
models, mining scripts, transliteration guide-
lines, and datasets are available at https://
github.com/AI4Bharat/IndicXlit under
open-source licenses.

Aksharantar: Open Indic-language Transliteration datasets and models for the Next Billion Users

We present 1-PAGER the first system that answers a question and retrieves evidence using a single Transformer-based model and decoding process. 1-PAGER incrementally partitions the retrieval corpus using constrained decoding to select a document and answer string, and we show that this is competitive with comparable retrieve-and-read alternatives according to both retrieval and answer accuracy metrics. 1-PAGER also outperforms the equivalent ‘closed-book’ question answering model, by grounding predictions in an evidence corpus. While 1-PAGER is not yet on-par with more expensive systems that read many more documents before generating an answer, we argue that it provides an important step toward attributed generation by folding retrieval into the sequence-to-sequence paradigm that is currently dominant in NLP. We also show that the search paths used to partition the corpus are easy to read and understand, paving a way forward for interpretable neural retrieval.

1-Pager: One Pass Answer Generation and Evidence Retrieval

Identifying intents from dialogue utterances forms an integral component of task-oriented dialogue systems. Intent-related tasks are typically formulated either as a classification task, where the utterances are classified into predefined categories or as a clustering task when new and previously unknown intent categories need to be discovered from these utterances. Further, the intent classification may be modeled in a multiclass (MC) or multilabel (ML) setup. While typically these tasks are modeled as separate tasks, we propose INTENDD a unified approach leveraging a shared utterance encoding backbone. INTENDD uses an entirely unsupervised contrastive learning strategy for representation learning, where pseudo labels for the unlabeled utterances are generated based on their lexical features. Additionally, we introduce a two-step post-processing setup for the classification tasks using modified adsorption. Here, first, the residuals in the training data are propagated followed by smoothing the labels both modeled in a transductive setting. Through extensive evaluations on various benchmark datasets, we find that our approach consistently outperforms competitive baselines across all three tasks. On average, INTENDD reports percentage improvements of 2.32 %, 1.26 %, and 1.52 % in their respective metrics for few-shot MC, few-shot ML, and the intent discovery tasks respectively.

IntenDD: A Unified Contrastive Learning Approach for Intent Detection and Discovery

Oral Presentations of Contextual Bangla Neural Stemmer: Finding Contextualized Root Word Representations for Bangla Words

Oral Presentation of CNS Model-Paper ID: 68 | VIDEO

Open-domain dialogue system usually requires different sources of knowledge to generate more informative and evidential responses. However, existing knowledge-grounded dialogue systems either focus on a single knowledge source or overlook the dependency between multiple sources of knowledge, which may result in generating inconsistent or even paradoxical responses. To incorporate multiple knowledge sources and dependencies between them, we propose SAFARI, a novel framework that leverages the exceptional capabilities of large language models (LLMs) in planning, understanding, and incorporating under both supervised and unsupervised settings. Specifically, SAFARI decouples the knowledge grounding into multiple sources and response generation, which allows easy extension to various knowledge sources including the possibility of not using any sources. To study the problem, we construct a personalized knowledge-grounded dialogue dataset Knowledge Behind Persona (KBP), which is the first to consider the dependency between persona and implicit knowledge. Experimental results on the KBP dataset demonstrate that the SAFARI framework can effectively produce persona-consistent and knowledge-enhanced responses.

Underline digital video library

Presidential Address

Test of Time Awards

The Vision Thing: Finding and Pursuing your Research Passion

A Zero-Shot Language Agent for Computer Control with Structured Reflection

A sanity check on emergent properties

Scaling Neural ITN for Numbers and Temporal Expressions in Tamil: Findings for an Agglutinative Low-resource Language | VIDEO

CLMSM: A Multi-Task Learning Framework for Pre-training on Procedural Text

Aksharantar: Open Indic-language Transliteration datasets and models for the Next Billion Users

1-Pager: One Pass Answer Generation and Evidence Retrieval

IntenDD: A Unified Contrastive Learning Approach for Intent Detection and Discovery

Oral Presentation of CNS Model-Paper ID: 68 | VIDEO

Large Language Models as Source Planner for Personalized Knowledge-grounded Dialogues

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES