What determines children's production of syntactic alternations? This study takes up this question with the double PP construction in English as the test case (e.g., write [on the paper] [with this new pen]). Leveraging data of spontaneous child-parent interactions, we investigate the roles of dependency length and parent input frequency, with the latter being operationalized as lexical frequency and contextual predictability. We found that when child and parent data was combined, all three factors turned out to have significant predictive power, with dependency length having the most pronounced role. Results from the developmental trajectory of DLM as well as logistic regression analysis suggest that child production of constituent ordering preferences starts becoming more parent-like after the age of 30 months.


Double PP Constituent Ordering Preferences in English Early Child Language

Generative syntax and a consistently ordered count routine are both understood to have central roles in learning a number system. However, there has been little experimental exploration of how the diversity of each of these features alters the inductive landscape of number learning, as most empirical work has been constrained to correlational studies. We present a causal manipulation both of syntactic structures and counting procedures, using an artificial language paradigm. Our findings suggest that (1) learners have a greater facility with conjunctive over multiplicative rules of composition, (2) counting procedures help learners to recall words independently of syntax, and (3) predictable syntax helps learners to wield numerical concepts, independently of - and possibly despite - counting routines.


How does the syntax of counting affect learnability? Evidence from artificial language learning.

The Phrase-as-Lemma construction

Few-shot Named Entity Recognition (NER) is imperative for entity tagging in limited resource domains and thus received proper attention in recent years. Existing approaches for few-shot NER are evaluated mainly under in-domain settings. In contrast, little is known about how these inherently faithful models perform in cross-domain NER using a few labeled in-domain examples. This paper proposes a two-step rationale-centric data augmentation method to improve the model's generalization ability. Results on several datasets show that our model-agnostic method significantly improves the performance of cross-domain NER tasks compared to previous state-of-the-art methods, including the counterfactual data augmentation and prompt-tuning methods. Our codes are available at https://github.com/lifan-yuan/FactMix.

FactMix: Using a Few Labeled In-domain Examples to Generalize to Cross-domain Named Entity Recognition

Dependency parsing is an important fundamental natural language processing task which analyzes the syntactic structure of an input sentence by illustrating the syntactic relations between words. To improve dependency parsing, leveraging existing dependency parsers and extra data (e.g., through semi-supervised learning) has been demonstrated to be effective, even though the final parsers are trained on inaccurate (but massive) data. In this paper, we propose a frustratingly easy approach to improve graph-based dependency parsing, where a structure-aware encoder is pre-trained on auto-parsed data by predicting the word dependencies and then fine-tuned on gold dependency trees, which differs from the usual pre-training process that aims to predict the context words along dependency paths. Experimental results and analyses demonstrate the effectiveness and robustness of our approach to benefit from the data (even with noise) processed by different parsers, where our approach outperforms strong baselines under different settings with different dependency standards and model architectures used in pre-training and fine-tuning. More importantly, further analyses find that only 2K auto-parsed sentences are required to obtain improvement when pre-training vanilla BERT-large based parser without requiring extra parameters.

Enhancing Structure-aware Encoder with Extremely Limited Data for Graph-based Dependency Parsing

Discourse parsing on multi-party dialogues is an important but difficult task in dialogue systems and conversational analysis.
It is believed that speaker interactions are helpful for this task.
However, most previous research ignores speaker interactions between different speakers.
To this end, we present a speaker-aware model for this task.
Concretely, we propose a speaker-context interaction joint encoding~(SCIJE) approach,
using the interaction features between different speakers.
In addition, we propose a second-stage pre-training task, same speaker prediction~(SSP),
enhancing the conversational context representations by predicting whether two utterances are from the same speaker.
Experiments on two standard benchmark datasets show that the proposed model achieves the best-reported performance in the literature. 
We will release the codes of this paper to facilitate future research

Speaker-Aware Discourse Parsing on Multi-Party Dialogues

Semantic role labeling (SRL) is a fundamental yet challenging task in the NLP community. Recent works of SRL mainly fall into two lines: 1) BIO-based; 2) span-based. Despite ubiquity, they share some intrinsic drawbacks of not considering internal argument structures, potentially hindering the model's expressiveness. The key challenge is arguments are flat structures, and there are no determined subtree realizations for words inside arguments. To remedy this, in this paper, we propose to regard flat argument spans as latent subtrees, accordingly reducing SRL to a tree parsing task. In particular, we equip our formulation with a novel span-constrained TreeCRF to make tree structures span-aware and further extend it to the second-order case. We conduct extensive experiments on CoNLL05 and CoNLL12 benchmarks. Results reveal that our methods perform favorably better than all previous syntax-agnostic works, achieving new state-of-the-art under both end-to-end and w/ gold predicates settings.

Semantic Role Labeling as Dependency Parsing: Exploring Latent Tree Structures Inside Arguments

Most existing slot filling models tend to memorize inherent patterns of entities and corresponding contexts from training data. However, these models can lead to system failure or undesirable outputs when being exposed to spoken language perturbation or variation in practice. We propose a perturbed semantic structure awareness transferring method for training perturbation-robust slot filling models. Specifically, we introduce two MLM-based training strategies to respectively learn contextual semantic structure and word distribution from unsupervised language perturbation corpus. Then, we transfer semantic knowledge learned from upstream training procedure into the original samples and filter generated data by consistency processing. These procedures aim to enhance the robustness of slot filling models. Experimental results show that our method consistently outperforms the previous basic methods and gains strong generalization while preventing the model from memorizing inherent patterns of entities and contexts.

PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling

Annotation conversion is an effective way to construct datasets under new annotation guidelines based on existing datasets with little human labour. Previous work has been limited in conversion between tree-structured datasets and mainly focused on feature-based models which are not easily applicable to new conversions. In this paper, we propose two simple and effective graph-to-graph annotation conversion approaches, namely Label Switching and Graph2Graph Linear Transformation, which use pseudo data and inherit parameters to guide graph conversions respectively. These methods are able to deal with conversion between graph-structured annotations and require no manually designed features. To verify their effectiveness, we manually construct a graph-structured parallel annotated dataset and evaluate the proposed approaches on it as well as other existing parallel annotated datasets. Experimental results show that the proposed approaches outperform strong baselines with higher conversion scores. To further validate the quality of converted graphs, we utilize them to train the target parser and find graphs generated by our approaches lead to higher parsing scores than those generated by the baselines.

Simple and Effective Graph-to-Graph Annotation Conversion

Abstract Meaning Representation (AMR) offers a unified semantic representation for natural language sentences. Thus transformation between AMR and text yields two transition tasks in opposite directions, i.e., Text-to-AMR parsing and AMR-to-Text generation. Existing AMR studies only focus on one-side improvements despite the duality of the two tasks, and their improvements are greatly attributed to the inclusion of large extra training data or complex structure modifications which harm the inference speed. Instead, we propose data-efficient Bidirectional Bayesian learning (BiBL) to facilitate bidirectional information transition by adopting a single-stage multitasking strategy so that the resulting model may enjoy much lighter training at the same time. Evaluation on benchmark datasets shows that our proposed BiBL outperforms strong previous seq2seq refinements without the help of extra data which is indispensable in existing counterpart models.

BiBL: AMR Parsing and Generation with Bidirectional Bayesian Learning

This paper considers the task of parsing low-resource languages in a scenario where parallel English data and also a limited seed of annotated sentences in the target language are available, as for example in bootstrapping parallel treebanks.
We focus on constituency parsing using Role and Reference Grammar (RRG), a theory that has so far been understudied in computational linguistics but that is widely used in typological research, i.e., in particular in the context of low-resource languages. Starting from an existing RRG parser, we propose two strategies for low-resource parsing: first, we extend the parsing model into a cross-lingual parser, exploiting the parallel data in the high-resource language and unsupervised word alignments by providing internal states of the source-language parser to the target-language parser. Second, we adopt self-training, thereby iteratively expanding the training data, starting from the seed, by including the most
confident new parses in each round.
Both in simulated scenarios and with a real low-resource language (Daakaka), we find substantial and complementary improvements from both self-training and cross-lingual parsing. Moreover, we also experimented with using gloss embeddings in addition to token embeddings in the target language, and this also improves results. Finally, starting from what we have for Daakaka, we also consider parsing a related language (Dalkalaen) where glosses and English translations are available but no annotated trees at all, i.e., a no-resource scenario wrt.~syntactic annotations. We start with cross-lingual parser trained on Daakaka with glosses and use self-training to adapt it to Dalkalaen. The results are surprisingly good.

Improving Low-resource RRG Parsing with Cross-lingual Self-training

In the presentation, we present our work "Multi-Layer Pseudo-Siamese Biaffine Model for Dependency Parsing". In this work, we propose a multi-layer architecture for dependency parsing, where the biaffine method plays important roles in both scorer and attention mechanism at the same time in each layer. The presentation contains the following in order: 1) We introduce the task and our motivation. 2) We show the model. 3) We show the experiments for comparison of three aspects. 4) We show the key points of analysis. 5) We show the conclusion.

Underline digital video library

Double PP Constituent Ordering Preferences in English Early Child Language

How does the syntax of counting affect learnability? Evidence from artificial language learning.

The Phrase-as-Lemma construction

FactMix: Using a Few Labeled In-domain Examples to Generalize to Cross-domain Named Entity Recognition

Enhancing Structure-aware Encoder with Extremely Limited Data for Graph-based Dependency Parsing

Speaker-Aware Discourse Parsing on Multi-Party Dialogues

Semantic Role Labeling as Dependency Parsing: Exploring Latent Tree Structures Inside Arguments

PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling

Simple and Effective Graph-to-Graph Annotation Conversion

BiBL: AMR Parsing and Generation with Bidirectional Bayesian Learning

Improving Low-resource RRG Parsing with Cross-lingual Self-training

Multi-Layer Pseudo-Siamese Biaffine Model for Dependency Parsing

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES