Dominican Republic

Infusing factual knowledge into pretrained models is fundamental for many knowledge-intensive tasks. In this paper, we propose Mixture-of-Partitions (MoP), an infusion approach that can handle a very large knowledge graph (KG) by partitioning it into smaller sub-graphs and infusing their specific knowledge into various BERT models using lightweight adapters. To leverage the overall factual knowledge for a target task, these sub-graph adapters are further fine-tuned along with the underlying BERT through a mixture layer. We evaluate our MoP with three biomedical BERTs (SciBERT, BioBERT, PubmedBERT) on six downstream tasks (inc. NLI, QA, Classification), and the results show that our MoP consistently enhances the underlying BERTs in task performance, and achieves new SOTA performances on five evaluated datasets.

EMNLP 2021

Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible. The experience for on-site participants would closely approximate a normal pre-COVID *ACL conference, with 5-6 thematically organized parallel sessions and live Q/A and interactive discussion immediately after the talks. Presentations by virtual participants will be equitably interleaved with those of on-site participants, projected on the auditorium screens as if on-site, and also followed immediately by live Q/A and interactive discussion at a time during reasonable waking hours for the virtual presenter. For all participants, on-site and virtual, who are unable to attend a session due to either time-zone issues or because they are participating in another session live, talk recordings and slides will be available online at a minimum after the live presentation (and in many cases before as well), and questions may be submitted in advance on session-specific discussion boards and answered live in session with the usual visual aids if desired.

<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/ceby/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>

Please Note: The EMNLP registration system is not currently connected to the underline site as we are still in the process of building out EMNLP 2021. You will receive access instructions from underline the week of November 1st. 

Access is given only to EMNLP upon registration, if you have not registered please do so [here](https://2021.emnlp.org/registration).

Registered attendees will receive access the week of November 1st.

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible.

poster

Cross-Policy Compliance Detection via Question Answering

Meta-LMTC: Meta-Learning for Large-Scale Multi-Label Text Classification

Unsupervised Multi-View Post-OCR Error Correction With Language Models

Parallel Refinements for Lexically Constrained Text Generation with BART

Nearest Neighbour Few-Shot Learning for Cross-lingual Classification

Role of Language Relatedness in Multilingual Fine­-tuning of Language Models: A Case Study in Indo-­Aryan Languages

Comparing Feature-Engineering and Feature-Learning Approaches for Multilingual Translationese Classification

Multi-Sentence Resampling: A Simple Approach to Alleviate Dataset Length Bias and Beam-Search Degradation

Controlling Machine Translation for Multiple Attributes with Additive Interventions

A Role-Selected Sharing Network for Joint Machine-Human Chatting Handoff and Service Satisfaction Analysis

https://d3smihljt9218e.cloudfront.net/transcript/24366/subtitle/6dbf4f1bc6d8bbcfea95ca720f694101.vtt

English (automatic)

Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT

Downloads

Next from EMNLP 2021

Cross-Policy Compliance Detection via Question Answering

Similar lecture

PAUSE: Positive and Annealed Unlabeled Sentence Embedding

VIDEO LIBRARY

EVENTS

COMPANY

RESOURCES