Lecture image placeholder

Premium content

Access to this content requires a subscription. You must be a premium user to view this content.

Monthly subscription - $9.99Pay per view - $4.99Access through your institutionLogin with Underline account
Need help?
Contact us
Lecture placeholder background

workshop paper

ACL 2024

August 15, 2024

Bangkok, Thailand

ConvKGYarn: Spinning Configurable and Scalable Conversational Knowledge Graph QA datasets with Large Language Models

keywords:

conversational qa

synthetic data

large language models

knowledge graphs

question answering

The evolving landscape of Large Language Models (LLMs) and conversational assistants has ushered in a need for dynamic, up-to-date, scalable, and configurable conversational datasets to train and evaluate systems. Ideally, these datasets are tailored for different user interaction settings, such as text and voice, all of which introduce distinct nuances and modeling challenges. Knowledge Graphs (KGs), with their structured and continuously evolving nature, serve as an ideal reservoir for harnessing current and precise knowledge. While there exist human-curated conversational datasets grounded on KGs, it is hard to rely solely on them, as the information needs of users are in constant flux. Addressing this lacuna, we introduce ConvKGYarn, a scalable and effective method to generate up-to-date, configurable synthetic conversational KGQA datasets. Qualitative psychometric analyses elucidate the effectiveness of ConvKGYarn in generating high-quality conversational data that rivals a popular conversational KGQA dataset on various metrics while making strides in additional desirable properties like adhering to human interaction configurations and functioning at a much larger scale. We further demonstrate the utility of ConvKGYarn by testing LLMs on varied conversations to explore model behavior on conversational KGQA sets with different configurations grounded on the same fact set from the KG. Through our work, we aim to fortify the underpinnings of KGQA and evaluate the parametric knowledge of LLMs.

Next from ACL 2024

Knowlab's Submission to L+M Shared Task: All you need is continued pretraining of \chemistry texts even for molecule captioning
workshop paper

Knowlab's Submission to L+M Shared Task: All you need is continued pretraining of \chemistry texts even for molecule captioning

ACL 2024

Yunsoo Kim

15 August 2024

Stay up to date with the latest Underline news!

Select topic of interest (you can select more than one)

PRESENTATIONS

  • All Lectures
  • For Librarians
  • Resource Center
  • Free Trial
Underline Science, Inc.
1216 Broadway, 2nd Floor, New York, NY 10001, USA

© 2023 Underline - All rights reserved