Lecture image placeholder

Premium content

Access to this content requires a subscription. You must be a premium user to view this content.

Monthly subscription - $9.99Pay per view - $4.99Access through your institutionLogin with Underline account
Need help?
Contact us
Lecture placeholder background

workshop paper

ACL 2024

August 15, 2024

Bangkok, Thailand

Morphology Matters: Probing the Cross-linguistic Morphological Generalization Abilities of Large Language Models through a Wug Test

keywords:

cross-linguistic analysis

morphological generalization

wug test

morphological complexity

low-resource languages

morphology

large language models

multilingual language models

We develop a multilingual version of the Wug Test, an artificial word completion experiment that is typically used to test the morphological knowledge of children, and apply it to the GPT family of large language models (LLMs). LLMs' performance on this test was evaluated by native speakers of six different languages, who judged whether the inflected and derived forms generated by the models conform to the morphological rules of their language. Our results show that LLMs can generalize their morphological knowledge to new, unfamiliar words, but that their success in generating the "correct" generalization (as judged by native human speakers) is predicted by a language's morphological complexity (specifically, integrative complexity). We further find that the amount of training data has surprisingly little on LLMs' morphological generalization abilities within the scope of the analyzed languages. These findings highlight that morphology matters'', and have important implications for improving low-resource language modeling.

Next from ACL 2024

Evaluating Grammatical Well-Formedness in Large Language Models: A Comparative Study with Human Judgments
workshop paper

Evaluating Grammatical Well-Formedness in Large Language Models: A Comparative Study with Human Judgments

ACL 2024

Zhuang Qiu and 2 other authors

15 August 2024

Stay up to date with the latest Underline news!

Select topic of interest (you can select more than one)

PRESENTATIONS

  • All Lectures
  • For Librarians
  • Resource Center
  • Free Trial
Underline Science, Inc.
1216 Broadway, 2nd Floor, New York, NY 10001, USA

© 2023 Underline - All rights reserved