
Premium content
Access to this content requires a subscription. You must be a premium user to view this content.

workshop paper
Assessing LLMs' Ability to Navigate Cultural Knowledge Conflicts
keywords:
cultural knowledge conflict
cultural alignment
llm
Large language models (LLMs) often face knowledge conflicts when addressing questions that can yield different answers based on the cultural or regional context. In this work, we create the QARV benchmark to evaluate LLM's proficiency in managing these conflicts between two distinct countries, the US and Korea. The benchmark consists of 671 handwritten questions, each accompanied by two answers from each nation. Our evaluation of 42 different LLMs, each tested with nine unique prompts, reveals that most models inherently respond with an American bias. Additionally, we observe that prompting, and instruction-tuning, are all practical measures to enhance cultural alignment.