technical paper

AAAI 2024

February 25, 2024

Vancouver , Canada

BDIQA: A New Dataset for Video Question Answering to Explore Cognitive Reasoning through Theory of Mind


cognitive modeling & cognitive systems (cms) - cms: conceptual inference and reasoning cognitive modeling & cognitive systems (cms) - cms: simulating human behavior computer vision(cv) - cv: video understanding & activity analysis

As a foundational component of cognitive intelligence, theory of mind (ToM) can make AI more closely resemble human thought processes, thereby enhancing their interaction and collaboration with human. In particular, it can significantly improve a model's comprehension of videos in complex scenes. However, current video question answer (VideoQA) datasets focus on studying causal reasoning within events, few of them genuinely incorporating human ToM. Consequently, there is a lack of development in ToM reasoning tasks within the area of VideoQA. This paper presents BDIQA, the first benchmark to explore the cognitive reasoning capabilities of VideoQA models in the context of ToM. BDIQA is inspired by the cognitive development of children's ToM and addresses the current deficiencies in machine ToM within datasets and tasks. Specifically, it offers tasks at two difficulty levels, assessing belief, desire and intention (BDI) reasoning in both simple and complex scenarios. We conduct evaluations on several mainstream methods of VideoQA and diagnose their capabilities with zero-shot, few-shot and supervised learning. We find that the performance of pre-trained models on cognitive reasoning tasks remains unsatisfactory. To counter this challenge, we undertake thorough analysis and experimentation, ultimately presenting two guidelines to enhance cognitive reasoning derived from ablation analysis.


SlidesPaperTranscript English (automatic)

Next from AAAI 2024

Measuring Task Similarity and Its Implication in Fine-Tuning Graph Neural Networks | VIDEO
technical paper

Measuring Task Similarity and Its Implication in Fine-Tuning Graph Neural Networks | VIDEO

AAAI 2024

+4Chenglu PanRenhong Huang
Renhong Huang and 6 other authors

25 February 2024

Stay up to date with the latest Underline news!

Select topic of interest (you can select more than one)


  • All Lectures
  • For Librarians
  • Resource Center
  • Free Trial
Underline Science, Inc.
1216 Broadway, 2nd Floor, New York, NY 10001, USA

© 2023 Underline - All rights reserved