FROMANNUAL REVIEWS

EMNLP 2026

•

October 25, 2026

•

Budapest, Hungary

We introduce the Massive Video Embedding Benchmark (MVEB), a 23-task benchmark for video embeddings spanning classification, zero-shot classification, clustering, pair classification, retrieval, and video-centric question answering. We evaluate 33 models and find that no single model dominates: MLLM-based embeddings lead on classification, clustering, pair classification, and QA; multimodal binding leads on retrieval and zero-shot classification; generative MLLMs without contrastive adaptation collapse on cross-modal tasks. Paired video-only vs. audio+video evaluations show that audio's contribution depends on dataset annotation provenance: audio helps when labels were produced from both modalities and hurts when they were produced from visuals alone, a six-point gap consistent across model families. MVEB is derived from MVEB+, a 184-task pool, and is designed to maintain task diversity while reducing evaluation cost. It integrates into the MTEB ecosystem for unified evaluation across text, image, audio, and video. We release MVEB and all 184 tasks along with code and a leaderboard at https://github.com/embeddings-benchmark/mteb.

Downloads

Paper

Next from EMNLP 2026

On the Value of Open Research in Language Modeling - An Adversarial Dialog
keynote

On the Value of Open Research in Language Modeling - An Adversarial Dialog

EMNLP 2026

Graham Neubig
Graham Neubig

26 October 2026

Similar lecture

Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
poster

Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding

EMNLP 2023

Hang Zhang and 2 other authors

09 December 2023

Stay up to date with the latest Underline news!

Select topic of interest (you can select more than one)

PRESENTATIONS

  • All Presentations
  • For Librarians
  • Resource Center
  • Free Trial
Underline Science, Inc.
1216 Broadway, 2nd Floor, New York, NY 10001, USA

© 2026 Underline - All rights reserved

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.