Papers
arxiv:2607.17423

TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs

Published on Jul 19
Ā· Submitted by
Xiangyu Zeng
on Jul 21
#1 Paper of the day
Authors:
,
,
,
,
,
,
,
,
,

Abstract

Video multimodal large language models (MLLMs) can describe what happens in a video, but rarely identify when the supporting evidence occurs. We study generalist video temporal grounding, in which one model predicts a variable-cardinality set of evidence intervals across video lengths, domains, query forms, and viewpoints. Existing training strategies are misaligned with this set-valued task: long-video labels often rely on brittle one-pass annotation, while reinforcement-learning rewards either fail to distinguish non-overlapping predictions or require fragile segment matching. TimeLens2 treats temporal evidence as an interval set throughout supervision and optimization. TimeLens2-93K constructs reliable multi-span supervision through caption-derived proposals, independent localization, cross-agent consensus, semantic verification, and boundary refinement. Our temporal Wasserstein reward computes exact one-dimensional \(W_1\) between uniform distributions over merged interval supports, providing dense, matching-free feedback under unequal cardinalities and equivalent fragmentation; temporal IoU complements it with precise-overlap feedback. Across seven benchmarks, TimeLens2-2B outperforms all size-matched baselines on every benchmark, while the 4B and 8B variants achieve state-of-the-art performance, surpassing open-source models with up to 397B parameters. The 2B, 4B, and 8B variants improve over their Qwen3-VL backbones by 14.2, 13.0, and 18.1 mIoU points, respectively.

Community

Paper author Paper submitter

Hi everyone! We are excited to share TimeLens2, a generalist video temporal grounding MLLM that identifies not only what happens in a video, but also when the supporting evidence occurs.

TimeLens2 treats temporal evidence as a variable-cardinality set of intervals across short and long videos, single- and multi-span events, declarative and question-form queries, and third- and first-person viewpoints.

Our two key contributions are:

  • TimeLens2-93K: 93,232 verified grounding instances from 23,793 diverse videos, constructed through caption-derived proposals, dual-agent localization, temporal consensus, semantic verification, and boundary refinement.
  • Temporal Wasserstein reward: dense, matching-free feedback over merged interval supports, complementing temporal IoU when predictions do not overlap or contain different numbers of intervals.

Across seven benchmarks, TimeLens2-2B outperforms all size-matched baselines, while the 4B and 8B variants achieve state-of-the-art performance. Notably, TimeLens2-4B surpasses Qwen3.5-397B-A17B on every benchmark by 7.5 average mIoU points.

šŸ¤— Models and resources

We would love to hear your thoughts on generalist temporal grounding, interval-set-aware optimization, and applications that require traceable video evidence!

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2607.17423
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 3

Datasets citing this paper 1

Spaces citing this paper 2

Collections including this paper 2