Release STRIDE-4B

fa6f3cf 14 days ago

5.83 kB

library_name: transformers
license: apache-2.0
base_model: Qwen/Qwen3-VL-4B-Instruct
tags:
  - video-understanding
  - streaming
  - proactive
  - activation-model
  - masked-diffusion
  - multimodal
  - plug-and-play
language:
  - en
pipeline_tag: video-classification
model-index:
  - name: STRIDE-4B
    results:
      - task:
          type: video-classification
          name: Proactive Streaming Activation
        dataset:
          type: custom
          name: OVO-Bench
        metrics:
          - type: accuracy
            value: 60.27
            name: Overall (w/ Qwen3-VL-8B)
      - task:
          type: video-classification
          name: Proactive Streaming Activation
        dataset:
          type: custom
          name: StreamingBench
        metrics:
          - type: accuracy
            value: 59.98
            name: Overall (w/ Qwen3-VL-8B)

STRIDE-4B

STRIDE (Structured Temporal Refinement with Iterative DEnoising) is a lightweight proactive activation model for streaming video understanding. It decides when a downstream Video-LLM should respond during a live video stream — without waiting for explicit user queries.

Paper: STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understanding

Junho Kim*, Hosu Lee*, James M. Rehg, Minsu Kim, Yong Man Ro

UIUC, KAIST, Google DeepMind

What is STRIDE?

Existing streaming Video-LLMs are reactive — they only respond when a user explicitly asks a question. STRIDE makes them proactive by adding a lightweight front-end that continuously monitors incoming frames and predicts coherent activation spans indicating when to trigger a response.

The key insight is that activation in streaming video is not a point-wise binary decision ("should I respond now?"), but a span-structured sequence modeling problem — the model must capture consistent onset (0 → 1), persistence (1 → 1), and offset (1 → 0) transitions. STRIDE achieves this through masked diffusion over a temporal activation window, jointly predicting and iteratively refining activation signals across the window.

Two-Stage Architecture

Video Stream
  │
  ▼
[STRIDE Activation Model]  ← this model (4B)
  │
  │ trigger (only if active)
  ▼
[Downstream Video-LLM]     ← frozen, any off-the-shelf
  │
  ▼
Response

Stage 1 — Activation (STRIDE): Monitors the stream at 1 FPS, maintains a sliding activation window, and iteratively denoises binary activation labels via masked diffusion.
Stage 2 — Response (Downstream LLM): When triggered, the frozen downstream Video-LLM receives the accumulated frame cache and generates a response. STRIDE is fully plug-and-play — compatible with any off-the-shelf Video-LLM.

Results

OVO-Bench (Online Video Understanding)

Method	Real-Time Perception	Backward Tracing	Forward Active Responding	Overall
Flash-VStream-7B	28.37	27.38	45.09	33.61
Dispider	54.55	36.06	34.72	41.78
TimeChat-Online-7B	58.60	42.00	36.40	45.60
QueryStream-7B	61.40	42.10	39.03	47.51
StreamAgent-7B	61.30	41.70	45.40	49.40
STRIDE-4B + Gemma3-4B	60.58	34.60	57.57	50.92
STRIDE-4B + InternVL3-8B	66.63	47.77	57.33	57.24
STRIDE-4B + Qwen3-VL-8B	69.77	48.67	62.37	60.27

StreamingBench (Streaming Comprehension)

Method	Real-Time Visual	Omni-Source	Contextual	Overall
Flash-VStream-7B	23.23	26.00	24.12	24.04
VideoLLM-Online-8B	35.99	28.45	26.55	32.48
Dispider	67.63	35.66	33.61	53.12
StreamAgent-7B	74.31	36.26	34.62	57.02
STRIDE-4B + Gemma3-4B	59.93	36.40	41.00	50.49
STRIDE-4B + InternVL3-8B	73.82	40.90	40.90	59.19
STRIDE-4B + Qwen3-VL-8B	76.01	40.00	39.90	59.98

Usage

For the full streaming inference pipeline and evaluation scripts, please refer to the STRIDE GitHub repository.

Training

Architecture: Qwen3VLForSTRIDE (Qwen3-VL backbone with a temporal activation head)
Base model: Qwen/Qwen3-VL-4B-Instruct
Training data: Temporal activation annotations curated from eight publicly available video understanding datasets (ActivityNet-Captions, LITA, YouCook2, ET-Instruct, Charades, CharadesEgo, DiDeMo, Grounded-VideoLLM). STRIDE-4B is trained under the same data and training configuration as STRIDE-2B.

Model Variants

Model	Params	Description
STRIDE-2B	2B	Default activation model
STRIDE-4B (this)	4B	Scaled variant with improved accuracy

Citation

@article{kim2026stride,
  title={STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understanding},
  author={Kim, Junho and Lee, Hosu and Rehg, James M. and Kim, Minsu and Ro, Yong Man},
  journal={arXiv preprint arXiv:2603.27593},
  year={2026}
}

License

This model is released under the Apache 2.0 License.