OpenMOSS-Team

/

MOSS-VL-Realtime

like 108

Follow

OpenMOSS 1.18k

Ascend NPU (910B, CANN 9.0.0) support for the realtime checkpoint.
CUDA behavior unchanged — every NPU path is guarded.

• modeling_moss_vl.py
  • empty_cache: release NPU HBM after realtime sessions (torch_npu import
    guarded; the CUDA branch is unchanged)
  • realtime idle wait: a bare while True: continue busy-spin monopolizes
    the CPython GIL and starves co-resident threads (measured: an in-process
    ASR decode inflates ~150ms -> 40-80s while a session idles in
    <|silence|>). Replaced with a 20ms sleep poll. Platform-neutral — CUDA
    deployments embedding this checkpoint benefit equally.
• processing / video_processing: the 10-D vision permute+reshape is routed
  through CPU on NPU (NPU ops support at most 8-D tensors); CUDA keeps the
  on-device path untouched.
• preprocessor_config.json: declare interpolation=BICUBIC explicitly (NPU has
  no implicit resize kernel; explicit declaration keeps CUDA outputs
  identical — pixel_values verified bitwise-equal on identical inputs,
  including forced-resize cases).

Verified on 910B2C x8 with the realtime demo (ASR/TTS/VLM E2E green); the
GIL fix restores in-process ASR latency from 52.5s to ~150ms.

对于这个 processing_moss_vl.py 的修改 我理解这里默认值好像没必要改
是否类似于 video_processing_moss_vl.py 的修改部分会更好一些 我理解