Joiin0392 commited on
Commit
6e2417b
·
verified ·
1 Parent(s): 06b0676

fix: yield GIL in realtime idle wait — busy-wait starves co-resident threads

Browse files

The realtime loop's idle branch (<|silence|>, nothing pending) spins on
`while True: continue`, monopolizing the CPython GIL. Every other thread in
the process is starved for as long as the session idles — e.g. an in-process
ASR decode inflates from ~150ms to 40-80s.

Measured (one busy-spin thread + SenseVoice decode in the same process):
584ms -> 42.6s (73x). Replacing the bare `continue` with a 20ms sleep poll
restores 132-152ms; input-detection latency stays negligible.

Platform-neutral: GIL behavior is identical on CUDA deployments — any
co-process embedding (ASR, TTS, metrics) is affected the same way.

Files changed (1) hide show
  1. modeling_moss_vl.py +5 -2
modeling_moss_vl.py CHANGED
@@ -3099,8 +3099,11 @@ class MossVLForConditionalGeneration(MossVLPreTrainedModel, GenerationMixin):
3099
  break
3100
 
3101
  if self.continue_generating and should_wait_for_new_input and not frames_to_process and not prompts_to_process:
3102
- # Busy-wait matches VideoMllama reference. Caller controls cadence via
3103
- # `max_tokens_per_turn` sleeping in `_real_time_sample`.
 
 
 
3104
  continue
3105
  break
3106
 
 
3099
  break
3100
 
3101
  if self.continue_generating and should_wait_for_new_input and not frames_to_process and not prompts_to_process:
3102
+ # GIL-friendly wait: a bare `continue` busy-spin monopolizes the GIL
3103
+ # and starves in-process siblings (in-process ASR decode inflates
3104
+ # ~150ms -> 40-80s while a realtime session idles in <|silence|>).
3105
+ # 20ms poll keeps input-detection latency negligible.
3106
+ time.sleep(0.02)
3107
  continue
3108
  break
3109