Spaces:
Running on Zero
Running on Zero
Multi-user concurrency: compose outside the GPU queue, parallel assists/examples/videos
#5
by multimodalart HF Staff - opened
Simple mode ran the composer LLM call inside the concurrency_id="gpu" (limit 1) event, so after clicking Generate the LLM request did not even START until every queued song from other users finished — the delay users are reporting. Also, Gradio's default concurrency limit is 1 per function, so one user's assist/example/video blocked everyone else's.
- Simple flow split: simple_compose (limit 8, starts immediately) -> .success -> studio_generate on the GPU queue. Status shows 'waiting for a GPU slot' between the two.
- compose_assist: limit 8. render_video: limit 4. load_example (cached files): unlimited.
- player.stop cancels the compose step too.
API note: the /simple_generate endpoint is replaced by /simple_compose + the chained sing step.
multimodalart changed pull request status to merged