PlagueKind V1.5 workflow: linear_quadratic sigmas, RCAS, FILM, split unquantized deployment

README.md

@@ -1,18 +1,99 @@
 ---
-title: MiniMax-H3
-emoji: 🎬
+title: PlagueKind MiniMax H3
+emoji: 🦊
 colorFrom: gray
 colorTo: indigo
 sdk: gradio
-sdk_version: 6.22.0
+sdk_version: 6.20.0
 app_file: app.py
-short_description: MiniMax-H3 video generation with synchronized audio
-python_version: "3.12"
+short_description: The PlagueKind V1.5 workflow for MiniMax-H3 video + audio
+license: gpl-3.0
 startup_duration_timeout: 1h
+suggested_hardware: zero-a10g
 ---
 
-# MiniMax-H3
+# `Plaguekind/Minimax-H3` — the V1.5 workflow, as a Space
 
-MiniMax-H3 is a 33B parameter state-of-the-art video generation model that produces video and a fully synchronized soundtrack (ambience, foley, speech). This Space runs the FL2VA variant with Int8 weight-only quantization on ZeroGPU.
+[`Plaguekind/Minimax-H3`](https://huggingface.co/Plaguekind/Minimax-H3) ships **no weights**. It is a ComfyUI graph
+(`PlagueKind-MinimaxH3-V1.5.json`) over [`Comfy-Org/MiniMax-H3`](https://huggingface.co/Comfy-Org/MiniMax-H3), and
+everything it contributes is in the sampling and the post chain. So what this Space reproduces is the *graph*, on the
+[`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3) diffusers checkpoint.
 
-Based on [Plaguekind/Minimax-H3](https://huggingface.co/Plaguekind/Minimax-H3) (ComfyUI workflow wrapper) and [MiniMaxAI/MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) (original model).
+**MiniMax-H3** is a 33B-parameter single-stream omni DiT that denoises video and a synchronized stereo soundtrack —
+ambience, foley, speech — as one packed sequence, in one pass. Text-to-video, first frame, last frame, or both.
+
+## What the workflow changes
+
+| ComfyUI node | widget | here |
+|---|---|---|
+| `KSamplerSelect` | `euler` | MiniMax-H3's only sampler; the checkpoint is CFG-distilled, so one forward per step and no negative prompt |
+| `BasicScheduler` | `linear_quadratic`, 15 steps, denoise 1.0 | **Sigma schedule** / **Steps** — `pk_workflow.linear_quadratic_sigmas` |
+| `MiniMaxH3ImageToVideo` | prompt, first/last frame | **Prompt** / **First frame** / **Last frame** |
+| `UnifiedResizeImageMask` ("Target Dimension") | longer side 1344 | **Target dimension** |
+| `ImageSharpenKJ` | `rcas`, 0.3 | **RCAS sharpening** — `pk_workflow.rcas` |
+| `FrameInterpolate` + `FrameInterpolationModelLoader` | `film_net_fp16.safetensors`, multiplier 2 | **FILM frame interpolation** — `pk_workflow.interpolate` |
+| `CreateVideo` | fps `24 * 2` | 48 fps output |
+| `ComfyMathExpression` | `max(5, round(a*24)) + (5 - (… % 17)) % 17` | **Duration** snapped to `17n + 5` frames |
+| `RTXVideoSuperResolution` | 2x, `ULTRA` | **not reproduced** |
+| `PathchSageAttentionKJ` | `sageattn_qk_int8_pv_fp8_cuda++` | cuDNN fused attention |
+
+The sigma schedule is the part that changes the pixels most, and the part that is easy to get subtly wrong.
+`linear_quadratic` is Mochi's schedule, ported from `comfy/samplers.py`: half the steps crawl through the first 2.5 %
+of the trajectory and the rest sprint the remaining 97.5 %, which is why PlagueKind's 15 steps hold up against ~28 of
+MiniMax-H3's native grid.
+
+Transplanting it into `diffusers` exactly needs one observation. MiniMax-H3 carries **two** rectified-flow schedules
+per request, `shift = 12` for the video rows and `shift = 3` for the audio rows. `diffusers` builds both from one
+`linspace(1, 0, steps)` base grid; ComfyUI instead samples the *video* schedule and derives the audio one in closed
+form (`comfy/ldm/minimax/model.py::time_shift_sigma`). The two agree, because the exponential shift is a bijection of
+the base grid that fixes both 0 and 1 — so handing `MiniMaxH3Scheduler.set_timesteps` the `linear_quadratic` grid for
+the video stream and `time_shift_sigma(grid, 12, 3)` for the audio stream is the ComfyUI path, not an approximation of
+it.
+
+### Two deliberate deviations
+
+- **`RTXVideoSuperResolution`** is NVIDIA's NGX super-resolution, shipped as a driver-level Windows/RTX component with
+  no Linux Python path. The workflow's 2x upscale is therefore missing; pick a larger **Target dimension** instead of
+  upscaling a small one.
+- **SageAttention**'s `qk_int8_pv_fp8_cuda++` kernel is not built for this pool's sm120 cards. Attention runs cuDNN's
+  fused kernel, which is both the fastest available here and the numerically faithful choice — SageAttention is a
+  quantized approximation of it.
+
+### One upgrade
+
+The workflow loads `minimax_h3_fl2va_pruned_int8_convrot.safetensors` and a `qwen3vl_32b_…_int8_convrot` text encoder
+because that is what fits a consumer card. This Space runs both **unquantized bfloat16**.
+
+## Why the deployment is split
+
+MiniMax-H3 at bfloat16 is 195.9 GiB, and a Space is evicted above 150 GB of storage. So the halves live apart:
+
+- **this Space** — the 61.73 GiB transformer and the two autoencoders (10.43 GiB, float32: a bfloat16 audio VAE
+  decodes the soundtrack about 20 dB too quiet). 77.3 GB downloaded.
+- **[`multimodalart/qwen3vl-conditioner`](https://huggingface.co/spaces/multimodalart/qwen3vl-conditioner)** — the
+  62.14 GiB Qwen3-VL text encoder, called over the gradio API once per request. `prompt_embeds` +
+  `text_token_tags` in a safetensors file is the whole wire format. `gradio_client` forwards the caller's own ZeroGPU
+  token, so that booking is billed to whoever asked for the video.
+
+`H3_PLACEMENT=pack` moves only the transformer to CUDA at startup: `spaces` packs every startup-resident CUDA tensor
+into a second on-disk copy, and packing all 77.3 GB busts the storage quota while the 61.7 GB transformer alone fits.
+The VAEs move on the first GPU call. `H3_GPU_SIZE=xlarge` is required — `large` does not fit. `H3_AOTI=1` loads
+[`multimodalart/minimax-h3-aoti`](https://huggingface.co/multimodalart/minimax-h3-aoti), one ahead-of-time-compiled
+transformer block serving all 50, which removes roughly 0.5 s/step.
+
+## Files
+
+| file | |
+|---|---|
+| `app.py` | the demo |
+| `pk_workflow.py` | the workflow's own parts: `linear_quadratic` sigmas, RCAS, FILM |
+| `film_net.py` | FILM, vendored from ComfyUI (GPL-3.0) |
+| `h3_split_blocks.py` | the modular blocks that skip the text encoder |
+| `h3_aoti.py` | the AoTI package loader |
+
+## License
+
+The demo code is GPL-3.0, because `film_net.py` is vendored from [ComfyUI](https://github.com/comfyanonymous/ComfyUI)
+and `pk_workflow.py` ports kernels from ComfyUI and
+[ComfyUI-KJNodes](https://github.com/kijai/ComfyUI-KJNodes), both GPL-3.0. The workflow itself is MIT; the
+`MiniMaxAI/MiniMax-H3` weights carry their own license.

app.py

@@ -1,27 +1,42 @@
-"""MiniMax-H3 video generation with synchronized audio — FL2VA (text / first-last-frame to video+audio).
+"""`Plaguekind/Minimax-H3` — the PlagueKind V1.5 ComfyUI workflow for MiniMax-H3, as a Space.
 
-Uses the pruned NVFP4 transformer and local truncated NVFP4-AWQ Qwen3-VL conditioner for compact
-downloads (~42 GB total vs 119 GB BF16), fitting in ZeroGPU xlarge (96 GB VRAM).
+The candidate repository holds no weights: it is a ComfyUI graph over `Comfy-Org/MiniMax-H3`, so what is
+reproduced here is the *graph*, on the `MiniMaxAI/MiniMax-H3` diffusers checkpoint. See `pk_workflow.py` for the
+node-by-node mapping; the short version is euler + `linear_quadratic` at 15 steps, FSR RCAS sharpening at 0.3, and
+FILM 2x frame interpolation to 48 fps.
+
+Deployment is the split one the unquantized MiniMax-H3 needs: 195.9 GiB of bfloat16 does not fit under a Space's
+150 GB storage quota, so the 62.14 GiB Qwen3-VL text encoder runs in a separate Space
+(`multimodalart/qwen3vl-conditioner`) that this one calls per request, and this Space holds the 61.73 GiB
+transformer and the two autoencoders. `prompt_embeds` + `text_token_tags` is the whole wire format.
 """
 
 from __future__ import annotations
 
 import os
-
-# Allocator config for memory pressure (video DiTs have large transient allocations)
-os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
-
-import spaces  # MUST come before torch / any CUDA-touching import
-import gradio as gr
-
 import tempfile
 import time
 import traceback
+from functools import cache
 
-MODEL_REPO = os.environ.get("H3_MODEL_REPO", "MiniMaxAI/MiniMax-H3")
-GPU_SIZE = "xlarge"
+# Before anything that could initialize CUDA: `import spaces` patches `torch.cuda` so the 72 GiB load can happen at
+# startup rather than on GPU time.
+import spaces
+import gradio as gr
+
+import pk_workflow as pk
 
-# Canvas presets matching the original model's supported resolutions
+MODEL_REPO = os.environ.get("H3_MODEL_REPO", "MiniMaxAI/MiniMax-H3")
+CONDITIONER_SPACE = os.environ.get("H3_CONDITIONER", "multimodalart/qwen3vl-conditioner")
+# `pack` places the transformer at startup, `lazy` moves everything on the first GPU call.
+PLACEMENT = os.environ.get("H3_PLACEMENT", "pack").lower()
+# cuDNN's fused attention is 10-20% faster than the SDPA default on this pool and needs nothing installed. It is
+# also the closest available stand-in for the workflow's SageAttention patch, which is a sm90 build.
+ATTENTION = os.environ.get("H3_ATTENTION", "_native_cudnn").lower()
+GPU_SIZE = os.environ.get("H3_GPU_SIZE", "xlarge")
+
+# Must stay identical to the conditioner's table: the *label* goes over the wire, so a canvas that half does not
+# know is rejected there and surfaces as a failure here. This is the workflow's "Target Dimension" node.
 CANVASES = {
     # 16:9
     "960x544 · 16:9 fast": (544, 960),
@@ -45,13 +60,32 @@ CANVASES = {
     "1152x512 · 21:9 fast": (512, 1152),
     "1536x672 · 21:9 full": (672, 1536),
 }
+# PlagueKind's V1.5 note: "FFLF is unreliable at res above 640". 960x544 keeps the short edge under that and is the
+# canvas where the AoTI package pays most, so it is the default; the full 768 short edge is one dropdown away.
 DEFAULT_CANVAS = "960x544 · 16:9 fast"
+
 FPS, FRAMES_PER_CHUNK, LATENTS_PER_CHUNK = 24, 17, 5
-MIN_UI_DURATION, MAX_UI_DURATION = 2, 10
+# It is the *snapped* frame count the ceiling has to hold for: 15 s is 360 frames, which rounds up to 362, i.e.
+# 15.083 s, and is refused.
+MIN_UI_DURATION, MAX_UI_DURATION = 2, 14
+
+SCHEDULES = {
+    "linear_quadratic · PlagueKind": "linear_quadratic",
+    "native · shift 12": "native",
+}
+DEFAULT_SCHEDULE = "linear_quadratic · PlagueKind"
+INTERPOLATION = {"off · 24 fps": 1, "2x · 48 fps (PlagueKind)": 2, "4x · 96 fps": 4}
+DEFAULT_INTERPOLATION = "2x · 48 fps (PlagueKind)"
+DEFAULT_SHARPEN = 0.3
+DEFAULT_STEPS = 15
 
 
 def snap_frames(seconds: float) -> int:
-    """The frame count MiniMax-H3's video VAE can decode: the next 17*n+5 at 24 fps."""
+    """The frame count MiniMax-H3's video VAE can decode: the next `17 * n + 5` at 24 fps.
+
+    Identical to the workflow's `ComfyMathExpression`,
+    `max(5, round(a*24)) + (5 - (max(5, round(a*24)) % 17)) % 17` — 5 s is 124 frames, i.e. 5.167 s.
+    """
     frames = max(1, round(float(seconds) * FPS))
     while frames % FRAMES_PER_CHUNK != LATENTS_PER_CHUNK:
         frames += 1
@@ -59,27 +93,43 @@ def snap_frames(seconds: float) -> int:
 
 
 def lower_duration_floor(seconds: float = MIN_UI_DURATION) -> None:
-    """Let the pipeline generate below its 5 s floor."""
+    """Let the pipeline generate below its 5 s floor. 56 frames (2.33 s) is fine on the released checkpoint."""
     from diffusers.modular_pipelines.minimax_h3.modular_pipeline import MiniMaxH3ModularPipeline
 
     MiniMaxH3ModularPipeline.min_duration = property(lambda self: float(seconds))
 
 
 PIPE = None
-COND_PIPE = None
-COND_ERROR: str | None = None
+FILM = None
+FILM_ERROR: str | None = None
 LOAD_ERROR: str | None = None
 LOADED_IN: float | None = None
 
 
+def status() -> str:
+    if LOAD_ERROR:
+        return LOAD_ERROR
+    if PIPE is None:
+        return f"Loading `{MODEL_REPO}` (transformer + VAEs, 77.3 GB). Watch the Space logs."
+    import h3_aoti
+
+    film = "FILM **ready**" if FILM is not None else f"FILM **off** ({FILM_ERROR})"
+    return (
+        f"Ready · transformer + VAEs **bfloat16, unquantized** · placement `{PLACEMENT}` · attention "
+        f"`{ATTENTION}` · {h3_aoti.status()} · {film} · loaded in {LOADED_IN:.0f}s · conditioner "
+        f"`{CONDITIONER_SPACE}`"
+    )
+
+
 def load_models() -> str | None:
-    """Load the compact generator and local truncated conditioner at startup.
+    """Load the denoising half at startup, plus FILM.
 
-    Uses MiniMaxH3GeneratorBlocks (only VAEs + schedulers + video_processor from MiniMaxAI/MiniMax-H3,
-    ~10 GB) plus the pruned NVFP4 transformer from lilcheaty/MiniMax-H3-NVFP4 (~16 GB) and the local
-    NVFP4-AWQ Qwen3-VL conditioner from Comfy-Org/MiniMax-H3 (~16 GB). Total download: ~42 GB.
+    `MiniMaxH3GeneratorBlocks` declares `transformer`, `vae`, `audio_vae`, the two schedulers and `video_processor`,
+    so `load_components` fetches exactly those subfolders — `text_encoder/` and `transformer_ref/` are never
+    touched. Both autoencoders carry `_keep_in_fp32_modules` over every module and stay float32: a bfloat16 audio
+    VAE decodes the soundtrack roughly 20 dB too quiet.
     """
-    global PIPE, COND_PIPE, COND_ERROR, LOAD_ERROR, LOADED_IN
+    global PIPE, FILM, FILM_ERROR, LOAD_ERROR, LOADED_IN
 
     if PIPE is not None or LOAD_ERROR is not None:
         return LOAD_ERROR
@@ -96,166 +146,286 @@ def load_models() -> str | None:
         blocks = MiniMaxH3GeneratorBlocks()
         print(f"[gen] loading {[c.name for c in blocks.expected_components]} from {MODEL_REPO} ...", flush=True)
         pipe = blocks.init_pipeline(MODEL_REPO, components_manager=manager, collection="h3")
+        pipe.load_components(dtype=torch.bfloat16)
+        pipe.transformer.set_attention_backend(ATTENTION)
 
-        # Load only VAEs, schedulers, and video_processor from the main repo (~10 GB)
-        pipe.load_components(
-            names=["vae", "audio_vae", "scheduler", "audio_scheduler", "video_processor"],
-            dtype=torch.bfloat16,
-        )
-
-        # Load the pruned NVFP4 transformer from the separate checkpoint repo
-        from h3_nvfp4 import load_transformer
+        # Still startup, still free: an AoTI package carries no weights and opens its archive lazily inside the GPU
+        # worker.
+        import h3_aoti
 
-        pipe.update_components(transformer=load_transformer())
-        pipe.transformer.set_attention_backend("_native_cudnn")
+        h3_aoti.maybe_load(pipe.transformer)
 
-        # Load the local truncated NVFP4-AWQ Qwen3-VL conditioner
-        try:
-            from h3_local_conditioner import load_local_conditioner
-            from h3_split_blocks import MiniMaxH3ConditionerBlocks
-
-            print("[cond] loading the local truncated NVFP4-AWQ conditioner ...", flush=True)
-            text_encoder, tokenizer, processor = load_local_conditioner()
-            cond_pipe = MiniMaxH3ConditionerBlocks().init_pipeline(MODEL_REPO)
-            cond_pipe.update_components(
-                text_encoder=text_encoder,
-                tokenizer=tokenizer,
-                processor=processor,
-            )
-        except Exception as error:
-            traceback.print_exc()
-            COND_ERROR = f"{type(error).__name__}: {error}"
-            print(f"[cond] local load failed ({COND_ERROR})", flush=True)
+        if PLACEMENT == "pack":
+            # Scoped to the transformer. `spaces` packs every startup-resident CUDA tensor into a second on-disk
+            # copy, and packing all 77.3 GB busts the 150 GB storage quota; the 61.7 GB transformer alone fits. The
+            # ~10 GB of fp32 VAEs move on the first GPU call instead.
+            pipe.transformer.to("cuda")
 
-        PIPE, COND_PIPE = pipe, cond_pipe
+        PIPE = pipe
         LOADED_IN = time.time() - started
         print(f"[gen] ready in {LOADED_IN:.0f}s", flush=True)
     except Exception as error:
         traceback.print_exc()
-        LOAD_ERROR = f"**Loading `{MODEL_REPO}` failed** after {time.time() - started:.0f}s: `{type(error).__name__}: {error}`"
+        LOAD_ERROR = (
+            f"**Loading `{MODEL_REPO}` failed** after {time.time() - started:.0f}s: "
+            f"`{type(error).__name__}: {error}`"
+        )
+        return LOAD_ERROR
+
+    # 69 MB of post-processing, and the demo is still a demo without it, so a failure here is not fatal.
+    try:
+        FILM = pk.load_film()
+        print("[gen] FILM loaded", flush=True)
+    except Exception as error:
+        FILM_ERROR = f"{type(error).__name__}: {error}"
+        print(f"[gen] FILM unavailable ({FILM_ERROR}); frame interpolation disabled", flush=True)
+
     return LOAD_ERROR
 
 
-def status() -> str:
-    if LOAD_ERROR:
-        return LOAD_ERROR
-    if PIPE is None:
-        return "Loading pruned NVFP4 transformer + local NVFP4 conditioner + full-precision VAEs (~42 GB). Watch the Space logs."
-    import h3_nvfp4
-    engine_status = h3_nvfp4.status()
-    cond_status = "local NVFP4-AWQ" if COND_PIPE is not None else f"unavailable ({COND_ERROR})"
-    return f"Ready · {engine_status} · VAEs full precision · loaded in {LOADED_IN:.0f}s · conditioner {cond_status}"
+@cache
+def conditioner():
+    """The other half, over the gradio API. `gradio_client` attaches the caller's own ZeroGPU token per call, so
+    the conditioner's booking is billed to whoever asked for the video."""
+    from gradio_client import Client
 
+    return Client(CONDITIONER_SPACE)
+
+
+def encode_remote(prompt, image_path, last_image_path, canvas, num_frames, rewrite_prompt=False):
+    """`/encode` on the conditioner Space: a safetensors file holding `prompt_embeds` + `text_token_tags`, with the
+    resolved `height` / `width` / `num_frames` in its metadata, plus the plan. `canvas` is the label."""
+    from gradio_client import handle_file
+    from safetensors import safe_open
+
+    path, plan = conditioner().predict(
+        prompt=prompt,
+        image_path=handle_file(image_path) if image_path else None,
+        last_image_path=handle_file(last_image_path) if last_image_path else None,
+        canvas=canvas,
+        num_frames=num_frames,
+        rewrite_prompt=bool(rewrite_prompt),
+        api_name="/encode",
+    )
+    with safe_open(path, framework="pt") as handle:
+        metadata = handle.metadata()
+        return handle.get_tensor("prompt_embeds"), handle.get_tensor("text_token_tags"), metadata, plan
 
-# Duration estimation
+
+# Seconds of GPU one request needs, from the packed video rows it is about to denoise: linear in the rows for the
+# matmuls, quadratic for the attention, against the AoTI block package this Space runs.
 _DUR_B, _DUR_C = 1.1745e-4, 3.8396e-9
+# The two resident decoders, the post chain and the mux, which scale with the output rather than with the step
+# count. `_DEFAULT_CANVAS_PIXELS` is 960x544x124, the default request.
 _DECODE_BASE, _DECODE_PER_DEFAULT_CANVAS, _DEFAULT_CANVAS_PIXELS = 15, 15, 960 * 544 * 124
+# FILM, per *emitted* intermediate frame at the default canvas. Re-measured against the live Space; a 2x pass over
+# 124 frames at 960x544 is 123 of them.
+_FILM_PER_FRAME = 0.16
+# `pack` mode: only the ~10 GB of VAEs move on a cold worker.
 _PLACEMENT_ALLOWANCE, _PAD = 12, 10
 
 
-def get_duration(prompt, image, last_image, height, width, num_frames, steps, seed, *a, **k):
-    """Estimate GPU seconds needed for this request."""
+def get_duration(
+    prompt_embeds,
+    text_token_tags,
+    first_frame,
+    last_frame,
+    height,
+    width,
+    num_frames,
+    steps,
+    schedule,
+    sharpen,
+    multiplier,
+    seed,
+    *a,
+    **k,
+):
     height, width, num_frames, steps = int(height), int(width), int(num_frames), int(steps)
+    multiplier = max(1, int(multiplier))
     latent_frames = (num_frames - LATENTS_PER_CHUNK) // FRAMES_PER_CHUNK * LATENTS_PER_CHUNK + 2
     patches = (height // 32) * (width // 32)
-    rows = latent_frames * patches + (int(image is not None) + int(last_image is not None)) * patches
+    keyframes = int(first_frame is not None) + int(last_frame is not None)
+    rows = latent_frames * patches + keyframes * patches
     denoise = steps * (_DUR_B * rows + _DUR_C * rows**2)
+    pixel_ratio = (height * width) / (960 * 544)
     decode = _DECODE_BASE + _DECODE_PER_DEFAULT_CANVAS * (height * width * num_frames) / _DEFAULT_CANVAS_PIXELS
-    local_conditioning = 20
-    return max(60, int(denoise + decode) + local_conditioning + _PLACEMENT_ALLOWANCE + _PAD)
+    film = 0.0
+    if multiplier > 1 and FILM is not None:
+        film = (num_frames - 1) * (multiplier - 1) * _FILM_PER_FRAME * pixel_ratio
+    return max(60, int(denoise + decode + film) + _PLACEMENT_ALLOWANCE + _PAD)
 
 
 @spaces.GPU(duration=get_duration, size=GPU_SIZE)
-def generate(prompt, image, last_image, height, width, num_frames, steps, seed,
-             acceleration="Balanced", progress=gr.Progress(track_tqdm=True)):
-    """Generate video with synchronized audio from text and optional keyframes."""
+def _generate(
+    prompt_embeds,
+    text_token_tags,
+    first_frame,
+    last_frame,
+    height,
+    width,
+    num_frames,
+    steps,
+    schedule,
+    sharpen,
+    multiplier,
+    seed,
+):
+    """The only thing on GPU time: the denoise loop, the two decoders and the workflow's post chain.
+
+    The mp4 is muxed here rather than in the caller: a `@spaces.GPU` return crosses a process boundary by pickling,
+    and a 2x-interpolated 124-frame clip is several hundred MB of frames against a few MB of h264.
+    """
     import torch
 
-    # Move models to CUDA on each cold worker
-    if COND_PIPE is not None:
-        COND_PIPE.text_encoder.to("cuda")
-    PIPE.to("cuda")
+    from diffusers.utils import encode_video
 
-    # Local conditioning
-    condition_state = COND_PIPE(
-        prompt=prompt,
-        image=image,
-        last_image=last_image,
-        height=int(height),
-        width=int(width),
-    )
-    prompt_embeds = condition_state.get("prompt_embeds")
-    text_token_tags = condition_state.get("text_token_tags")
+    global FILM
 
-    begin_request = getattr(PIPE.transformer, "begin_request", None)
-    end_request = getattr(PIPE.transformer, "end_request", None)
-    if begin_request is not None:
-        begin_request(int(steps), acceleration)
-    try:
-        with torch.inference_mode():
-            state = PIPE(
-                prompt_embeds=prompt_embeds.to("cuda", non_blocking=True),
-                text_token_tags=text_token_tags,
-                image=image,
-                last_image=last_image,
-                height=int(height),
-                width=int(width),
-                num_frames=int(num_frames),
-                num_inference_steps=int(steps),
-                generator=torch.Generator("cpu").manual_seed(int(seed)),
-            )
-    finally:
-        if end_request is not None:
-            end_request()
+    if PLACEMENT == "lazy":
+        PIPE.to("cuda")
+    elif PLACEMENT == "pack":
+        PIPE.vae.to("cuda")
+        PIPE.audio_vae.to("cuda")
 
-    from diffusers.utils import encode_video
+    steps = int(steps)
+    multiplier = max(1, int(multiplier))
+    linear_quadratic = schedule == "linear_quadratic"
+    # `linear_quadratic` hands `set_timesteps` a finished `steps + 1` sigma grid, so it runs `steps` forwards. The
+    # native grid counts its terminal zero as one of `num_inference_steps`, so it needs one more to match.
+    requested_steps = steps if linear_quadratic else steps + 1
 
-    videos = state.get("videos")
-    audio = state.get("audio")
-    sampling_rate = state.get("sampling_rate")
+    started = time.time()
+    with pk.use_linear_quadratic(PIPE, steps, enabled=linear_quadratic):
+        state = PIPE(
+            prompt_embeds=prompt_embeds.to("cuda"),
+            text_token_tags=text_token_tags,
+            image=first_frame,
+            last_image=last_frame,
+            height=height,
+            width=width,
+            num_frames=num_frames,
+            num_inference_steps=requested_steps,
+            output_type="pt",
+            generator=torch.Generator("cpu").manual_seed(int(seed)),
+        )
+    denoised = time.time() - started
 
-    directory = os.path.join(tempfile.gettempdir(), "h3-outputs")
+    video = state.get("videos")[0]  # (frames, 3, H, W), float in [0, 1], on the card
+    audio = state.get("audio")[0].cpu()
+    sampling_rate = state.get("sampling_rate")
+    del state
+    # The post chain runs on the allocator the denoise loop just left fragmented (78.5 GiB at the full canvas), and
+    # RCAS and FILM both want a few contiguous gigabytes.
+    torch.cuda.empty_cache()
+
+    post = time.time()
+    video = pk.rcas(video, float(sharpen))
+    if multiplier > 1:
+        if FILM is None:
+            multiplier = 1
+        else:
+            FILM = FILM.to("cuda")
+            video = pk.interpolate(FILM, video, multiplier)
+    fps = FPS * multiplier
+    frames = (video.permute(0, 2, 3, 1).float() * 255.0).round_().clamp_(0, 255).to(torch.uint8).cpu()
+    del video
+    post_seconds = time.time() - post
+
+    directory = os.path.join(tempfile.gettempdir(), "pk-h3-outputs")
     os.makedirs(directory, exist_ok=True)
-    path = os.path.join(directory, f"h3-{int(time.time() * 1000)}.mp4")
-    encode_video(
-        videos[0],
-        fps=FPS,
-        output_path=path,
-        audio=audio[0].cpu(),
-        audio_sample_rate=sampling_rate,
-    )
-    return path
-
-
-def run_generate(prompt, image_path=None, last_image_path=None, canvas=DEFAULT_CANVAS,
-                 duration=5, steps=28, seed=42, acceleration="Balanced",
-                 progress=gr.Progress(track_tqdm=True)):
-    """Handle a generation request from the Gradio UI."""
+    path = os.path.join(directory, f"pk-h3-{int(time.time() * 1000)}.mp4")
+    encode_video(frames, fps=fps, output_path=path, audio=audio, audio_sample_rate=sampling_rate)
+
+    return path, denoised, post_seconds, int(frames.shape[0]), fps, multiplier
+
+
+def generate(
+    prompt,
+    canvas=DEFAULT_CANVAS,
+    first_frame=None,
+    last_frame=None,
+    duration=5,
+    steps=DEFAULT_STEPS,
+    schedule=DEFAULT_SCHEDULE,
+    sharpen=DEFAULT_SHARPEN,
+    interpolation=DEFAULT_INTERPOLATION,
+    seed=42,
+    upsample=False,
+    progress=gr.Progress(track_tqdm=True),
+):
+    """One request through the PlagueKind graph. Every parameter but the prompt carries the default its UI
+    component carries, so an example that fills only `prompt` (and `canvas`) behaves exactly like the button."""
     if LOAD_ERROR:
         raise gr.Error(LOAD_ERROR)
     if PIPE is None:
-        raise gr.Error("The model is still loading. Please wait a moment and try again.")
+        raise gr.Error("The denoiser is still loading.")
     if not prompt or not prompt.strip():
         raise gr.Error("MiniMax-H3 always takes a prompt, keyframes or not.")
 
     from PIL import Image, ImageOps
 
+    canvas = canvas or DEFAULT_CANVAS
+    schedule_key = SCHEDULES.get(schedule, "linear_quadratic")
+    multiplier = INTERPOLATION.get(interpolation, 2)
     num_frames = snap_frames(duration)
-    height, width = CANVASES[canvas]
+
+    progress(
+        0.0,
+        desc=(
+            f"Upsampling the prompt on {CONDITIONER_SPACE} ..."
+            if upsample
+            else f"Conditioning on {CONDITIONER_SPACE} ..."
+        ),
+    )
+    conditioned = time.time()
+    prompt_embeds, text_token_tags, metadata, plan = encode_remote(
+        prompt, first_frame, last_frame, canvas, num_frames, rewrite_prompt=upsample
+    )
+    condition_seconds = time.time() - conditioned
+    height, width, num_frames = (int(metadata[key]) for key in ("height", "width", "num_frames"))
+    refined = plan.get("refined_prompt") or ""
 
     def keyframe(path):
+        # The conditioning latents encoded here have to be of the image the conditioner looked at, which it
+        # prepares exactly this way.
         return ImageOps.exif_transpose(Image.open(path)).convert("RGB") if path else None
 
-    first_frame = keyframe(image_path) if image_path else None
-    final_frame = keyframe(last_image_path) if last_image_path else None
+    progress(0.1, desc=f"Denoising {int(steps)} steps at {width}x{height}, {num_frames} frames ...")
+    path, denoise_seconds, post_seconds, out_frames, fps, multiplier = _generate(
+        prompt_embeds,
+        text_token_tags,
+        keyframe(first_frame),
+        keyframe(last_frame),
+        height,
+        width,
+        num_frames,
+        int(steps),
+        schedule_key,
+        float(sharpen),
+        multiplier,
+        int(seed),
+    )
 
-    progress(0.05, desc=f"Denoising {steps} steps at {width}x{height}, {num_frames} frames ...")
-    path = generate(prompt, first_frame, final_frame, height, width, num_frames, steps, seed, acceleration, progress)
-    return path
+    post = [f"RCAS {float(sharpen):.2f}" if float(sharpen) > 0 else "no sharpening"]
+    post.append(f"FILM {multiplier}x -> {fps} fps" if multiplier > 1 else f"{fps} fps")
+    report = (
+        f"`{width}x{height}`, {num_frames} frames ({num_frames / FPS:.3f} s) -> {out_frames} frames at {fps} fps · "
+        f"{int(steps)} steps of `{schedule_key}` · {' · '.join(post)} · seed {int(seed)}\n\n"
+        f"conditioner {condition_seconds:.0f}s ({plan['num_text_tokens']} tokens"
+        f"{', upsampled' if refined else ''}) · denoise + decode {denoise_seconds:.0f}s "
+        f"({denoise_seconds / max(1, int(steps)):.1f} s/step) · post {post_seconds:.0f}s"
+    )
+    if refined:
+        report += f"\n\n**Upsampled prompt**\n\n{refined}"
+    print(f"[gen] {report}", flush=True)
+    return path, report
 
 
 def _fit_keyframe(image_path, current_canvas):
-    """Cover-crop an uploaded keyframe to the closest supported aspect ratio."""
+    """Cover-crop an uploaded keyframe to the closest supported aspect ratio and select that ratio's smallest
+    (fastest) canvas, unless the user already picked a matching ratio. The workflow's "Target Dimension" node does
+    the same job by hand."""
     if not image_path:
         return gr.update(), gr.update()
     from PIL import Image as _Image
@@ -292,74 +462,183 @@ def _fit_keyframe(image_path, current_canvas):
 
 load_models()
 
-INTRO = """# MiniMax-H3
+INTRO = """# PlagueKind · MiniMax-H3
 
 <div align="center">
+  <a href="https://huggingface.co/Plaguekind/Minimax-H3" target="_blank" rel="noopener"><strong>[ workflow ]</strong></a> &nbsp;
   <a href="https://huggingface.co/MiniMaxAI/MiniMax-H3" target="_blank" rel="noopener"><strong>[ model ]</strong></a> &nbsp;
-  <a href="https://www.minimax.io/blog/minimax-h3" target="_blank" rel="noopener"><strong>[ blog ]</strong></a> &nbsp;
-  <a href="https://huggingface.co/Plaguekind/Minimax-H3" target="_blank" rel="noopener"><strong>[ ComfyUI weights ]</strong></a>
+  <a href="https://github.com/PlagueKind/Comfyui-PlagueKind-Nodes" target="_blank" rel="noopener"><strong>[ nodes ]</strong></a>
 </div>
 
-**MiniMax-H3** is a 33B parameter state-of-the-art video generation model that produces video and a
-fully synchronized soundtrack (ambience, foley, speech). Supports text-to-video and first/last-frame-to-video.
+**MiniMax-H3** is a 33B parameter video generation model that produces video and a fully synchronized soundtrack
+(ambience, foley, speech) in one pass. **PlagueKind's V1.5 workflow** is a tuning of it: euler on a
+`linear_quadratic` sigma grid at 15 steps, FSR **RCAS** sharpening at 0.3, and **FILM** 2x frame interpolation to
+48 fps. Text-to-video, first frame, last frame, or both.
 """
 
 CSS = """
 .main.fillable {max-width: 1250px !important}
 .dark .gradio-container { color: var(--body-text-color); }
+.status p {font-size: 0.8rem; opacity: 0.65; text-align: center;}
 """
 
-with gr.Blocks(title="MiniMax-H3") as demo:
+with gr.Blocks(title="PlagueKind · MiniMax-H3", theme=gr.themes.Citrus(), css=CSS) as demo:
     gr.Markdown(INTRO)
+    gr.Markdown(status(), elem_classes="status")
 
     with gr.Row():
         with gr.Column():
             prompt = gr.Textbox(
                 label="Prompt",
                 lines=3,
-                value="A red fox trotting through a snowy pine forest at dawn, snow crunching underfoot",
+                value=(
+                    "A red fox trotting through a snowy pine forest at dawn, snow crunching underfoot, "
+                    "distant birdsong"
+                ),
+            )
+            canvas = gr.Dropdown(
+                label="Target dimension", choices=list(CANVASES), value=DEFAULT_CANVAS
             )
             with gr.Row():
-                image = gr.Image(label="First frame (optional)", type="filepath")
-                last_image = gr.Image(label="Last frame (optional)", type="filepath")
+                first_frame = gr.Image(label="First frame (optional)", type="filepath")
+                last_frame = gr.Image(label="Last frame (optional)", type="filepath")
             run = gr.Button("Generate", variant="primary")
+
             with gr.Accordion("Advanced options", open=False):
-                canvas = gr.Dropdown(label="Canvas", choices=list(CANVASES), value=DEFAULT_CANVAS)
-                duration = gr.Slider(label="Duration (s)", minimum=MIN_UI_DURATION, maximum=MAX_UI_DURATION, step=1, value=5)
-                acceleration = gr.Radio(
-                    label="Acceleration",
-                    choices=["Balanced", "Exact"],
-                    value="Balanced",
-                    info="Balanced uses adaptive step reuse; Exact evaluates every step.",
+                duration = gr.Slider(
+                    label="Duration (s)",
+                    minimum=MIN_UI_DURATION,
+                    maximum=MAX_UI_DURATION,
+                    step=1,
+                    value=5,
+                )
+                steps = gr.Slider(
+                    label="Steps",
+                    minimum=8,
+                    maximum=40,
+                    step=1,
+                    value=DEFAULT_STEPS,
+                    info="PlagueKind: 15-20 on the linear_quadratic grid.",
+                )
+                schedule = gr.Dropdown(
+                    label="Sigma schedule",
+                    choices=list(SCHEDULES),
+                    value=DEFAULT_SCHEDULE,
+                    info="`linear_quadratic` front-loads half the steps into the first 2.5% of the trajectory.",
+                )
+                sharpen = gr.Slider(
+                    label="RCAS sharpening",
+                    minimum=0.0,
+                    maximum=1.0,
+                    step=0.05,
+                    value=DEFAULT_SHARPEN,
+                    info="FidelityFX Robust Contrast Adaptive Sharpening. PlagueKind: 0.3 is very natural.",
+                )
+                interpolation = gr.Dropdown(
+                    label="FILM frame interpolation",
+                    choices=list(INTERPOLATION),
+                    value=DEFAULT_INTERPOLATION,
+                    info="MiniMax-H3 generates 24 fps; FILM synthesizes the frames in between.",
                 )
-                steps = gr.Slider(label="Steps", minimum=10, maximum=40, step=1, value=28)
                 seed = gr.Number(label="Seed", value=42, precision=0)
+                upsample = gr.Checkbox(
+                    label="Upsample prompt",
+                    value=False,
+                    info="Rewrite the prompt on the conditioner Space first, MiniMax's Context-IR style.",
+                )
 
         with gr.Column():
             video = gr.Video(label="Video + soundtrack")
-
-    image.upload(_fit_keyframe, [image, canvas], [image, canvas])
-
+            report = gr.Markdown()
+
+    first_frame.upload(_fit_keyframe, [first_frame, canvas], [first_frame, canvas])
+    last_frame.upload(_fit_keyframe, [last_frame, canvas], [last_frame, canvas])
+
+    controls = [
+        prompt,
+        canvas,
+        first_frame,
+        last_frame,
+        duration,
+        steps,
+        schedule,
+        sharpen,
+        interpolation,
+        seed,
+        upsample,
+    ]
+
+    # Two blocks rather than one with empty keyframe cells: the positional order below matches `generate`'s
+    # signature, so each block fills a prefix of it and the defaults cover the rest.
     gr.Examples(
+        label="Text to video",
         examples=[
-            ["A red fox trotting through a snowy pine forest at dawn, snow crunching underfoot", None, None, "960x544 · 16:9 fast"],
-            ["A busy night market, neon signs reflecting in puddles, sizzling street food", None, None, "544x960 · 9:16 fast"],
-            ["A cellist playing a slow melody in an empty concert hall", None, None, "544x544 · 1:1 fast"],
+            ["A red fox trotting through a snowy pine forest at dawn, snow crunching underfoot", DEFAULT_CANVAS],
+            ["A busy night market, neon signs reflecting in puddles, sizzling street food", "544x960 · 9:16 fast"],
+            ["A cellist playing a slow melody in an empty concert hall", "544x544 · 1:1 fast"],
         ],
-        inputs=[prompt, image, last_image, canvas],
-        outputs=[video],
-        fn=run_generate,
+        inputs=[prompt, canvas],
+        outputs=[video, report],
+        fn=generate,
         cache_examples=True,
         cache_mode="lazy",
     )
-
-    run.click(
-        run_generate,
-        [prompt, image, last_image, canvas, duration, steps, seed, acceleration],
-        [video],
-        api_name="generate",
+    gr.Examples(
+        label="First and last frame",
+        examples=[
+            [
+                "A slow seamless camera move from the first view to the last, wind in the pines",
+                "1152x640 · 16:9",
+                "examples/first.png",
+                "examples/last.png",
+            ],
+            [
+                "The fox looks around, then trots deeper into the forest",
+                "1152x640 · 16:9",
+                "examples/first.png",
+                "examples/first.png",
+            ],
+        ],
+        inputs=[prompt, canvas, first_frame, last_frame],
+        outputs=[video, report],
+        fn=generate,
+        cache_examples=True,
+        cache_mode="lazy",
     )
 
+    with gr.Accordion("What this workflow changes, and what it cannot", open=False):
+        gr.Markdown(
+            """
+`Plaguekind/Minimax-H3` ships no weights — it is a ComfyUI graph over `Comfy-Org/MiniMax-H3`. Its nodes map onto
+this Space as:
+
+| ComfyUI node | widget | here |
+|---|---|---|
+| `KSamplerSelect` | `euler` | MiniMax-H3's only sampler; the checkpoint is CFG-distilled, so one forward per step and no negative prompt |
+| `BasicScheduler` | `linear_quadratic`, 15 steps | **Sigma schedule** / **Steps** |
+| `MiniMaxH3ImageToVideo` | prompt, first/last frame | **Prompt** / **First frame** / **Last frame** |
+| `UnifiedResizeImageMask` ("Target Dimension") | 1344x768 | **Target dimension** |
+| `ImageSharpenKJ` | `rcas`, 0.3 | **RCAS sharpening** |
+| `FrameInterpolate` + `film_net_fp16` | multiplier 2 | **FILM frame interpolation** |
+| `CreateVideo` | fps `24 * 2` | 48 fps output |
+| `RTXVideoSuperResolution` | 2x `ULTRA` | **not reproduced** |
+| `PathchSageAttentionKJ` | `sageattn_qk_int8_pv_fp8_cuda++` | cuDNN fused attention |
+
+Two deliberate deviations. `RTXVideoSuperResolution` is NVIDIA's NGX super-resolution, shipped as a driver-level
+Windows/RTX component with no Linux Python path, so the 2x upscale is missing — pick a larger **Target dimension**
+instead of upscaling a small one. And SageAttention's `qk_int8_pv_fp8_cuda++` kernel is not built for this pool's
+sm120 cards, so attention runs cuDNN's fused kernel, which is the fastest available here and is numerically the
+faithful one (SageAttention is a quantized approximation).
+
+One upgrade: the workflow loads `minimax_h3_fl2va_pruned_int8_convrot.safetensors` and a
+`qwen3vl_32b_..._int8_convrot` text encoder because that is what fits a consumer card. This Space runs both
+**unquantized bfloat16**, off `MiniMaxAI/MiniMax-H3`, with the 62 GiB text encoder in a
+[second Space](https://huggingface.co/spaces/multimodalart/qwen3vl-conditioner).
+"""
+        )
+
+    run.click(generate, controls, [video, report], api_name="generate")
+
 
 if __name__ == "__main__":
-    demo.queue().launch(show_error=True, theme=gr.themes.Citrus(), css=CSS, mcp_server=True)
+    demo.launch(show_error=True)

film_net.py

@@ -0,0 +1,270 @@
+"""FILM: Frame Interpolation for Large Motion (ECCV 2022).
+
+Vendored verbatim from ComfyUI (`comfy_extras/frame_interpolation_models/film_net.py`,
+https://github.com/comfyanonymous/ComfyUI, GPL-3.0) apart from the two lines below: ComfyUI's
+`comfy.ops.disable_weight_init` is only `torch.nn` with the parameter initialisers turned into no-ops, and this Space
+loads a checkpoint over every parameter anyway, so plain `torch.nn` is a drop-in.
+
+This is the `FrameInterpolate` half of the PlagueKind workflow, which runs `film_net_fp16.safetensors`
+(`Comfy-Org/frame_interpolation`) at multiplier 2 to take MiniMax-H3's 24 fps output to 48 fps.
+
+Because of this file the Space as a whole is GPL-3.0.
+"""
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+ops = nn
+
+
+class FilmConv2d(nn.Module):
+    """Conv2d with optional LeakyReLU and FILM-style padding."""
+
+    def __init__(self, in_channels, out_channels, size, activation=True, device=None, dtype=None, operations=ops):
+        super().__init__()
+        self.even_pad = not size % 2
+        self.conv = operations.Conv2d(in_channels, out_channels, kernel_size=size, padding=size // 2 if size % 2 else 0, device=device, dtype=dtype)
+        self.activation = nn.LeakyReLU(0.2) if activation else None
+
+    def forward(self, x):
+        if self.even_pad:
+            x = F.pad(x, (0, 1, 0, 1))
+        x = self.conv(x)
+        if self.activation is not None:
+            x = self.activation(x)
+        return x
+
+
+def _warp_core(image, flow, grid_x, grid_y):
+    dtype = image.dtype
+    H, W = flow.shape[2], flow.shape[3]
+    dx = flow[:, 0].float() / (W * 0.5)
+    dy = flow[:, 1].float() / (H * 0.5)
+    grid = torch.stack([grid_x[None, None, :] + dx, grid_y[None, :, None] + dy], dim=3)
+    return F.grid_sample(image.float(), grid, mode="bilinear", padding_mode="border", align_corners=False).to(dtype)
+
+
+def build_image_pyramid(image, pyramid_levels):
+    pyramid = [image]
+    for _ in range(1, pyramid_levels):
+        image = F.avg_pool2d(image, 2, 2)
+        pyramid.append(image)
+    return pyramid
+
+
+def flow_pyramid_synthesis(residual_pyramid):
+    flow = residual_pyramid[-1]
+    flow_pyramid = [flow]
+    for residual_flow in residual_pyramid[:-1][::-1]:
+        flow = F.interpolate(flow, size=residual_flow.shape[2:4], mode="bilinear", scale_factor=None).mul_(2).add_(residual_flow)
+        flow_pyramid.append(flow)
+    flow_pyramid.reverse()
+    return flow_pyramid
+
+
+def multiply_pyramid(pyramid, scalar):
+    return [image * scalar[:, None, None, None] for image in pyramid]
+
+
+def pyramid_warp(feature_pyramid, flow_pyramid, warp_fn):
+    return [warp_fn(features, flow) for features, flow in zip(feature_pyramid, flow_pyramid)]
+
+
+def concatenate_pyramids(pyramid1, pyramid2):
+    return [torch.cat([f1, f2], dim=1) for f1, f2 in zip(pyramid1, pyramid2)]
+
+
+class SubTreeExtractor(nn.Module):
+    def __init__(self, in_channels=3, channels=64, n_layers=4, device=None, dtype=None, operations=ops):
+        super().__init__()
+        convs = []
+        for i in range(n_layers):
+            out_ch = channels << i
+            convs.append(nn.Sequential(
+                FilmConv2d(in_channels, out_ch, 3, device=device, dtype=dtype, operations=operations),
+                FilmConv2d(out_ch, out_ch, 3, device=device, dtype=dtype, operations=operations)))
+            in_channels = out_ch
+        self.convs = nn.ModuleList(convs)
+
+    def forward(self, image, n):
+        head = image
+        pyramid = []
+        for i, layer in enumerate(self.convs):
+            head = layer(head)
+            pyramid.append(head)
+            if i < n - 1:
+                head = F.avg_pool2d(head, 2, 2)
+        return pyramid
+
+
+class FeatureExtractor(nn.Module):
+    def __init__(self, in_channels=3, channels=64, sub_levels=4, device=None, dtype=None, operations=ops):
+        super().__init__()
+        self.extract_sublevels = SubTreeExtractor(in_channels, channels, sub_levels, device=device, dtype=dtype, operations=operations)
+        self.sub_levels = sub_levels
+
+    def forward(self, image_pyramid):
+        sub_pyramids = [self.extract_sublevels(image_pyramid[i], min(len(image_pyramid) - i, self.sub_levels))
+                        for i in range(len(image_pyramid))]
+        feature_pyramid = []
+        for i in range(len(image_pyramid)):
+            features = sub_pyramids[i][0]
+            for j in range(1, self.sub_levels):
+                if j <= i:
+                    features = torch.cat([features, sub_pyramids[i - j][j]], dim=1)
+            feature_pyramid.append(features)
+            # Free sub-pyramids no longer needed by future levels
+            if i >= self.sub_levels - 1:
+                sub_pyramids[i - self.sub_levels + 1] = None
+        return feature_pyramid
+
+
+class FlowEstimator(nn.Module):
+    def __init__(self, in_channels, num_convs, num_filters, device=None, dtype=None, operations=ops):
+        super().__init__()
+        self._convs = nn.ModuleList()
+        for _ in range(num_convs):
+            self._convs.append(FilmConv2d(in_channels, num_filters, 3, device=device, dtype=dtype, operations=operations))
+            in_channels = num_filters
+        self._convs.append(FilmConv2d(in_channels, num_filters // 2, 1, device=device, dtype=dtype, operations=operations))
+        self._convs.append(FilmConv2d(num_filters // 2, 2, 1, activation=False, device=device, dtype=dtype, operations=operations))
+
+    def forward(self, features_a, features_b):
+        net = torch.cat([features_a, features_b], dim=1)
+        for conv in self._convs:
+            net = conv(net)
+        return net
+
+
+class PyramidFlowEstimator(nn.Module):
+    def __init__(self, filters=64, flow_convs=(3, 3, 3, 3), flow_filters=(32, 64, 128, 256), device=None, dtype=None, operations=ops):
+        super().__init__()
+        in_channels = filters << 1
+        predictors = []
+        for i in range(len(flow_convs)):
+            predictors.append(FlowEstimator(in_channels, flow_convs[i], flow_filters[i], device=device, dtype=dtype, operations=operations))
+            in_channels += filters << (i + 2)
+        self._predictor = predictors[-1]
+        self._predictors = nn.ModuleList(predictors[:-1][::-1])
+
+    def forward(self, feature_pyramid_a, feature_pyramid_b, warp_fn):
+        levels = len(feature_pyramid_a)
+        v = self._predictor(feature_pyramid_a[-1], feature_pyramid_b[-1])
+        residuals = [v]
+        # Coarse-to-fine: shared predictor for deep levels, then specialized predictors for fine levels
+        steps = [(i, self._predictor) for i in range(levels - 2, len(self._predictors) - 1, -1)]
+        steps += [(len(self._predictors) - 1 - k, p) for k, p in enumerate(self._predictors)]
+        for i, predictor in steps:
+            v = F.interpolate(v, size=feature_pyramid_a[i].shape[2:4], mode="bilinear").mul_(2)
+            v_residual = predictor(feature_pyramid_a[i], warp_fn(feature_pyramid_b[i], v))
+            residuals.append(v_residual)
+            v = v.add_(v_residual)
+        residuals.reverse()
+        return residuals
+
+
+def _get_fusion_channels(level, filters):
+    # Per direction: multi-scale features + RGB image (3ch) + flow (2ch), doubled for both directions
+    return (sum(filters << i for i in range(level)) + 3 + 2) * 2
+
+
+class Fusion(nn.Module):
+    def __init__(self, n_layers=4, specialized_layers=3, filters=64, device=None, dtype=None, operations=ops):
+        super().__init__()
+        self.output_conv = operations.Conv2d(filters, 3, kernel_size=1, device=device, dtype=dtype)
+        self.convs = nn.ModuleList()
+        in_channels = _get_fusion_channels(n_layers, filters)
+        increase = 0
+        for i in range(n_layers)[::-1]:
+            num_filters = (filters << i) if i < specialized_layers else (filters << specialized_layers)
+            self.convs.append(nn.ModuleList([
+                FilmConv2d(in_channels, num_filters, 2, activation=False, device=device, dtype=dtype, operations=operations),
+                FilmConv2d(in_channels + (increase or num_filters), num_filters, 3, device=device, dtype=dtype, operations=operations),
+                FilmConv2d(num_filters, num_filters, 3, device=device, dtype=dtype, operations=operations)]))
+            in_channels = num_filters
+            increase = _get_fusion_channels(i, filters) - num_filters // 2
+
+    def forward(self, pyramid):
+        net = pyramid[-1]
+        for k, layers in enumerate(self.convs):
+            i = len(self.convs) - 1 - k
+            net = layers[0](F.interpolate(net, size=pyramid[i].shape[2:4], mode="nearest"))
+            net = layers[2](layers[1](torch.cat([pyramid[i], net], dim=1)))
+        return self.output_conv(net)
+
+
+class FILMNet(nn.Module):
+    def __init__(self, pyramid_levels=7, fusion_pyramid_levels=5, specialized_levels=3, sub_levels=4,
+                 filters=64, flow_convs=(3, 3, 3, 3), flow_filters=(32, 64, 128, 256), device=None, dtype=None, operations=ops):
+        super().__init__()
+        self.pyramid_levels = pyramid_levels
+        self.fusion_pyramid_levels = fusion_pyramid_levels
+        self.extract = FeatureExtractor(3, filters, sub_levels, device=device, dtype=dtype, operations=operations)
+        self.predict_flow = PyramidFlowEstimator(filters, flow_convs, flow_filters, device=device, dtype=dtype, operations=operations)
+        self.fuse = Fusion(sub_levels, specialized_levels, filters, device=device, dtype=dtype, operations=operations)
+        self._warp_grids = {}
+
+    def get_dtype(self):
+        return self.extract.extract_sublevels.convs[0][0].conv.weight.dtype
+
+    def memory_used_forward(self, shape, dtype):
+        return 1700 * shape[1] * shape[2] * dtype.itemsize
+
+    def _build_warp_grids(self, H, W, device):
+        """Pre-compute warp grids for all pyramid levels."""
+        if (H, W) in self._warp_grids:
+            return
+        self._warp_grids = {}  # clear old resolution grids to prevent memory leaks
+        for _ in range(self.pyramid_levels):
+            self._warp_grids[(H, W)] = (
+                torch.linspace(-(1 - 1 / W), 1 - 1 / W, W, dtype=torch.float32, device=device),
+                torch.linspace(-(1 - 1 / H), 1 - 1 / H, H, dtype=torch.float32, device=device),
+            )
+            H, W = H // 2, W // 2
+
+    def warp(self, image, flow):
+        grid_x, grid_y = self._warp_grids[(flow.shape[2], flow.shape[3])]
+        return _warp_core(image, flow, grid_x, grid_y)
+
+    def extract_features(self, img):
+        """Extract image and feature pyramids for a single frame. Can be cached across pairs."""
+        image_pyramid = build_image_pyramid(img, self.pyramid_levels)
+        feature_pyramid = self.extract(image_pyramid)
+        return image_pyramid, feature_pyramid
+
+    def forward(self, img0, img1, timestep=0.5, cache=None):
+        # FILM uses a scalar timestep per batch element (spatially-varying timesteps not supported)
+        t = timestep.mean(dim=(1, 2, 3)).item() if isinstance(timestep, torch.Tensor) else timestep
+        return self.forward_multi_timestep(img0, img1, [t], cache=cache)
+
+    def forward_multi_timestep(self, img0, img1, timesteps, cache=None):
+        """Compute flow once, synthesize at multiple timesteps. Expects batch=1 inputs."""
+        self._build_warp_grids(img0.shape[2], img0.shape[3], img0.device)
+
+        image_pyr0, feat_pyr0 = cache["img0"] if cache and "img0" in cache else self.extract_features(img0)
+        image_pyr1, feat_pyr1 = cache["img1"] if cache and "img1" in cache else self.extract_features(img1)
+
+        fwd_flow = flow_pyramid_synthesis(self.predict_flow(feat_pyr0, feat_pyr1, self.warp))[:self.fusion_pyramid_levels]
+        bwd_flow = flow_pyramid_synthesis(self.predict_flow(feat_pyr1, feat_pyr0, self.warp))[:self.fusion_pyramid_levels]
+
+        # Build warp targets and free full pyramids (only first fpl levels needed from here)
+        fpl = self.fusion_pyramid_levels
+        p2w = [concatenate_pyramids(image_pyr0[:fpl], feat_pyr0[:fpl]),
+               concatenate_pyramids(image_pyr1[:fpl], feat_pyr1[:fpl])]
+        del image_pyr0, image_pyr1, feat_pyr0, feat_pyr1
+
+        results = []
+        dt_tensors = torch.tensor(timesteps, device=img0.device, dtype=img0.dtype)
+        for idx in range(len(timesteps)):
+            batch_dt = dt_tensors[idx:idx + 1]
+            bwd_scaled = multiply_pyramid(bwd_flow, batch_dt)
+            fwd_scaled = multiply_pyramid(fwd_flow, 1 - batch_dt)
+            fwd_warped = pyramid_warp(p2w[0], bwd_scaled, self.warp)
+            bwd_warped = pyramid_warp(p2w[1], fwd_scaled, self.warp)
+            aligned = [torch.cat([fw, bw, bf, ff], dim=1)
+                       for fw, bw, bf, ff in zip(fwd_warped, bwd_warped, bwd_scaled, fwd_scaled)]
+            del fwd_warped, bwd_warped, bwd_scaled, fwd_scaled
+            results.append(self.fuse(aligned))
+            del aligned
+        return torch.cat(results, dim=0)

h3_aoti.py

@@ -0,0 +1,307 @@
+"""ZeroGPU AoTI for MiniMax-H3: one compiled `MiniMaxH3TransformerBlock` package, reused by all 50 blocks.
+
+Shared byte-identically by every MiniMax-H3 Space. A Space only calls `maybe_load()`; the rest is the build path.
+"""
+
+from __future__ import annotations
+
+import os
+from pathlib import Path
+
+AOTI = os.environ.get("H3_AOTI", "0") == "1"
+AOTI_REPO = os.environ.get("H3_AOTI_REPO", "multimodalart/minimax-h3-aoti")
+AOTI_REPO_TYPE = os.environ.get("H3_AOTI_REPO_TYPE", "model")
+# A package is valid for exactly one `<width>/torch<X.Y>/sm<cc>/<shape>`, and a mismatched one segfaults rather than
+# raising, so `maybe_load` refuses anything but this key.
+AOTI_KEY = os.environ.get("H3_AOTI_KEY", "bf16/torch2.11/sm120/dynamic")
+# `dynamic` is the sequence dimension: `build_packed_sequence` pads nothing, so `S` moves with the prompt as well as
+# the canvas and a static package would serve one prompt length.
+AOTI_SHAPE = os.environ.get("H3_AOTI_SHAPE", "dynamic")
+AOTI_DURATION = int(os.environ.get("H3_AOTI_DURATION", "1500"))
+
+# Where a step spends its time. `MiniMaxH3TokenRefinerBlock` is also repeated but runs a handful of text rows.
+BLOCK_CONTAINER = "transformer_blocks"
+
+# Height of the AdaLN table baked into the package. `temb` grows from 1 row (step 0, both streams at one noise level)
+# to 2 (from step 1, sigmas diverged), and the block gathers from `3 * rows`, so the row count is part of the compiled
+# shape and is pinned by padding on both sides of the compile. Must match the package's `H3_AOTI_TEMB_ROWS`.
+TEMB_ROWS = int(os.environ.get("H3_AOTI_TEMB_ROWS", "4"))
+
+_LOADED: set[int] = set()
+
+
+def pad_temb(temb, rows: int = TEMB_ROWS):
+    """Grow `temb` to exactly `rows` timestep rows by repeating its last one."""
+    present = temb.shape[0]
+    if present == rows:
+        return temb
+    if present > rows:
+        raise RuntimeError(
+            f"{present} distinct timesteps, but this AoTI package holds at most {rows}. "
+            f"Recompile with H3_AOTI_TEMB_ROWS>={present}."
+        )
+    import torch
+
+    return torch.cat([temb, temb[-1:].expand(rows - present, *temb.shape[1:])], dim=0)
+
+
+def width() -> str:
+    """Which transformer these artifacts belong to: `bf16`, `fp8`, `nvfp4`, ..."""
+    if explicit := os.environ.get("H3_WIDTH"):
+        return explicit.lower()
+    try:
+        import h3_core
+
+        return h3_core.WIDTH
+    except Exception:
+        return "bf16"
+
+
+def artifact_key() -> str | None:
+    """`<width>/torch<X.Y>/sm<cc>/<shape>` of the card this process is on, or `None` when there is no CUDA."""
+    try:
+        import torch
+
+        torch_version = ".".join(torch.__version__.split(".")[:2])
+        major, minor = torch.cuda.get_device_capability()
+    except Exception:
+        return None
+    return f"{width()}/torch{torch_version}/sm{major}{minor}/{AOTI_SHAPE}"
+
+
+def status() -> str:
+    return (
+        f"AoTI **on** · `{AOTI_REPO}` ({AOTI_REPO_TYPE}) · shape `{AOTI_SHAPE}`"
+        if AOTI
+        else "AoTI **off** (`H3_AOTI=1` to load compiled blocks)"
+    )
+
+
+def patch_blocks(transformer, package_dir) -> None:
+    """Point all 50 blocks at the one compiled package, binding each block's own weights on its first call.
+
+    `spaces.aoti_load_from_package_dir` with two changes. Weights are read on the first forward rather than at patch
+    time, because this runs at startup and `Module.to` later rebinds `param.data` to fresh CUDA tensors. And `temb` is
+    padded to the height the package was exported with — see `TEMB_ROWS`.
+    """
+    from spaces.zero.torch.aoti import LazyAOTIModel, _shallow_clone_module
+    from torch._functorch._aot_autograd.subclass_parametrization import unwrap_tensor_subclass_parameters
+
+    # `LazyAOTIModel` binds constants by name and silently keeps what it cannot match, which is a SIGSEGV rather than
+    # an error. The patch resolves anonymous names through the compile side's sidecar and raises if it still cannot.
+    try:
+        from spaces_constant_binding_patch import apply_spaces_constant_binding_patch
+
+        apply_spaces_constant_binding_patch()
+    except ImportError:
+        print("[h3-aoti] spaces_constant_binding_patch.py is missing; an unbindable constant would segfault", flush=True)
+
+    model = LazyAOTIModel(Path(package_dir) / "submodules" / BLOCK_CONTAINER / "package.pt2")
+
+    for block in getattr(transformer, BLOCK_CONTAINER):
+        bound: dict = {}
+
+        def forward(hidden_states, temb, *rest, _block=block, _bound=bound):
+            first = not _bound
+            if first:
+                clone = _shallow_clone_module(_block)
+                unwrap_tensor_subclass_parameters(clone)
+                _bound["weights"] = clone.state_dict()
+            return model(_bound["weights"], first, hidden_states, pad_temb(temb), *rest)
+
+        block.forward = forward
+    print(f"[h3-aoti] {len(getattr(transformer, BLOCK_CONTAINER))} blocks patched (temb padded to {TEMB_ROWS})", flush=True)
+
+
+def maybe_load(transformer) -> None:
+    """Patch the block stack with its compiled package, or leave it eager. Safe to call at **startup**.
+
+    Off unless `H3_AOTI=1`, and anything that does not line up — another card, another torch, no `spaces` AoTI
+    helpers, no published package — falls back to eager with one line rather than raising or segfaulting. Nothing here
+    touches a GPU: the download is CPU work and the `.pt2` is not opened until the first forward.
+    """
+    if not AOTI or id(transformer) in _LOADED:
+        return
+
+    key = artifact_key()
+    if key is None:
+        print("[h3-aoti] no CUDA device visible; running eager", flush=True)
+        return
+    if key != AOTI_KEY:
+        print(f"[h3-aoti] this card wants `{key}`, only `{AOTI_KEY}` is published; running eager", flush=True)
+        return
+
+    try:
+        from huggingface_hub import snapshot_download
+        from spaces.zero.torch.aoti import LazyAOTIModel  # noqa: F401
+    except Exception as error:
+        print(f"[h3-aoti] no AoTI loader here ({type(error).__name__}: {error}); running eager", flush=True)
+        return
+
+    print(f"[h3-aoti] loading {AOTI_REPO}:{key} ...", flush=True)
+    try:
+        local = snapshot_download(repo_id=AOTI_REPO, repo_type=AOTI_REPO_TYPE, allow_patterns=f"{key}/package/*")
+    except Exception as error:
+        print(f"[h3-aoti] {AOTI_REPO}:{key} unreachable ({type(error).__name__}: {error}); running eager", flush=True)
+        return
+    package_dir = Path(local) / key / "package"
+    if not package_dir.is_dir():
+        print(f"[h3-aoti] no package at `{AOTI_REPO}:{key}/package`; running eager", flush=True)
+        return
+
+    patch_blocks(transformer, package_dir)
+    _LOADED.add(id(transformer))
+    print(f"[h3-aoti] compiled blocks in place (temb padded to {TEMB_ROWS} rows)", flush=True)
+
+
+def export_block(pipe, height: int, width: int, num_frames: int, prompt: str):
+    """Capture one block call out of a real request and export it with a dynamic sequence dimension.
+
+    Runs on the GPU, after the transformer has been quantized and moved there: a package compiled for one
+    quantization mode is meaningless for another.
+    """
+    import torch
+    import spaces
+
+    import h3_core as h3
+
+    transformer = h3.transformer_of(pipe)
+    blocks = getattr(transformer, BLOCK_CONTAINER)
+
+    # Keep the widest `temb` over a short real run rather than `spaces.aoti_capture`'s first call, which is the
+    # 1-row one — see `TEMB_ROWS`.
+    original_forward = blocks[0].forward
+    widest = {"args": (), "kwargs": {}, "rows": -1}
+    seen = []
+
+    def recording(*args, **kwargs):
+        rows = int(args[1].shape[0]) if len(args) > 1 and hasattr(args[1], "shape") else -1
+        seen.append(rows)
+        if rows > widest["rows"]:
+            widest.update(args=args, kwargs=kwargs, rows=rows)
+        return original_forward(*args, **kwargs)
+
+    blocks[0].forward = recording
+    try:
+        pipe(
+            prompt=prompt,
+            height=height,
+            width=width,
+            num_frames=num_frames,
+            num_inference_steps=int(os.environ.get("H3_AOTI_CAPTURE_STEPS", "4")),
+            generator=torch.Generator("cpu").manual_seed(42),
+        )
+    finally:
+        blocks[0].forward = original_forward
+    call = type("Captured", (), widest)
+    if not call.args:
+        raise RuntimeError("Nothing was captured — the transformer block was never called.")
+    print(f"[h3-aoti] temb rows seen: {sorted(set(seen))}; exporting with {TEMB_ROWS} (padded)", flush=True)
+
+    # `block(hidden_states, temb, adaln_indices, rotary_emb, attention_mask)`, `attention_mask` being `None` for the
+    # padless sequences these pipelines build. Only the sequence is dynamic: `torch.export` specializes size-1
+    # dimensions unconditionally, so a `Dim` on `temb`'s rows cannot be expressed at all.
+    if AOTI_SHAPE == "dynamic":
+        sequence = torch.export.Dim("sequence", min=2048, max=262144)
+        dynamic_shapes = ({1: sequence}, None, {0: sequence}, ({0: sequence}, {0: sequence}), None)
+        dynamic_shapes = dynamic_shapes[: len(call.args)]
+    else:
+        dynamic_shapes = None
+
+    args = (call.args[0], pad_temb(call.args[1]), *call.args[2:])
+
+    # Export the **live** block, non-strict. A shallow clone under non-strict tracing lifts every weight twice — once
+    # named, once as an anonymous `CONSTANT_TENSOR` aliasing it — and the loader binds by name, so the compiled block
+    # dereferences constants nobody set. The clone is only for flattening tensor-subclass parameters, which inductor's
+    # constant handling cannot wrap back into a `Parameter`, and it needs `strict=True`.
+    from spaces.zero.torch.aoti import _shallow_clone_module
+    from torch._functorch._aot_autograd.subclass_parametrization import unwrap_tensor_subclass_parameters
+
+    subclassed = sorted({type(p).__name__ for p in blocks[0].parameters()} - {"Parameter"})
+    if subclassed:
+        block = _shallow_clone_module(blocks[0])
+        unwrap_tensor_subclass_parameters(block)
+        strict = True
+        print(f"[h3-aoti] tensor-subclass parameters {subclassed}: exporting a flattened clone, strict=True", flush=True)
+    else:
+        block = blocks[0]
+        strict = False
+        print("[h3-aoti] plain parameters: exporting the live block, non-strict", flush=True)
+
+    # `torch.export` only gives a lifted tensor a real FQN when it is a registered parameter or buffer; a plain
+    # attribute becomes an anonymous constant the loader can never match. Only ever on the clone, since this
+    # re-registers attributes and the live block is what the eager path runs.
+    if block is not blocks[0]:
+        try:
+            from spaces_constant_binding_patch import register_loose_tensors
+
+            if loose := register_loose_tensors(block):
+                print(f"[h3-aoti] re-registered {len(loose)} loose tensors as buffers: {loose[:6]}", flush=True)
+        except ImportError:
+            pass
+
+    print(f"[h3-aoti] exporting {type(blocks[0]).__name__}, shapes={AOTI_SHAPE}, strict={strict} ...", flush=True)
+    try:
+        exported = torch.export.export(block, args, call.kwargs or None, dynamic_shapes=dynamic_shapes, strict=strict)
+    except Exception as error:
+        if not strict:
+            raise
+        print(f"[h3-aoti] strict export failed ({type(error).__name__}: {error}); retrying non-strict", flush=True)
+        exported = torch.export.export(block, args, call.kwargs or None, dynamic_shapes=dynamic_shapes)
+
+    anonymous = [
+        spec.target for spec in exported.graph_signature.input_specs if spec.kind.name == "CONSTANT_TENSOR"
+    ]
+    if anonymous:
+        print(
+            f"[h3-aoti] WARNING {len(anonymous)} constants lifted anonymously: {anonymous[:6]}. The loader binds by "
+            f"name, so `compile_and_save` writes the alias sidecar and `patch_blocks` raises rather than segfaulting.",
+            flush=True,
+        )
+    return exported
+
+
+def compile_and_save(exported_program, destination: str | os.PathLike[str]) -> Path:
+    """Inductor-compile the exported block into `<destination>/package/submodules/transformer_blocks/package.pt2`.
+
+    That layout is what `aoti_load_from_package_dir` walks, resolving the submodule name to the transformer's
+    `transformer_blocks` `ModuleList` and patching every block in it with this one package.
+    """
+    import spaces
+
+    package_dir = Path(destination) / "package"
+    print("[h3-aoti] inductor compile (minutes) ...", flush=True)
+    spaces.aoti_compile_and_save(package_dir, exported_program, submodule=BLOCK_CONTAINER)
+
+    # The compiled artifact drops a constant's FQN when the export lifted it anonymously; the `ExportedProgram` still
+    # has the real names, so record the mapping for the loader while it is available.
+    try:
+        from spaces_constant_binding_patch import write_constant_aliases
+
+        if sidecar := write_constant_aliases(package_dir, exported_program, submodule=BLOCK_CONTAINER):
+            print(f"[h3-aoti] constant alias sidecar written: {sidecar.name}", flush=True)
+    except ImportError:
+        pass
+
+    files = sorted(str(path.relative_to(package_dir)) for path in package_dir.rglob("*") if path.is_file())
+    print(f"[h3-aoti] package written: {files}", flush=True)
+    return package_dir
+
+
+def upload(package_dir: str | os.PathLike[str], key: str) -> str:
+    """Push the package under its `<width>/torch<X.Y>/sm<cc>/<shape>` key. CPU work — never inside GPU time."""
+    from huggingface_hub import HfApi
+
+    token = os.environ.get("HF_TOKEN")
+    if not token:
+        raise RuntimeError("`HF_TOKEN` is needed to push the AoTI package.")
+    api = HfApi(token=token)
+    api.create_repo(repo_id=AOTI_REPO, repo_type=AOTI_REPO_TYPE, private=False, exist_ok=True)
+    api.upload_folder(
+        folder_path=str(package_dir),
+        path_in_repo=f"{key}/package",
+        repo_id=AOTI_REPO,
+        repo_type=AOTI_REPO_TYPE,
+        commit_message=f"AoTI package for {key}",
+    )
+    return f"https://huggingface.co/{'datasets/' if AOTI_REPO_TYPE == 'dataset' else ''}{AOTI_REPO}/tree/main/{key}"

h3_local_conditioner.py

@@ -1,174 +0,0 @@
-"""Local, truncated Qwen3-VL conditioner for MiniMax-H3.
-
-The canonical diffusers checkpoint stores all 64 language layers plus the LM head in BF16 (66.7 GB), although H3
-only reads the unnormalized state after layer 50.  ComfyUI's Apache-2.0 conversion removes the unused tail and head,
-keeps the vision tower in BF16, and stores the 50 language layers as NVFP4-AWQ.  This adapter loads that single
-15.7 GB file directly into Transformers' Qwen3-VL architecture and exposes the tiny contract used by diffusers.
-
-No ComfyUI application or server is launched. Preprocessing remains Transformers' canonical Qwen3-VL processor.
-By default the checkpoint's quality-oriented weight-only policy is honored: compact NVFP4-AWQ weights are
-dequantized one layer at a time for BF16 GEMMs. Native W4A4 is available as an aggressive opt-in.
-"""
-
-from __future__ import annotations
-
-import copy
-import os
-from types import SimpleNamespace
-
-import torch
-import torch.nn as nn
-
-from h3_nvfp4 import H3Linear
-
-
-CONDITIONER_REPO = os.environ.get("H3_LOCAL_CONDITIONER_REPO", "Comfy-Org/MiniMax-H3")
-CONDITIONER_FILE = os.environ.get(
-    "H3_LOCAL_CONDITIONER_FILE",
-    "text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors",
-)
-SOURCE_REPO = os.environ.get("H3_MODEL_REPO", "MiniMaxAI/MiniMax-H3")
-LAYERS = 50
-NATIVE_NVFP4 = os.environ.get("H3_CONDITIONER_NATIVE_NVFP4", "0") == "1"
-
-
-class QuantizedEmbedding(nn.Module):
-    """Row-wise INT8 token lookup without dequantizing the 1.56 GB BF16 vocabulary table."""
-
-    def __init__(self, handle, prefix: str):
-        super().__init__()
-        self.register_buffer("weight", handle.get_tensor(f"{prefix}.weight"))
-        self.register_buffer("scale", handle.get_tensor(f"{prefix}.weight_scale").float())
-
-    def forward(self, input_ids: torch.Tensor) -> torch.Tensor:
-        flat = input_ids.reshape(-1)
-        values = self.weight.index_select(0, flat).reshape(*input_ids.shape, self.weight.shape[1])
-        scales = self.scale.index_select(0, flat).reshape(*input_ids.shape, 1)
-        return values.to(torch.bfloat16).mul_(scales.to(torch.bfloat16))
-
-
-class Layer50Backbone(nn.Module):
-    """Avoid retaining 50 intermediate tensors merely to satisfy diffusers' hidden-state indexing API."""
-
-    def __init__(self, core: nn.Module):
-        super().__init__()
-        self.core = core
-
-    def forward(self, *args, **kwargs):
-        kwargs.pop("output_hidden_states", None)
-        kwargs.pop("return_dict", None)
-        kwargs["use_cache"] = False
-        output = self.core(*args, **kwargs)
-        # get_qwen3vl_prompt_embeds asks for hidden_states[50]. The first 50 entries need not be materialized.
-        return SimpleNamespace(hidden_states=(None,) * LAYERS + (output.last_hidden_state,))
-
-
-class LocalH3Conditioner(nn.Module):
-    """The subset of Qwen3VLForConditionalGeneration that MiniMax-H3 actually calls."""
-
-    def __init__(self, core: nn.Module, source_config):
-        super().__init__()
-        public_config = copy.deepcopy(source_config)
-        # Diffusers rejects a nominally 50-layer model because a normal last_hidden_state is post-norm. This adapter
-        # removes the final norm and returns the raw 50th-layer state, so advertise index 50 as available explicitly.
-        public_config.text_config.num_hidden_layers = LAYERS + 1
-        self.config = public_config
-        self.model = Layer50Backbone(core)
-
-    @property
-    def dtype(self) -> torch.dtype:
-        return torch.bfloat16
-
-    @property
-    def device(self) -> torch.device:
-        return self.model.core.visual.patch_embed.proj.weight.device
-
-
-def _target_name(checkpoint_name: str) -> str:
-    if checkpoint_name.startswith("model.layers."):
-        return "language_model.layers." + checkpoint_name.removeprefix("model.layers.")
-    if checkpoint_name.startswith("visual."):
-        return checkpoint_name
-    raise KeyError(checkpoint_name)
-
-
-def _build_core(handle):
-    from accelerate import init_empty_weights
-    from transformers import Qwen3VLConfig
-    from transformers.models.qwen3_vl.modeling_qwen3_vl import Qwen3VLModel
-
-    config = Qwen3VLConfig.from_pretrained(SOURCE_REPO, subfolder="text_encoder")
-    config.text_config.num_hidden_layers = LAYERS
-    config.text_config.use_cache = False
-    config.text_config._attn_implementation = "sdpa"
-    config.vision_config._attn_implementation = "sdpa"
-
-    with init_empty_weights(include_buffers=False):
-        core = Qwen3VLModel(config)
-
-    keys = set(handle.keys())
-    embedding_prefix = "model.embed_tokens"
-    core.language_model.embed_tokens = QuantizedEmbedding(handle, embedding_prefix)
-    consumed = {
-        key for key in keys if key == f"{embedding_prefix}.comfy_quant" or key.startswith(f"{embedding_prefix}.weight")
-    }
-
-    quantized_prefixes = sorted(
-        key.removesuffix(".comfy_quant")
-        for key in keys
-        if key.startswith("model.layers.") and key.endswith(".comfy_quant")
-    )
-    if len(quantized_prefixes) != LAYERS * 7:
-        raise RuntimeError(f"Expected {LAYERS * 7} quantized language linears, found {len(quantized_prefixes)}.")
-
-    for source_prefix in quantized_prefixes:
-        target_prefix = _target_name(source_prefix)
-        parent_name, child_name = target_prefix.rsplit(".", 1)
-        parent = core.get_submodule(parent_name)
-        original = getattr(parent, child_name)
-        linear = H3Linear(original.in_features, original.out_features, bias=original.bias is not None)
-        linear.load(handle, source_prefix)
-        if NATIVE_NVFP4:
-            linear.full_precision_mm = False
-        setattr(parent, child_name, linear)
-        consumed.update(key for key in keys if key.startswith(f"{source_prefix}."))
-
-    # MiniMax-H3 consumes the raw output of layer 49. The released Comfy checkpoint intentionally has no final norm.
-    core.language_model.norm = nn.Identity()
-
-    plain_state = {}
-    for source_name in sorted(keys - consumed):
-        if source_name.startswith("visual.") or source_name.startswith("model.layers."):
-            plain_state[_target_name(source_name)] = handle.get_tensor(source_name)
-            consumed.add(source_name)
-
-    unknown = keys - consumed
-    if unknown:
-        raise RuntimeError(f"Unhandled local-conditioner tensors: {sorted(unknown)[:12]}")
-
-    core.load_state_dict(plain_state, strict=False, assign=True)
-    meta = [name for name, value in core.named_parameters() if value.is_meta]
-    if meta:
-        raise RuntimeError(f"Local conditioner still has uninitialized parameters: {meta[:12]}")
-    core.eval()
-    return core, config
-
-
-def load_local_conditioner():
-    from huggingface_hub import hf_hub_download
-    from safetensors import safe_open
-    from transformers import Qwen3VLProcessor
-
-    path = hf_hub_download(CONDITIONER_REPO, CONDITIONER_FILE)
-    with safe_open(path, framework="pt", device="cpu") as handle:
-        core, config = _build_core(handle)
-
-    processor = Qwen3VLProcessor.from_pretrained(SOURCE_REPO, subfolder="text_encoder")
-    model = LocalH3Conditioner(core, config).eval()
-    print(f"[h3-cond] loaded local layer-50 conditioner {CONDITIONER_REPO}/{CONDITIONER_FILE}", flush=True)
-    return model, processor.tokenizer, processor
-
-
-def status() -> str:
-    compute = "native W4A4" if NATIVE_NVFP4 else "BF16 GEMM"
-    return f"local layer-50 Qwen3-VL NVFP4-AWQ weights / {compute} · `{CONDITIONER_REPO}`"

h3_nvfp4.py

@@ -1,964 +0,0 @@
-"""Blackwell-native MiniMax-H3 transformer for the pruned ComfyUI NVFP4 checkpoint.
-
-The public diffusers checkpoint spends 13.04B of its 33.12B parameters on per-block
-AdaLN projections.  ComfyUI's pruned checkpoint replaces those projections with an
-interpolated 1025-point timestep curve, fuses Q/K/V, and stores the four large linear
-layers in every block as NVFP4.  This adapter keeps diffusers' packed-sequence contract
-so the rest of the split Space (schedulers, VAEs and remote conditioner) stays unchanged.
-
-The kernel/layout conventions follow ComfyUI's Apache-2.0 implementation:
-https://github.com/Comfy-Org/ComfyUI/blob/master/comfy/ldm/minimax/model.py
-"""
-
-from __future__ import annotations
-
-import json
-import math
-import os
-from types import SimpleNamespace
-
-import torch
-import torch.nn as nn
-import torch.nn.functional as F
-import comfy_kitchen as kitchen
-from comfy_kitchen.tensor import QuantizedTensor, TensorCoreNVFP4Layout
-from diffusers.models.attention_dispatch import dispatch_attention_fn
-
-try:
-    import triton
-    import triton.language as tl
-except ImportError:  # PyTorch CUDA wheels include Triton; retain a portable fallback for source inspection/tests.
-    triton = None
-    tl = None
-
-
-NVFP4_REPO = os.environ.get("H3_NVFP4_REPO", "lilcheaty/MiniMax-H3-NVFP4")
-NVFP4_FILE = os.environ.get("H3_NVFP4_FILE", "minimax_h3_fl2va_pruned_nvfp4.safetensors")
-
-HIDDEN = 5376
-HEADS = 56
-HEAD_DIM = 128
-FFN = 14336
-TEXT_DIM = 5120
-TIME_DIM = 8
-VIDEO_DIM = 24 * 1 * 2 * 2
-AUDIO_DIM = 32
-LAYERS = 50
-REFINER_LAYERS = 2
-EPS = 1e-5
-
-# EasyCache is the conservative profile.  The Ultra Fast profile uses a bounded linear residual forecast:
-# three exact warmup evaluations, at most three forecasts in a row, and two exact tail evaluations.  Unlike blind
-# output reuse, forecasting follows the local denoising trajectory while making the amount of saved work predictable.
-EASYCACHE_THRESHOLD = max(0.0, float(os.environ.get("H3_EASYCACHE_THRESHOLD", "0.10")))
-EASYCACHE_START = min(1.0, max(0.0, float(os.environ.get("H3_EASYCACHE_START", "0.15"))))
-EASYCACHE_END = min(1.0, max(EASYCACHE_START, float(os.environ.get("H3_EASYCACHE_END", "0.95"))))
-EASYCACHE_SUBSAMPLE = max(1, int(os.environ.get("H3_EASYCACHE_SUBSAMPLE", "8")))
-FIRST_BLOCK_THRESHOLD = max(0.0, float(os.environ.get("H3_FIRST_BLOCK_THRESHOLD", "0.08")))
-FIRST_BLOCK_DENSE_START = max(1, int(os.environ.get("H3_FIRST_BLOCK_DENSE_START", "3")))
-FIRST_BLOCK_DENSE_END = max(1, int(os.environ.get("H3_FIRST_BLOCK_DENSE_END", "2")))
-FORECAST_BLEND = min(1.0, max(0.0, float(os.environ.get("H3_FORECAST_BLEND", "0.65"))))
-FUSED_ADALN = os.environ.get("H3_FUSED_ADALN", "0") == "1" and triton is not None
-SOL_ATTN = os.environ.get("H3_SOL_ATTN", "1") == "1"
-SOL_ATTN_TAU = float(os.environ.get("H3_SOL_ATTN_TAU", "1.0"))
-SOL_ATTN_DENSE_STEPS = max(0, int(os.environ.get("H3_SOL_ATTN_DENSE_STEPS", "10")))
-SOL_ATTN_DENSE_LAYERS = max(0, int(os.environ.get("H3_SOL_ATTN_DENSE_LAYERS", "2")))
-SOL_ATTN_MIN_TOKENS = max(0, int(os.environ.get("H3_SOL_ATTN_MIN_TOKENS", "8192")))
-
-
-if triton is not None:
-
-    @triton.jit
-    def _adaln_modulate_kernel(
-        x, shift, scale, row_ids, elements: tl.constexpr, hidden: tl.constexpr, modulation_stride: tl.constexpr
-    ):
-        offsets = tl.program_id(0) * 256 + tl.arange(0, 256)
-        mask = offsets < elements
-        columns = offsets % hidden
-        rows = offsets // hidden
-        modulation_rows = tl.load(row_ids + rows, mask=mask, other=0)
-        modulation_offsets = modulation_rows * modulation_stride + columns
-        values = tl.load(x + offsets, mask=mask)
-        shifts = tl.load(shift + modulation_offsets, mask=mask)
-        scales = tl.load(scale + modulation_offsets, mask=mask)
-        tl.store(x + offsets, values * (1.0 + scales) + shifts, mask=mask)
-
-    @triton.jit
-    def _adaln_gate_kernel(
-        x, update, gate, row_ids, elements: tl.constexpr, hidden: tl.constexpr, modulation_stride: tl.constexpr
-    ):
-        offsets = tl.program_id(0) * 256 + tl.arange(0, 256)
-        mask = offsets < elements
-        columns = offsets % hidden
-        rows = offsets // hidden
-        modulation_rows = tl.load(row_ids + rows, mask=mask, other=0)
-        gates = tl.load(gate + modulation_rows * modulation_stride + columns, mask=mask)
-        values = tl.load(x + offsets, mask=mask)
-        updates = tl.load(update + offsets, mask=mask)
-        tl.store(x + offsets, values + updates * gates, mask=mask)
-
-
-class H3StepCache:
-    """ComfyUI EasyCache-style adaptive reuse of a complete H3 denoising result.
-
-    This caches the model residual, not the generated video.  A request with a new prompt, seed, canvas or keyframe
-    starts from an empty cache.  Decisions use a sparse sample of generated video latent rows, while the reused
-    residual contains every video and audio row so their joint denoising trajectory stays coupled.
-    """
-
-    def __init__(self):
-        self.total_steps = 0
-        self.step = 0
-        self.skipped = 0
-        self.profile = "balanced"
-        self.consecutive_skips = 0
-        self.last_actual_step = None
-        self.previous_input = None
-        self.previous_output = None
-        self.previous_output_norm = None
-        self.relative_rate = None
-        self.accumulated_change = None
-        self.video_residual = None
-        self.audio_residual = None
-        self.video_residual_slope = None
-        self.audio_residual_slope = None
-        self.pending_input = None
-        self.pending_input_change = None
-        self.pending_track = False
-        self.head_residual = None
-        self.tail_residual = None
-        self.first_block_output = None
-
-    def begin(self, total_steps: int | None, profile: str = "balanced") -> None:
-        self.__init__()
-        self.total_steps = max(0, int(total_steps or 0))
-        self.profile = str(profile or "balanced").lower()
-
-    @property
-    def enabled(self) -> bool:
-        return self.profile != "exact" and self.total_steps > 2
-
-    def _forecast(self, video_input, audio_input):
-        distance = max(1, self.step - int(self.last_actual_step or 0))
-        video_residual = self.video_residual
-        audio_residual = self.audio_residual
-        if self.video_residual_slope is not None:
-            video_residual = video_residual + self.video_residual_slope * (distance * FORECAST_BLEND)
-            audio_residual = audio_residual + self.audio_residual_slope * (distance * FORECAST_BLEND)
-        self.skipped += 1
-        self.consecutive_skips += 1
-        self.step += 1
-        return video_input + video_residual, audio_input + audio_residual
-
-    def try_reuse(self, video_input, audio_input, condition_rows: int):
-        self.pending_input = None
-        self.pending_input_change = None
-        self.pending_track = False
-        if not self.enabled:
-            return None
-
-        # Balanced uses NVIDIA's H3 FirstBlockCache below, after block 0 has produced a high-signal residual.
-        # Only the deliberately aggressive Ultra profile forecasts a whole transformer call before block 0.
-        if not self.profile.startswith("ultra"):
-            return None
-
-        # Ultra Fast is deliberately bounded: no more than three forecasts can separate exact transformer calls, and
-        # the high-noise warmup plus low-noise tail remain exact.  At the default 16 steps this executes 7 full DiT
-        # evaluations instead of 16 while still sampling the original 16-step scheduler trajectory.
-        if self.profile.startswith("ultra"):
-            can_forecast = (
-                self.step >= 3
-                and self.step < self.total_steps - 2
-                and self.consecutive_skips < 3
-                and self.last_actual_step is not None
-                and self.video_residual is not None
-                and self.audio_residual is not None
-                and self.video_residual.shape == video_input.shape
-                and self.audio_residual.shape == audio_input.shape
-            )
-            if can_forecast:
-                return self._forecast(video_input, audio_input)
-            return None
-
-        if EASYCACHE_THRESHOLD <= 0.0:
-            return None
-
-        end_step = math.floor(self.total_steps * EASYCACHE_END)
-        if self.step >= end_step:
-            return None
-
-        # Condition latents are static. Excluding them makes the change estimate reflect the generated trajectory.
-        sampled_input = video_input[0, condition_rows::EASYCACHE_SUBSAMPLE].detach().float()
-        self.pending_input = sampled_input
-        self.pending_track = True
-        if self.previous_input is not None:
-            self.pending_input_change = (sampled_input - self.previous_input).abs().mean()
-
-        start_step = math.ceil(self.total_steps * EASYCACHE_START)
-        can_reuse = (
-            self.step >= start_step
-            and self.pending_input_change is not None
-            and self.relative_rate is not None
-            and self.previous_output_norm is not None
-            and self.video_residual is not None
-            and self.audio_residual is not None
-            and self.video_residual.shape == video_input.shape
-            and self.audio_residual.shape == audio_input.shape
-        )
-        if not can_reuse:
-            return None
-
-        estimated_change = self.relative_rate * self.pending_input_change
-        estimated_change = estimated_change / self.previous_output_norm.clamp_min(1e-6)
-        accumulated = estimated_change if self.accumulated_change is None else self.accumulated_change + estimated_change
-        if bool((accumulated < EASYCACHE_THRESHOLD).item()):
-            self.accumulated_change = accumulated
-            self.skipped += 1
-            self.step += 1
-            return video_input + self.video_residual, audio_input + self.audio_residual
-        return None
-
-    def first_block_decision(self, block_input: torch.Tensor, block_output: torch.Tensor) -> bool:
-        """Return True when blocks 1..49 can reuse their previous joint residual.
-
-        This is the single-GPU equivalent of NVIDIA Sol-Engine's H3 FirstBlockCache at threshold 0.08. The first
-        block is always evaluated. Its normalized residual change is a much stronger predictor than raw latent
-        motion, while the cached tail residual still covers the complete text/video/audio packed sequence.
-        """
-        if not self.enabled or self.profile.startswith("ultra"):
-            return False
-        if FIRST_BLOCK_THRESHOLD <= 0.0:
-            self.first_block_output = block_output.detach().clone()
-            return False
-
-        keep_dense = self.step < FIRST_BLOCK_DENSE_START or self.step >= self.total_steps - FIRST_BLOCK_DENSE_END
-        residual = block_output - block_input
-        reusable = (
-            not keep_dense
-            and self.head_residual is not None
-            and self.tail_residual is not None
-            and self.tail_residual.shape == block_output.shape
-        )
-        should_reuse = False
-        if reusable:
-            difference = (residual - self.head_residual).abs().mean()
-            reference = self.head_residual.abs().mean().clamp_min(1e-8)
-            should_reuse = bool(((difference / reference) <= FIRST_BLOCK_THRESHOLD).item())
-
-        if should_reuse:
-            self.skipped += 1
-            self.consecutive_skips += 1
-            self.step += 1
-            return True
-
-        # This engine's residual/gate operations update `packed` in place. Preserve the head output before later
-        # blocks mutate the same storage; diffusers' reference blocks are out-of-place and do not need this clone.
-        self.first_block_output = block_output.detach().clone()
-        self.head_residual = residual.detach()
-        return False
-
-    def update_first_block_tail(self, final_block_output: torch.Tensor) -> None:
-        if self.first_block_output is None:
-            return
-        self.tail_residual = (final_block_output - self.first_block_output).detach()
-        self.last_actual_step = self.step
-        self.consecutive_skips = 0
-        self.step += 1
-        self.first_block_output = None
-
-    def update(self, video_input, audio_input, video_output, audio_output, condition_rows: int) -> None:
-        # Balanced's clock and state are updated at the block-stack boundary by FirstBlockCache.
-        if self.enabled and not self.profile.startswith("ultra"):
-            return
-        if self.pending_track:
-            sampled_output = video_output[0, condition_rows::EASYCACHE_SUBSAMPLE].detach().float()
-            if self.previous_output is not None and self.pending_input_change is not None:
-                output_change = (sampled_output - self.previous_output).abs().mean()
-                self.relative_rate = output_change / self.pending_input_change.clamp_min(1e-6)
-            self.previous_input = self.pending_input.clone()
-            self.previous_output = sampled_output.clone()
-            self.previous_output_norm = sampled_output.abs().mean()
-            if not self.profile.startswith("ultra"):
-                self.video_residual = (video_output - video_input).detach()
-                self.audio_residual = (audio_output - audio_input).detach()
-            self.accumulated_change = None
-        if self.profile.startswith("ultra"):
-            new_video_residual = (video_output - video_input).detach()
-            new_audio_residual = (audio_output - audio_input).detach()
-            if self.video_residual is not None and self.last_actual_step is not None:
-                gap = max(1, self.step - self.last_actual_step)
-                self.video_residual_slope = (new_video_residual - self.video_residual) / gap
-                self.audio_residual_slope = (new_audio_residual - self.audio_residual) / gap
-            self.video_residual = new_video_residual
-            self.audio_residual = new_audio_residual
-        self.last_actual_step = self.step
-        self.consecutive_skips = 0
-        self.step += 1
-        self.pending_input = None
-        self.pending_input_change = None
-        self.pending_track = False
-
-    def finish(self) -> dict:
-        stats = {
-            "steps": self.step,
-            "computed": max(0, self.step - self.skipped),
-            "forecasted": self.skipped,
-            "profile": self.profile,
-        }
-        if self.enabled and self.step:
-            computed = max(1, self.step - self.skipped)
-            print(
-                f"[h3-nvfp4] adaptive step cache skipped {self.skipped}/{self.step} transformer evaluations "
-                f"({self.step / computed:.2f}x denoiser-work reduction)",
-                flush=True,
-            )
-        self.begin(None)
-        return stats
-
-
-class H3SolAttention:
-    """NVIDIA Sol-Attn policy adapted to H3's single-GPU packed attention.
-
-    The packed prefix (text, conditioning video and generated audio) remains an exact KV sink and its query rows are
-    recomputed densely. Only target-video query/key interactions become sparse, after ten dense denoising steps and
-    outside the first two transformer blocks. Any unavailable/JIT-failing backend falls back to cuDNN for the request.
-    """
-
-    def __init__(self):
-        self.enabled = SOL_ATTN
-        self.step = 0
-        self.video_start = 0
-        self.sparse_calls = 0
-        self.dense_calls = 0
-        self.failure = None
-
-    def begin(self):
-        self.step = 0
-        self.video_start = 0
-        self.sparse_calls = 0
-        self.dense_calls = 0
-        self.failure = None
-
-    def observe(self, video_indices: torch.Tensor, sequence: int, step: int) -> None:
-        self.step = int(step)
-        if not self.video_start:
-            deltas = video_indices[1:] - video_indices[:-1]
-            breaks = (deltas != 1).nonzero().flatten()
-            start = int(breaks[-1]) + 1 if len(breaks) else 0
-            self.video_start = int(video_indices[start]) if video_indices.numel() else sequence
-
-    def __call__(self, query, key, value, layer: int):
-        tokens = int(query.shape[1])
-        if (
-            not self.enabled
-            or self.failure is not None
-            or self.step < SOL_ATTN_DENSE_STEPS
-            or layer < SOL_ATTN_DENSE_LAYERS
-            or tokens < SOL_ATTN_MIN_TOKENS
-            or not 0 < self.video_start < tokens
-        ):
-            self.dense_calls += 1
-            return None
-        try:
-            from sol_attn import sol_attn
-
-            q, k, v = (tensor.contiguous() for tensor in (query, key, value))
-            attended = sol_attn(
-                q,
-                k,
-                v,
-                tau=SOL_ATTN_TAU,
-                thresh_type="diag",
-                kv_splits=1,
-                sink_start=0,
-                sink_tokens=self.video_start,
-            )
-            # An exact KV sink does not make the prefix's own queries dense. H3 jointly generates audio in that
-            # prefix, so reproduce those rows with exact attention as NVIDIA's H3 integration does.
-            prefix = self.video_start
-            dense_prefix = F.scaled_dot_product_attention(
-                q[:, :prefix].transpose(1, 2),
-                k.transpose(1, 2),
-                v.transpose(1, 2),
-                dropout_p=0.0,
-                is_causal=False,
-            ).transpose(1, 2)
-            attended[:, :prefix] = dense_prefix
-            self.sparse_calls += 1
-            return attended
-        except Exception as error:
-            self.failure = f"{type(error).__name__}: {error}"
-            print(f"[h3-sol-attn] falling back to dense attention: {self.failure}", flush=True)
-            self.dense_calls += 1
-            return None
-
-
-def _quant_config(handle, prefix: str) -> dict | None:
-    key = f"{prefix}.comfy_quant"
-    if key not in handle.keys():
-        return None
-    return json.loads(handle.get_tensor(key).numpy().tobytes())
-
-
-class H3Linear(nn.Module):
-    """A plain or comfy-kitchen NVFP4 linear, selected by checkpoint metadata."""
-
-    def __init__(
-        self,
-        in_features: int,
-        out_features: int,
-        bias: bool = False,
-        compute_dtype: torch.dtype | None = None,
-    ):
-        super().__init__()
-        self.in_features = in_features
-        self.out_features = out_features
-        self.compute_dtype = compute_dtype
-        self.register_parameter("weight", None)
-        self.register_parameter("bias", None)
-        self.register_buffer("input_scale", None)
-        self.register_buffer("pre_quant_scale", None)
-        self.quantized = False
-        self.full_precision_mm = False
-
-    def load(self, handle, prefix: str) -> None:
-        config = _quant_config(handle, prefix)
-        weight = handle.get_tensor(f"{prefix}.weight")
-
-        if config is None:
-            self.weight = nn.Parameter(
-                weight if self.compute_dtype is None else weight.to(self.compute_dtype), requires_grad=False
-            )
-        elif config.get("format") == "nvfp4":
-            block_scale = handle.get_tensor(f"{prefix}.weight_scale")
-            if block_scale.dtype == torch.uint8:
-                block_scale = block_scale.view(torch.float8_e4m3fn)
-            tensor_scale = handle.get_tensor(f"{prefix}.weight_scale_2").float()
-            params = TensorCoreNVFP4Layout.Params(
-                scale=tensor_scale,
-                block_scale=block_scale,
-                orig_dtype=torch.bfloat16,
-                orig_shape=(self.out_features, self.in_features),
-            )
-            quantized = QuantizedTensor(weight.to(torch.uint8), "TensorCoreNVFP4Layout", params)
-            self.weight = nn.Parameter(quantized, requires_grad=False)
-            self.quantized = True
-            self.full_precision_mm = bool(config.get("full_precision_matrix_mult", False))
-            for name in ("input_scale", "pre_quant_scale"):
-                key = f"{prefix}.{name}"
-                if key in handle.keys():
-                    setattr(self, name, handle.get_tensor(key))
-        else:
-            raise ValueError(f"Unsupported quantization on {prefix}: {config}")
-
-        bias_key = f"{prefix}.bias"
-        if bias_key in handle.keys():
-            bias = handle.get_tensor(bias_key)
-            self.bias = nn.Parameter(
-                bias if self.compute_dtype is None else bias.to(self.compute_dtype), requires_grad=False
-            )
-
-    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
-        if self.pre_quant_scale is not None:
-            hidden_states = hidden_states * self.pre_quant_scale.to(
-                device=hidden_states.device, dtype=hidden_states.dtype
-            )
-        if not self.quantized:
-            hidden_states = hidden_states.to(self.weight.dtype)
-            return F.linear(
-                hidden_states,
-                self.weight,
-                self.bias,
-            )
-
-        if self.full_precision_mm:
-            # Some AWQ checkpoints use NVFP4 as a compact weight format but deliberately retain BF16 activations and
-            # GEMMs. Dequantization is layer-local, so residency stays compact without adding activation error.
-            weight = self.weight.dequantize().to(hidden_states.dtype)
-            return F.linear(hidden_states, weight, None if self.bias is None else self.bias.to(hidden_states.dtype))
-
-        shape = hidden_states.shape
-        flat = hidden_states.reshape(-1, shape[-1])
-        scale = None if self.input_scale is None else self.input_scale.to(flat.device)
-        quantized_input = QuantizedTensor.from_float(flat, "TensorCoreNVFP4Layout", scale=scale)
-        output = F.linear(
-            quantized_input,
-            self.weight,
-            None if self.bias is None else self.bias.to(hidden_states.dtype),
-        )
-        return output.reshape(*shape[:-1], self.out_features)
-
-
-class H3RMSNorm(nn.Module):
-    def __init__(self, width: int, eps: float = EPS):
-        super().__init__()
-        self.width = width
-        self.eps = eps
-        self.register_parameter("weight", None)
-
-    def load(self, handle, prefix: str) -> None:
-        self.weight = nn.Parameter(handle.get_tensor(f"{prefix}.weight"), requires_grad=False)
-
-    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
-        return F.rms_norm(
-            hidden_states,
-            (self.width,),
-            self.weight,
-            self.eps,
-        )
-
-
-class H3Attention(nn.Module):
-    def __init__(self):
-        super().__init__()
-        self.qkv_proj = H3Linear(HIDDEN, 3 * HEADS * HEAD_DIM)
-        self.q_norm = H3RMSNorm(HEAD_DIM)
-        self.k_norm = H3RMSNorm(HEAD_DIM)
-        self.out_proj = H3Linear(HEADS * HEAD_DIM, HIDDEN)
-
-    def load(self, handle, prefix: str) -> None:
-        self.qkv_proj.load(handle, f"{prefix}.qkv_proj")
-        self.q_norm.load(handle, f"{prefix}.q_norm")
-        self.k_norm.load(handle, f"{prefix}.k_norm")
-        self.out_proj.load(handle, f"{prefix}.out_proj")
-
-    def forward(self, hidden_states, rope_table, backend: str, sparse=None, layer: int = -1):
-        sequence = hidden_states.shape[0]
-        qkv = self.qkv_proj(hidden_states)
-        query, key, value = qkv.split(HEADS * HEAD_DIM, dim=-1)
-        query = query.view(1, sequence, HEADS, HEAD_DIM)
-        key = key.view(1, sequence, HEADS, HEAD_DIM)
-        value = value.view(1, sequence, HEADS, HEAD_DIM)
-
-        # One in-place kernel replaces Q RMSNorm, K RMSNorm and both partial RoPE applications.
-        kitchen.rms_rope_split_half_(
-            query,
-            key,
-            rope_table,
-            self.q_norm.weight,
-            self.k_norm.weight,
-            epsilon=self.q_norm.eps,
-            rot_dim=rope_table.shape[-3] * 2,
-        )
-        attended = sparse(query, key, value, layer) if sparse is not None else None
-        if attended is None:
-            attended = dispatch_attention_fn(
-                query,
-                key,
-                value,
-                attn_mask=None,
-                dropout_p=0.0,
-                is_causal=False,
-                backend=backend,
-            )
-        return self.out_proj(attended.reshape(sequence, HEADS * HEAD_DIM))
-
-
-class H3MLP(nn.Module):
-    def __init__(self):
-        super().__init__()
-        self.fc1 = H3Linear(HIDDEN, 2 * FFN)
-        self.fc2 = H3Linear(FFN, HIDDEN)
-
-    def load(self, handle, prefix: str) -> None:
-        self.fc1.load(handle, f"{prefix}.fc1")
-        self.fc2.load(handle, f"{prefix}.fc2")
-
-    def forward(self, hidden_states):
-        gate, up = self.fc1(hidden_states).chunk(2, dim=-1)
-        return self.fc2(F.silu(gate).mul_(up))
-
-
-class H3RefinerBlock(nn.Module):
-    def __init__(self):
-        super().__init__()
-        self.norm1 = H3RMSNorm(HIDDEN)
-        self.attn = H3Attention()
-        self.norm2 = H3RMSNorm(HIDDEN)
-        self.mlp = H3MLP()
-
-    def load(self, handle, prefix: str) -> None:
-        self.norm1.load(handle, f"{prefix}.norm1")
-        self.attn.load(handle, f"{prefix}.attn")
-        self.norm2.load(handle, f"{prefix}.norm2")
-        self.mlp.load(handle, f"{prefix}.mlp")
-
-
-class H3AdaLN(nn.Module):
-    def __init__(self, expand: int, modalities: int):
-        super().__init__()
-        self.expand = expand
-        self.modalities = modalities
-        # Curve checkpoints deliberately evaluate interpolation and modulation projection in FP32. Expanding the
-        # checkpoint's tiny FP16 [*, 8] matrices once at load avoids 51 request-step casts.
-        self.linear = H3Linear(
-            TIME_DIM, expand * HIDDEN * modalities, bias=True, compute_dtype=torch.float32
-        )
-
-    def load(self, handle, prefix: str) -> None:
-        self.linear.load(handle, f"{prefix}.linear")
-
-    def forward(self, time_embedding, output_dtype=None):
-        projected = self.linear(time_embedding)
-        if output_dtype is not None:
-            # One contiguous conversion is numerically identical to converting the six chunk views independently,
-            # and removes five CUDA launches from every one of the 50 blocks.
-            projected = projected.to(output_dtype)
-        projected = projected.view(-1, self.expand * HIDDEN)
-        return projected.chunk(self.expand, dim=-1)
-
-
-class H3Block(nn.Module):
-    def __init__(self):
-        super().__init__()
-        self.norm1 = H3RMSNorm(HIDDEN)
-        self.attn = H3Attention()
-        self.norm2 = H3RMSNorm(HIDDEN)
-        self.mlp = H3MLP()
-        self.adaln_proj = H3AdaLN(6, 3)
-
-    def load(self, handle, prefix: str) -> None:
-        self.norm1.load(handle, f"{prefix}.norm1")
-        self.attn.load(handle, f"{prefix}.attn")
-        self.norm2.load(handle, f"{prefix}.norm2")
-        self.mlp.load(handle, f"{prefix}.mlp")
-        self.adaln_proj.load(handle, f"{prefix}.adaln_proj")
-
-
-class H3FinalLayer(nn.Module):
-    def __init__(self):
-        super().__init__()
-        self.norm = H3RMSNorm(HIDDEN)
-        self.adaln_proj = H3AdaLN(2, 1)
-        self.video_out = H3Linear(HIDDEN, VIDEO_DIM, bias=True, compute_dtype=torch.float32)
-        self.audio_out = H3Linear(HIDDEN, AUDIO_DIM, bias=True, compute_dtype=torch.float32)
-
-    def load(self, handle, prefix: str) -> None:
-        self.norm.load(handle, f"{prefix}.norm")
-        self.adaln_proj.load(handle, f"{prefix}.adaln_proj")
-        self.video_out.load(handle, f"{prefix}.video_out")
-        self.audio_out.load(handle, f"{prefix}.audio_out")
-
-
-class H3NVFP4Transformer(nn.Module):
-    """Diffusers-compatible H3 transformer backed by fused comfy-kitchen NVFP4 kernels."""
-
-    def __init__(self):
-        super().__init__()
-        # The modular pipeline reads these values through the diffusers component config rather than inspecting the
-        # module itself. Keep the public transformer contract even though this lean adapter is not a ConfigMixin.
-        self.config = SimpleNamespace(
-            patch_size=(1, 2, 2),
-            in_channels=24,
-            audio_in_channels=AUDIO_DIM,
-            text_dim=TEXT_DIM,
-        )
-        self.video_patch_proj = H3Linear(VIDEO_DIM, HIDDEN, bias=True, compute_dtype=torch.float32)
-        self.audio_patch_proj = H3Linear(AUDIO_DIM, HIDDEN, bias=True, compute_dtype=torch.float32)
-        self.condition_proj = H3Linear(TEXT_DIM, HIDDEN, bias=True)
-        self.token_refiner = nn.ModuleList([H3RefinerBlock() for _ in range(REFINER_LAYERS)])
-        self.token_refiner_norm = H3RMSNorm(HIDDEN)
-        self.blocks = nn.ModuleList([H3Block() for _ in range(LAYERS)])
-        self.final_layer = H3FinalLayer()
-        self.register_buffer("adaln_t_table", None)
-        self.register_buffer("rope_inv_freq", None)
-        self.attention_backend = "_native_cudnn"
-        self._text_cache = None
-        self._rope_cache = None
-        self._segment_cache = None
-        self._condition_video_rows = None
-        self._condition_video_embedding = None
-        self._output_indices = None
-        self._generated_rows = None
-        self._step_cache = H3StepCache()
-        self._sol_attention = H3SolAttention()
-
-    @property
-    def dtype(self) -> torch.dtype:
-        """Match ModelMixin's placement contract used by ModularPipeline.to()."""
-        return self.condition_proj.weight.dtype
-
-    @property
-    def device(self) -> torch.device:
-        return self.adaln_t_table.device
-
-    def load(self, path: str) -> None:
-        from safetensors import safe_open
-
-        with safe_open(path, framework="pt", device="cpu") as handle:
-            self.video_patch_proj.load(handle, "video_patch_proj")
-            self.audio_patch_proj.load(handle, "audio_patch_proj")
-            self.condition_proj.load(handle, "condition_proj")
-            for index, block in enumerate(self.token_refiner):
-                block.load(handle, f"token_refiner.blocks.{index}")
-            self.token_refiner_norm.load(handle, "token_refiner.final_norm")
-            for index, block in enumerate(self.blocks):
-                block.load(handle, f"blocks.{index}")
-            self.final_layer.load(handle, "final_layer")
-            self.adaln_t_table = handle.get_tensor("adaln_t_table")
-            self.rope_inv_freq = handle.get_tensor("rope.inv_freq")
-        # Every loaded tensor is already a frozen Parameter (or a buffer). Avoid mutating the quantized tensor
-        # subclass through a redundant requires_grad_ dispatch.
-        self.eval()
-
-    def set_attention_backend(self, backend: str) -> None:
-        self.attention_backend = backend
-
-    def begin_request(self, total_steps: int | None = None, profile: str = "balanced") -> None:
-        self._text_cache = None
-        self._rope_cache = None
-        self._segment_cache = None
-        self._condition_video_rows = None
-        self._condition_video_embedding = None
-        self._output_indices = None
-        self._generated_rows = None
-        self._step_cache.begin(total_steps, profile)
-        self._sol_attention.begin()
-
-    def end_request(self) -> dict:
-        stats = self._step_cache.finish()
-        stats["sol_sparse_calls"] = self._sol_attention.sparse_calls
-        stats["sol_dense_calls"] = self._sol_attention.dense_calls
-        stats["sol_failure"] = self._sol_attention.failure
-        self._text_cache = None
-        self._rope_cache = None
-        self._segment_cache = None
-        self._condition_video_rows = None
-        self._condition_video_embedding = None
-        self._output_indices = None
-        self._generated_rows = None
-        return stats
-
-    def _refine_text(self, text_states: torch.Tensor) -> torch.Tensor:
-        key = (text_states.data_ptr(), tuple(text_states.shape), text_states.device)
-        if self._text_cache is not None and self._text_cache[0] == key:
-            return self._text_cache[1]
-        hidden = self.condition_proj(text_states)
-        # Text is tiny compared with the video sequence; use the same fused QKV path with an identity RoPE omitted.
-        for block in self.token_refiner:
-            residual = hidden
-            normalized = block.norm1(hidden)
-            qkv = block.attn.qkv_proj(normalized)
-            query, key_states, value = qkv.split(HEADS * HEAD_DIM, dim=-1)
-            query = block.attn.q_norm(query.view(1, -1, HEADS, HEAD_DIM))
-            key_states = block.attn.k_norm(key_states.view(1, -1, HEADS, HEAD_DIM))
-            value = value.view(1, -1, HEADS, HEAD_DIM)
-            attended = dispatch_attention_fn(
-                query,
-                key_states,
-                value,
-                attn_mask=None,
-                dropout_p=0.0,
-                is_causal=False,
-                backend=self.attention_backend,
-            ).reshape(-1, HEADS * HEAD_DIM)
-            hidden = residual + block.attn.out_proj(attended)
-            hidden = hidden + block.mlp(block.norm2(hidden))
-        hidden = self.token_refiner_norm(hidden)
-        self._text_cache = (key, hidden)
-        return hidden
-
-    def _rope(self, position_ids: torch.Tensor, dtype: torch.dtype) -> torch.Tensor:
-        key = (position_ids.data_ptr(), tuple(position_ids.shape), position_ids.device, dtype)
-        if self._rope_cache is not None and self._rope_cache[0] == key:
-            return self._rope_cache[1]
-        positions = position_ids.to(torch.float32)
-        frequencies = positions.unsqueeze(-1) * self.rope_inv_freq.to(position_ids.device).view(1, 1, -1)
-        temporal, height, width = frequencies.unbind(dim=1)
-        angles = torch.cat((temporal, height, width), dim=-1)
-        cosine, sine = angles.cos(), angles.sin()
-        table = torch.stack((cosine, -sine, sine, cosine), dim=-1)
-        table = table.reshape(1, position_ids.shape[0], 1, angles.shape[-1], 2, 2).to(dtype)
-        self._rope_cache = (key, table)
-        return table
-
-    def _time_embedding(self, timestep: torch.Tensor) -> torch.Tensor:
-        table = self.adaln_t_table.to(timestep.device)
-        position = timestep.float().clamp(0.0, 1.0) * (table.shape[0] - 1)
-        lower = position.floor().long().clamp(max=table.shape[0] - 2)
-        return torch.lerp(table[lower], table[lower + 1], (position - lower).unsqueeze(1))
-
-    def _segments(self, indices: torch.Tensor):
-        if self._segment_cache is None:
-            host = indices.detach().cpu()
-            changes = (host[1:] != host[:-1]).nonzero().flatten().add(1).tolist()
-            bounds = [0, *changes, len(host)]
-            # Python row ids avoid indexing modulation tensors with CUDA scalar tensors in every block.
-            self._segment_cache = [
-                (start, stop, int(host[start])) for start, stop in zip(bounds[:-1], bounds[1:])
-            ]
-        return self._segment_cache
-
-    def _video_layout(self, video_indices: torch.Tensor) -> int:
-        """Number of leading, static keyframe-patch rows in the video latent tensor."""
-        if self._condition_video_rows is None:
-            host = video_indices.detach().cpu()
-            discontinuities = (host[1:] - host[:-1] != 1).nonzero().flatten()
-            self._condition_video_rows = int(discontinuities[0]) + 1 if len(discontinuities) else 0
-        return self._condition_video_rows
-
-    def _project_video(self, hidden_states: torch.Tensor, condition_rows: int, dtype: torch.dtype) -> torch.Tensor:
-        source = hidden_states[0]
-        if condition_rows == 0:
-            return self.video_patch_proj(source.float()).to(dtype)
-        if self._condition_video_embedding is None:
-            self._condition_video_embedding = self.video_patch_proj(source[:condition_rows].float()).to(dtype)
-        generated = self.video_patch_proj(source[condition_rows:].float()).to(dtype)
-        return torch.cat((self._condition_video_embedding, generated), dim=0)
-
-    @staticmethod
-    def _modulate(hidden, shift, scale, row_ids, segments):
-        if FUSED_ADALN and hidden.is_cuda and hidden.is_contiguous():
-            _adaln_modulate_kernel[(triton.cdiv(hidden.numel(), 256),)](
-                hidden, shift, scale, row_ids, hidden.numel(), HIDDEN, shift.stride(0), num_warps=4
-            )
-            return hidden
-        for start, stop, row in segments:
-            hidden[start:stop].mul_(1.0 + scale[row]).add_(shift[row])
-        return hidden
-
-    @staticmethod
-    def _gate(hidden, update, gate, row_ids, segments):
-        if FUSED_ADALN and hidden.is_cuda and hidden.is_contiguous() and update.is_contiguous():
-            _adaln_gate_kernel[(triton.cdiv(hidden.numel(), 256),)](
-                hidden, update, gate, row_ids, hidden.numel(), HIDDEN, gate.stride(0), num_warps=4
-            )
-            return hidden
-        for start, stop, row in segments:
-            hidden[start:stop].addcmul_(update[start:stop], gate[row])
-        return hidden
-
-    def forward(
-        self,
-        hidden_states,
-        audio_hidden_states,
-        encoder_hidden_states,
-        timestep,
-        timestep_indices,
-        token_tags,
-        position_ids,
-        video_indices,
-        audio_indices,
-        text_indices,
-        attention_kwargs=None,
-        return_dict=True,
-    ):
-        from diffusers.models.transformers.transformer_minimax_h3 import MiniMaxH3TransformerOutput
-
-        if hidden_states.shape[0] != 1:
-            raise ValueError("The NVFP4 MiniMax-H3 engine supports batch size 1.")
-
-        condition_rows = self._video_layout(video_indices)
-        reused = self._step_cache.try_reuse(hidden_states, audio_hidden_states, condition_rows)
-        if reused is not None:
-            video_output, audio_output = reused
-            if not return_dict:
-                return video_output, audio_output
-            return MiniMaxH3TransformerOutput(sample=video_output, audio_sample=audio_output)
-
-        text = self._refine_text(encoder_hidden_states[0].to(torch.bfloat16))
-        video = self._project_video(hidden_states, condition_rows, text.dtype)
-        audio = self.audio_patch_proj(audio_hidden_states[0].float()).to(text.dtype)
-        # Text, video and audio indices partition the packed sequence, so initialization would only add a full HBM
-        # write before the three index copies overwrite every row.
-        packed = text.new_empty((position_ids.shape[0], HIDDEN))
-        packed.index_copy_(0, text_indices, text)
-        packed.index_copy_(0, video_indices, video)
-        packed.index_copy_(0, audio_indices, audio)
-
-        time_embedding = self._time_embedding(timestep)
-        adaln_indices = timestep_indices * 3 + token_tags.clamp(min=0)
-        segments = self._segments(adaln_indices)
-        rope = self._rope(position_ids, packed.dtype)
-        use_sol_attention = self._step_cache.profile != "exact" and self._sol_attention.enabled
-        self._sol_attention.observe(video_indices, packed.shape[0], self._step_cache.step)
-
-        reused_tail = False
-        for layer, block in enumerate(self.blocks):
-            if layer == 0:
-                # Block 0 writes its residual updates in place, so retain the pre-block value for the official FBC
-                # signal `(head_output - head_input)`.
-                block_input = packed.detach().clone()
-            # One conversion per small modulation table, rather than one conversion per sequence segment.
-            modulations = block.adaln_proj(time_embedding, packed.dtype)
-            shift_attn, scale_attn, gate_attn, shift_mlp, scale_mlp, gate_mlp = modulations
-            normalized = self._modulate(block.norm1(packed), shift_attn, scale_attn, adaln_indices, segments)
-            packed = self._gate(
-                packed,
-                block.attn(
-                    normalized,
-                    rope,
-                    self.attention_backend,
-                    self._sol_attention if use_sol_attention else None,
-                    layer,
-                ),
-                gate_attn,
-                adaln_indices,
-                segments,
-            )
-            normalized = self._modulate(block.norm2(packed), shift_mlp, scale_mlp, adaln_indices, segments)
-            packed = self._gate(packed, block.mlp(normalized), gate_mlp, adaln_indices, segments)
-
-            if layer == 0:
-                if self._step_cache.first_block_decision(block_input, packed):
-                    packed = packed + self._step_cache.tail_residual
-                    reused_tail = True
-                    break
-            if layer == len(self.blocks) - 1 and not reused_tail:
-                self._step_cache.update_first_block_tail(packed)
-
-        shift, scale = self.final_layer.adaln_proj(time_embedding)
-
-        # Keyframe output rows are discarded by the scheduler. Avoid their FP32 output projection and put zeros in
-        # those unused slots to retain the pipeline's expected tensor shape.
-        generated_video_indices = video_indices[condition_rows:]
-        if self._output_indices is None:
-            self._generated_rows = generated_video_indices.shape[0]
-            self._output_indices = torch.cat((generated_video_indices, audio_indices))
-        generated_rows = self._generated_rows
-        normalized_output = self.final_layer.norm(packed.index_select(0, self._output_indices))
-        video_times = timestep_indices.index_select(0, generated_video_indices)
-        video_hidden = normalized_output[:generated_rows]
-        video_hidden = video_hidden * (1.0 + scale.index_select(0, video_times)) + shift.index_select(0, video_times)
-        generated_video_output = self.final_layer.video_out(video_hidden.float())
-        if condition_rows:
-            video_output = generated_video_output.new_zeros((1, hidden_states.shape[1], VIDEO_DIM))
-            video_output[0, condition_rows:] = generated_video_output
-        else:
-            video_output = generated_video_output.unsqueeze(0)
-
-        audio_times = timestep_indices.index_select(0, audio_indices)
-        audio_hidden = normalized_output[generated_rows:]
-        audio_hidden = audio_hidden * (1.0 + scale.index_select(0, audio_times)) + shift.index_select(0, audio_times)
-        audio_output = self.final_layer.audio_out(audio_hidden.float()).unsqueeze(0)
-
-        self._step_cache.update(
-            hidden_states,
-            audio_hidden_states,
-            video_output,
-            audio_output,
-            condition_rows,
-        )
-
-        if not return_dict:
-            return video_output, audio_output
-        return MiniMaxH3TransformerOutput(sample=video_output, audio_sample=audio_output)
-
-
-def load_transformer() -> H3NVFP4Transformer:
-    if torch.version.cuda is None or int(torch.version.cuda.split(".")[0]) < 13:
-        raise RuntimeError("NVFP4 requires the CUDA 13 PyTorch build.")
-    from huggingface_hub import hf_hub_download
-
-    path = hf_hub_download(repo_id=NVFP4_REPO, filename=NVFP4_FILE)
-    transformer = H3NVFP4Transformer()
-    transformer.load(path)
-    print(f"[h3-nvfp4] loaded {NVFP4_REPO}/{NVFP4_FILE}", flush=True)
-    return transformer
-
-
-def status() -> str:
-    return (
-        f"NVFP4 · linear residual forecast {FORECAST_BLEND:g} / adaptive cache {EASYCACHE_THRESHOLD:g} · "
-        f"pruned AdaLN curve · fused QKV/QK-norm/RoPE · `{NVFP4_REPO}`"
-    )

h3_split_blocks.py

@@ -1,8 +1,8 @@
-"""Composable conditioner and generator halves of MiniMax-H3, for both checkpoint partitions.
+"""The halves of a **split** MiniMax-H3 deployment, for both of its checkpoint partitions.
 
-The blocks cut `MiniMaxH3Blocks` at its `text_encoder` step. They can run in separate Spaces with `prompt_embeds` and
-`text_token_tags` as a wire format, or sequentially in one GPU worker when a compact local conditioner fits beside
-the generator.
+MiniMax-H3 is 195.9 GiB in bfloat16 and a ZeroGPU Space is evicted at 150 GB of storage, so `MiniMaxH3Blocks` is cut
+at its `text_encoder` step: the 62.14 GiB Qwen3-VL runs in the conditioner Space, everything else in a generator
+Space, and `prompt_embeds` + `text_token_tags` is the whole wire format between them.
 
 `resize` / `setup` run on **both** sides: they own no pretrained component, and each half needs the canvas and the
 prepared keyframes or normalized references. Both conditioner halves also return the resolved `height` / `width` /
@@ -84,7 +84,7 @@ class MiniMaxH3GeneratorBlocks(SequentialPipelineBlocks):
         return (
             "The denoising half of a split MiniMax-H3 deployment: the `t2va` / `fl2va` branch of `MiniMaxH3Blocks` "
             "without its text-encoder step, so `prompt_embeds` and `text_token_tags` come in as inputs and the "
-            "conditioner is supplied by the caller or by the preceding local conditioner half."
+            "62.14 GiB Qwen3-VL conditioner is never loaded here."
         )
 
     @property
@@ -139,7 +139,7 @@ class MiniMaxH3Ref2VAGeneratorBlocks(SequentialPipelineBlocks):
         return (
             "The denoising half of a split MiniMax-H3 `ref2va` deployment: the `ref2va` branch of `MiniMaxH3Blocks` "
             "without its text-encoder step, so `prompt_embeds` and `text_token_tags` come in as inputs and the "
-            "conditioner is supplied by the caller or preceding local half. The transformer is the `transformer_ref` partition."
+            "62.14 GiB Qwen3-VL conditioner is never loaded here. The transformer is the `transformer_ref` partition."
         )
 
     @property

packages.txt

@@ -1 +1 @@
-ffmpeg
+ffmpeg

pk_workflow.py

@@ -0,0 +1,217 @@
+"""The three things that make `Plaguekind/Minimax-H3` a *workflow* rather than just MiniMax-H3.
+
+`Plaguekind/Minimax-H3` ships no weights: it is a ComfyUI graph
+(`PlagueKind-MinimaxH3-V1.5.json`) over `Comfy-Org/MiniMax-H3`, and everything it contributes is in the
+sampling and the post chain. Read off the graph, that is:
+
+| ComfyUI node | widget | here |
+|---|---|---|
+| `KSamplerSelect` | `euler` | MiniMax-H3's only sampler; the checkpoint is CFG-distilled, one forward per step |
+| `BasicScheduler` | `linear_quadratic`, 15 steps, denoise 1.0 | `linear_quadratic_sigmas` |
+| `ImageSharpenKJ` | `rcas`, 0.3 | `rcas` |
+| `FrameInterpolate` + `FrameInterpolationModelLoader` | `film_net_fp16.safetensors`, multiplier 2 | `interpolate` |
+| `CreateVideo` | fps `24 * 2` | 48 fps out |
+| `RTXVideoSuperResolution` | 2x, `ULTRA` | **not portable** — NVIDIA NGX, Windows/RTX driver only |
+
+The sigma schedule is the one that changes the pixels most, and the one that is easy to get subtly wrong.
+"""
+
+from __future__ import annotations
+
+import torch
+
+
+# ----------------------------------------------------------------------------------------------------------------
+# BasicScheduler(linear_quadratic)
+# ----------------------------------------------------------------------------------------------------------------
+# MiniMax-H3 carries two rectified-flow schedules per request, `shift = 12` for the video rows and `shift = 3` for
+# the audio rows. diffusers builds both from one `linspace(1, 0, steps)` base grid; ComfyUI instead samples the
+# *video* schedule and derives the audio one from it in closed form
+# (`comfy/ldm/minimax/model.py::time_shift_sigma`). The two agree, because the shift is a bijection of the base
+# grid — which is what lets a schedule chosen in ComfyUI's video-sigma space be transplanted here exactly.
+#
+# `linear_quadratic` is Mochi's schedule (`comfy/samplers.py::linear_quadratic_schedule`) and it does **not** go
+# through the model's shift at all: it is `sigma_max = 1.0` scaled, so the grid PlagueKind's 15 steps actually run
+# is this one verbatim, in the video stream, with the audio stream shifted off it.
+VIDEO_SHIFT = 12.0
+AUDIO_SHIFT = 3.0
+
+
+def linear_quadratic_sigmas(
+    steps: int, threshold_noise: float = 0.025, linear_steps: int | None = None
+) -> torch.Tensor:
+    """ComfyUI's `linear_quadratic` sigma grid, in MiniMax-H3's video-sigma space.
+
+    Ported from `comfy/samplers.py::linear_quadratic_schedule` (itself from Mochi), with
+    `model_sampling.sigma_max == 1.0`, which is what a rectified-flow model has. Returns `steps + 1` strictly
+    decreasing sigmas from exactly 1.0 to exactly 0.0, so it drives `steps` forwards — ComfyUI's step count, not
+    diffusers' (where the terminal zero is one of the `num_inference_steps`).
+
+    Half the steps crawl through the first 2.5% of the trajectory and the rest sprint the remaining 97.5%: it is a
+    front-loaded schedule, which is why 15 steps of it hold up against ~28 of the native grid.
+    """
+    steps = int(steps)
+    if steps < 2:
+        return torch.tensor([1.0, 0.0], dtype=torch.float32)
+    if linear_steps is None:
+        linear_steps = steps // 2
+
+    linear = [i * threshold_noise / linear_steps for i in range(linear_steps)]
+    threshold_noise_step_diff = linear_steps - threshold_noise * steps
+    quadratic_steps = steps - linear_steps
+    quadratic_coef = threshold_noise_step_diff / (linear_steps * quadratic_steps**2)
+    linear_coef = threshold_noise / linear_steps - 2 * threshold_noise_step_diff / (quadratic_steps**2)
+    const = quadratic_coef * (linear_steps**2)
+    quadratic = [quadratic_coef * (i**2) + linear_coef * i + const for i in range(linear_steps, steps)]
+
+    schedule = linear + quadratic + [1.0]
+    return torch.tensor([1.0 - value for value in schedule], dtype=torch.float32)
+
+
+def time_shift_sigma(sigma: torch.Tensor, from_shift: float, to_shift: float) -> torch.Tensor:
+    """Move a sigma between two exponential shifts of the same base grid.
+
+    `comfy/ldm/minimax/model.py::time_shift_sigma`: invert `sigma = s*b / (1 + (s-1)*b)` back to the base grid `b`,
+    then re-apply the other shift. Monotonic, and it fixes both 0.0 and 1.0, so a strictly decreasing schedule that
+    ends at zero stays one.
+    """
+    if from_shift == to_shift:
+        return sigma
+    base = sigma / (from_shift + sigma * (1.0 - from_shift))
+    return to_shift * base / (1.0 + (to_shift - 1.0) * base)
+
+
+class use_linear_quadratic:
+    """Force MiniMax-H3's two schedulers onto the `linear_quadratic` grid for one pipeline call.
+
+    A context manager rather than a pipeline-block subclass on purpose: `MiniMaxH3Scheduler.set_timesteps` already
+    takes a fully-formed `sigmas=` schedule as public API, so nothing here reaches into the modular blocks, and the
+    override lives and dies inside one request.
+    """
+
+    def __init__(self, pipe, steps: int, threshold_noise: float = 0.025, enabled: bool = True):
+        self.schedulers = [pipe.scheduler, pipe.audio_scheduler] if enabled else []
+        self.steps = int(steps)
+        self.threshold_noise = float(threshold_noise)
+
+    def __enter__(self):
+        video_sigmas = linear_quadratic_sigmas(self.steps, self.threshold_noise)
+        for scheduler in self.schedulers:
+            sigmas = time_shift_sigma(video_sigmas, VIDEO_SHIFT, float(scheduler.shift))
+            unbound = type(scheduler).set_timesteps
+
+            def forced(num_inference_steps=None, device=None, sigmas=None, _s=scheduler, _grid=sigmas, _f=unbound):
+                return _f(_s, None, device, _grid)
+
+            scheduler.set_timesteps = forced
+        return self
+
+    def __exit__(self, *_):
+        for scheduler in self.schedulers:
+            scheduler.__dict__.pop("set_timesteps", None)
+        return False
+
+
+# ----------------------------------------------------------------------------------------------------------------
+# ImageSharpenKJ(rcas, 0.3)
+# ----------------------------------------------------------------------------------------------------------------
+def rcas(video: torch.Tensor, strength: float, chunk: int = 16) -> torch.Tensor:
+    """AMD FidelityFX **RCAS** — Robust Contrast Adaptive Sharpening — on `(frames, 3, H, W)` in `[0, 1]`.
+
+    The FidelityFX kernel, which is what `ImageSharpenKJ`'s `rcas` mode is: a 5-tap cross, a sharpening lobe whose
+    strength is limited per pixel so the ring it would create cannot leave `[0, 1]`, and a renormalised blend.
+
+        lobe = clamp(attenuation * min over channels of max(-min / 4*max, -(1 - max) / 4*(1 - min)), -0.1875, 0)
+        out  = (center + lobe * (n + s + e + w)) / (1 + 4 * lobe)
+
+    `lobe` is negative, so the neighbours are subtracted: a high-pass with a headroom-aware gain, which is why it
+    sharpens MiniMax-H3's slightly soft VAE output without haloing it. PlagueKind's 0.3 is the strength; the note in
+    the workflow calls it "very natural" and that matches — the lobe clamp caps it well below a visible ring.
+
+    Batched over `chunk` frames at a time rather than ComfyUI's one, and written back in place: the clip is already
+    resident on the card, but this runs immediately after the denoise loop's allocation peak, and a whole-clip pass at
+    the full 1344x768x124 would ask the allocator for ~8 GB of intermediates at exactly the wrong moment.
+    """
+    if strength <= 0:
+        return video
+
+    frames, _, height, width = video.shape
+    strength = float(strength)
+    for start in range(0, frames, chunk):
+        center = video[start : start + chunk]
+        padded = torch.nn.functional.pad(center, (1, 1, 1, 1), mode="reflect")
+        north = padded[:, :, 0:height, 1 : width + 1]
+        south = padded[:, :, 2 : height + 2, 1 : width + 1]
+        west = padded[:, :, 1 : height + 1, 0:width]
+        east = padded[:, :, 1 : height + 1, 2 : width + 2]
+
+        low = torch.minimum(torch.minimum(torch.minimum(torch.minimum(north, south), west), east), center)
+        high = torch.maximum(torch.maximum(torch.maximum(torch.maximum(north, south), west), east), center)
+
+        hit_min = -low / (high * 4.0 + 1e-6)
+        hit_max = -(1.0 - high) / ((1.0 - low) * 4.0 + 1e-6)
+        lobe = torch.maximum(hit_min, hit_max).amin(dim=1, keepdim=True)
+        lobe = (lobe * strength).clamp_(-0.1875, 0.0)
+        del low, high, hit_min, hit_max
+
+        neighbours = north + south + east + west
+        center.copy_(((center + lobe * neighbours) / (1.0 + 4.0 * lobe)).clamp_(0.0, 1.0))
+    return video
+
+
+# ----------------------------------------------------------------------------------------------------------------
+# FrameInterpolate(film_net_fp16, multiplier=2)
+# ----------------------------------------------------------------------------------------------------------------
+FILM_REPO = "Comfy-Org/frame_interpolation"
+FILM_FILE = "frame_interpolation/film_net_fp16.safetensors"
+
+
+def load_film():
+    """FILM, off the same checkpoint the workflow names. CPU work; `None` on any failure, and the caller skips."""
+    from huggingface_hub import hf_hub_download
+    from safetensors.torch import load_file
+
+    from film_net import FILMNet
+
+    path = hf_hub_download(FILM_REPO, FILM_FILE)
+    model = FILMNet()
+    model.load_state_dict(load_file(path))
+    return model.eval().to(torch.float16)
+
+
+@torch.no_grad()
+def interpolate(model, video: torch.Tensor, multiplier: int = 2) -> torch.Tensor:
+    """`multiplier`x frame interpolation of `(frames, 3, H, W)` in `[0, 1]`, FILM, on the card.
+
+    Mirrors ComfyUI's `FrameInterpolate`: one pass per adjacent pair, the flow computed once per pair and reused for
+    every intermediate timestep (`forward_multi_timestep`), and the feature pyramid of frame `i + 1` carried over as
+    frame `i` of the next pair — which halves the feature extractions. Output length is
+    `(frames - 1) * multiplier + 1`, i.e. 24 fps in, `24 * multiplier` fps out.
+    """
+    frames = video.shape[0]
+    if model is None or frames < 2 or multiplier < 2:
+        return video
+
+    dtype = torch.float16
+    timesteps = [t / multiplier for t in range(1, multiplier)]
+    # float16, not the input's float32: the buffer is the largest allocation of the whole post chain (a 2x pass over
+    # 124 frames at 1344x768 is 247 of them) and it happens right after the denoise loop's peak.
+    out = torch.empty(((frames - 1) * multiplier + 1, *video.shape[1:]), dtype=dtype, device=video.device)
+    out[0] = video[0]
+    cursor = 1
+
+    cache: dict = {}
+    for index in range(frames - 1):
+        first = video[index : index + 1].to(dtype)
+        second = video[index + 1 : index + 2].to(dtype)
+        cache["img0"] = cache.pop("next") if "next" in cache else model.extract_features(first)
+        cache["img1"] = model.extract_features(second)
+        cache["next"] = cache["img1"]
+
+        middles = model.forward_multi_timestep(first, second, timesteps, cache=cache)
+        out[cursor : cursor + len(timesteps)] = middles.to(video.dtype).clamp_(0.0, 1.0)
+        cursor += len(timesteps)
+        out[cursor] = video[index + 1]
+        cursor += 1
+
+    return out

requirements.txt

@@ -1,5 +1,10 @@
-# diffusers is installed from the canonical MiniMax-H3 PR
-# https://github.com/huggingface/diffusers/pull/14371 ("Minimax h3 follow up (review & refactor)")
+# `diffusers` is installed from the canonical MiniMax-H3 pull request,
+# https://github.com/huggingface/diffusers/pull/14371 ("Minimax h3 follow up (review & refactor)"), pinned to a
+# **commit** rather than to its `minimax-h3-refactor` branch: the PR is a WIP and its head moves, and this Space's
+# blocks subclass its block classes. Re-pin — and re-check `h3_split_blocks.py` against the block names of the new
+# head — whenever the PR updates.
+#
+#   665f578278365ea4a3318cb8c9b66ce6c01204b9 = refs/pull/14371/head at the time of this deploy
 --extra-index-url https://download.pytorch.org/whl/cu130
 diffusers @ git+https://github.com/huggingface/diffusers.git@665f578278365ea4a3318cb8c9b66ce6c01204b9
 torch==2.11.0
@@ -7,11 +12,15 @@ torchvision==0.26.0
 # The Qwen3-VL processor decides the vision patch count, so a different minor changes the conditioning.
 transformers==5.8.0
 accelerate==1.14.0
-# Blackwell-native NVFP4 GEMMs and the fused Q/K RMSNorm + split-half RoPE kernel used by h3_nvfp4.py.
-# CUDA 13 is mandatory: older builds emulate this path and are slower than BF16.
-comfy-kitchen==0.2.26
-# PyAV muxes the generated soundtrack onto the frames (encode_video)
+# diffusers pins <2.
+huggingface-hub==1.24.0
+gradio==6.20.0
+spaces==0.51.1
+# No `kernels` pin on purpose: the Hub attention backends want `kernels>=0.12.3`, and that version breaks
+# transformers 5.8.0 at import.
+# PyAV muxes the generated soundtrack onto the frames (`encode_video`).
 av
 pillow
 numpy
-safetensors>=0.8.0
+requests
+safetensors>=0.8.0