decompose_for_generation() runs model.generate(**inputs, max_new_tokens=2) once and hooks model.forward to capture the real prefill and decode kwargs (and the per-submodule kwargs via hooks on each encoder / projector / language model if the model is multi-modal). That’s why it works for any architecture — decoder-only, SSM, encoder-decoder, multi-modal — without per-model glue. export_for_generation is a one-liner over it.

The capture runs the model eagerly on inputs, so pass small but representative values — one short prompt, a single small image, a few audio frames. The exported program isn’t tied to those sizes (dynamic shapes still flow through), but smaller capture inputs make decompose_for_generation cheaper and keep symbolic-shape inference tractable.

Call decompose_for_generation directly when you want to do something between decomposing and exporting — run an eager forward for verification, swap a submodule’s inputs, skip a stage:

from transformers.exporters.utils import decompose_for_generation

components = decompose_for_generation(model, inputs)
# {"image_encoder": (submodel, fwd_kwargs), "language_model": (...), ..., "decode": (...)}

exported = {}
for name, (submodel, subinputs) in components.items():
    eager_outputs = submodel(**subinputs)
    exported[name] = exporter.export(submodel,subinputs, config=config)

Each exporter’s source file labels its stages as # ── Stage N: … ───── blocks; the tables below mirror that layout 1:1, so the file you read and the doc you read are the same map.

Two lifecycles are used consistently:

• Patches (registered via @register_patch(backend, *dotted_paths), installed via apply_patches(backend)) reversibly swap an attribute (a torch op, an ExecuTorch internal, a model class method) for the duration of the export. Pass multiple paths to a single decorator to share the same factory across targets — useful when the same method shape needs to be patched on several classes (e.g. _update_mamba_mask on Jamba/Bamba/…). Originals are restored on exit, even if the body raises.
• Fixes (registered via @register_fx_node_fix(backend) / @register_fx_program_fix(backend), applied via apply_fx_node_fixes(backend, gm) / apply_fx_program_fixes(backend, ep); ONNX-IR fixes still listed in _IR_FIXES and applied via apply_onnx_ir_fixes) mutate the in-progress graph or program in place. There’s no revert — they’re meant to permanently repair the artifact before the next pipeline step.

Every patch / fix sits in a backend-keyed registry (_PATCHES, _FX_NODE_FIXES, _FX_PROGRAM_FIXES in exporters/utils.py). Adding a new one is write a function and decorate it — nothing else.

DynamoExporter

The base exporter has one patch stage and four structural helpers. They run in this order inside DynamoExporter.export, against the original nn.Module:

OnnxExporter

OnnxExporter extends DynamoExporter with five numbered stages applied around torch.onnx.export. The labels match the # ── Stage N: … ── headers in the source:

A complete inventory of patches in the file is one grep away:

grep -nE "^def (_patch_|_fix_|_aten_)" src/transformers/exporters/exporter_onnx.py

ExecutorchExporter

ExecutorchExporter extends DynamoExporter with four numbered stages applied around to_edge_transform_and_lower and to_executorch:

When to patch the exporter vs. fix the model

The split is intentional:

• Modeling change if the pattern blocks export across multiple backends — data-dependent loops, stateful caches outside Cache, hand-written split-loop attention. Fix it once in the model and every exporter benefits.
• Exporter patch if the issue is a single backend’s lowering bug — a missing ONNX translation, an ORT validation quirk, an FX decomposition that emits a dead op. Keep the workaround in the exporter and the modeling code stays clean.

Known upstream workarounds

A small number of model classes hit confirmed bugs in onnxscript’s graph optimizer (constant folding crashing on SplitToSequence, FPN initialisers being dropped). For those, ONNX optimisation is selectively disabled via ONNX_DISABLE_OPTIMIZE_MODEL_CLASSES in the test suite — each entry is annotated with the upstream issue it works around. This list is expected to shrink as upstream bugs land; it is not an extension point for arbitrary skipping, and new entries should reference a specific upstream bug.

A second list, EXPORT_SKIP_MODEL_CLASSES, opts a handful of model classes out of the entire export sweep when the model itself is fundamentally non-exportable as-is (data-dependent control flow that can’t be vectorised, modules treated as forward arguments, …). Same expectations: every entry carries a TODO naming the underlying model change needed; the list should shrink, not grow.