speach1sdef178/MiniMax-H3-Semantic-Bridge

MiniMax H3 Semantic Bridge

Developed locally on a single NVIDIA RTX 3090 Ti 24 GB · No multi-GPU training cluster used

Cross-Architecture Semantic Transfer and Distillation for Video Generation

MiniMax H3 Semantic Bridge is a compact conditioning-space adapter for the standard MiniMax H3 FL2VA / text-conditioned generation path.

It grew out of an experimental cross-architecture representation-transfer project using SenseNova U1.5 as a semantic teacher. The final released adapter is standalone: SenseNova is not required at inference time.

In one line:

cross-architecture semantic transfer → conditioning-space teacher bridge → distillation → a ~11 MB standalone H3 adapter

This is not a LoRA, checkpoint merge, or conventional parameter graft. The adapter transforms native H3 conditioning before the video transformer and blends the learned semantic representation back into H3 at a controllable strength.

Scope: v1 is for standard H3 FL2VA / text-conditioned generation. Ref2VA / reference-conditioned workflows are not supported. Experimental reference-audio testing showed degraded singing/lip-sync when the adapter was inserted into Ref2VA conditioning.

---

Quick Start

Files

• MiniMaxH3_SemanticBridge_v1.safetensors — final standalone adapter.
• MiniMax_H3_Semantic_Bridge_v1.0.zip — ComfyUI custom node.
• RESEARCH_ARTICLE.md — full research narrative.
• research/ — prompt datasets, raw scripts, reports, and teacher-side research artifact.
• examples/ — controlled Native H3 vs Semantic Bridge A/B videos and their exact prompts.

Installation

1. Extract MiniMax_H3_Semantic_Bridge_v1.0.zip into:

ComfyUI/custom_nodes/

2. Create:

ComfyUI/models/semantic_bridge/

3. Put:

MiniMaxH3_SemanticBridge_v1.safetensors

inside that folder.

4. Restart ComfyUI.

5. Example ComfyUI Workflow

The Semantic Bridge is designed to fit into a standard MiniMax H3 FL2VA workflow without modifying or merging weights into the diffusion model itself.

The example below shows where the MiniMax H3 Image to Video + Semantic Bridge node is placed in a working ComfyUI pipeline.

Download the workflow

➡️ Download the example ComfyUI workflow (JSON)

The workflow is provided primarily as a practical integration and node-placement reference. It demonstrates how the Semantic Bridge node produces the conditioning and latent inputs that are passed into the standard downstream MiniMax H3 sampling pipeline.

The rest of the H3 generation pipeline can remain conventional. The Semantic Bridge operates in the conditioning path rather than modifying the diffusion transformer weights.

Recommended settings

Setting	Value	Notes
alpha	0.10	Recommended starting point for general use
magnitude_match	per_token	Recommended mode
alpha	0.15	Stronger semantic influence; used for the published A/B examples

Note: The example workflow shown above uses alpha = 0.15, matching the setting used for the qualitative A/B comparisons in this repository. For general use, alpha = 0.10 remains the recommended starting point.

The provided workflow is an example rather than a required preset. Resolution, sampler settings, model locations, VAE configuration, and other parts of the pipeline can be adapted to an existing MiniMax H3 setup.

Nodes

• MiniMax H3 Image to Video + Semantic Bridge
• MiniMax H3 Semantic Bridge
• MiniMax H3 Clear Semantic Bridge Cache

Recommended settings

alpha = 0.10
magnitude_match = per_token

For the qualitative A/B examples below, alpha = 0.15 was intentionally used to make the behavioral difference easier to observe.

---

What problem was this trying to solve?

Generative models can recognize all the concepts in a prompt while still failing to preserve the relationships between those concepts.

For example, a prompt may specify not only a person, table, bottle, mirror, and light source, but also:

• which hand holds which object;
• which hand must remain still;
• left/right ordering of several objects;
• which surfaces are transparent or reflective;
• how a mirror should correspond to the real scene;
• how light passes through one material but reflects from another;
• whether an action is explicitly requested or explicitly not requested.

The project therefore focused on semantic structure and prompt adherence, rather than adding new visual concepts to H3.

Areas explored during the research included:

• complex composition;
• spatial relationships;
• anatomy and body relationships;
• object counting;
• text rendering / textual constraints;
• materials and lighting;
• reflections, transparency, and occlusion;
• long prompts with several simultaneous constraints.

---

Hardware / Local Reproducibility

This project was developed and tested locally on a single NVIDIA RTX 3090 Ti with 24 GB of VRAM.

The representation extraction, cross-architecture bridge experiments, adapter distillation, evaluation, and MiniMax H3 A/B generation tests were carried out on this local workstation. No multi-GPU training cluster was used for the development of the Semantic Bridge.

One of the practical goals of the project was to explore whether cross-architecture representation transfer and distillation of this kind could be investigated using high-end consumer hardware rather than requiring datacenter-scale infrastructure.

The final Semantic Bridge is substantially lighter than the original research setup: normal inference requires only MiniMax H3 and the compact standalone adapter; the teacher model used during the research phase is not required.

Research path

1. Direct grafting failed

The project started as a direct grafting experiment between SenseNova U1.5 and MiniMax H3.

The architectures did not expose useful parameter-level correspondences. Exact shape matching, transpose matching, and simple input/output dimensional matching did not provide a meaningful path for direct tensor transplantation.

That negative result changed the question from:

Which weights can be copied?

to:

Can the models' internal representations of the same prompt be aligned?

2. Hidden-representation alignment

Hidden states from both systems were extracted across a deliberately varied semantic prompt set. Lightweight projections were trained between candidate representation spaces.

A substantially stronger correspondence emerged than the alternatives.

On held-out prompts from the original distribution, the strongest experimental mapping reached approximately:

validation cosine ≈ 0.904

3. Strict OOD test

A separate set of 160 prompts was constructed to stress harder combinations of anatomy, counting, materials/light, spatial structure, text, architecture/vehicles, reflection/occlusion, and long compositions.

With the bridge frozen, strict OOD similarity was approximately:

0.749

The drop was real, but the mapping did not collapse. This motivated testing the representation inside the actual H3 generation path.

4. Full teacher bridge

An experimental Full Bridge used SenseNova at inference time, projected the teacher-side representation into H3-compatible conditioning, magnitude-aligned it, and blended it with native H3 conditioning.

Conceptually:

H = native H3 conditioning
S = mapped teacher semantic representation
C = H + alpha * (S - H)

The Full Bridge produced coherent H3 generations and visible behavioral changes, demonstrating that the cross-model representation mapping survived the downstream video-generation process.

However, it required the full teacher model at runtime, which was impractical.

5. Distillation

The Full Bridge was then treated as a teacher. A compact H3-side student was trained to predict the teacher-derived representation directly from H3's own conditioning.

Early attempts to predict the correction delta directly were weak (best correction similarity around 0.51). Predicting the teacher-derived representation itself worked dramatically better.

The final result was a small standalone adapter with no SenseNova runtime dependency.

---

Final distillation results

The preserved V3 report records a 500-prompt training split and 100-prompt validation split containing both original-distribution and harder/OOD examples.

Metric	Result
Teacher representation cosine	0.995890
Semantic correction cosine	0.983558
Main-distribution correction	0.980888
OOD correction	0.989788
Minimum correction	0.935910
Blend cosine, alpha 0.10	0.999958
Blend cosine, alpha 0.20	0.999827
Blend cosine, alpha 0.30	0.999602

These are representation-space / distillation metrics. They do not mean that video quality improves by the same percentages, and they are not a substitute for controlled visual evaluation.

Final correction similarity by semantic category

Category	Similarity
Text	0.996061
Reflection / occlusion	0.995932
Complex counting	0.994557
Complex text	0.991912
Reflection / occlusion OOD	0.991855
Complex spatial	0.991125
Long composition	0.990637
Architecture / vehicle	0.990606
Complex material / light	0.987394
Material	0.986263
Complex anatomy	0.982104
Anatomy	0.980773
Lighting	0.978327
Spatial	0.974002
Counting	0.965250

---

Qualitative A/B examples

The following examples use the same prompt and generation setup within each pair. The intended comparison is Native H3 versus H3 with Semantic Bridge enabled. The published Bridge examples use alpha = 0.15 to make the effect easier to inspect visually.

These examples are qualitative observations, not a benchmark or proof of universal improvement.

Example 01 — Rooftop Train Chase

Focus: complex motion, anatomy, action sequencing, physical interaction, material response, and spatial continuity.

Native MiniMax H3	+ Semantic Bridge (alpha=0.15)
View native video	View bridge video

The exact prompt is preserved in:

examples/01_rooftop_train_chase/prompt.txt

This example was designed to stress several constraints simultaneously: two moving characters, pursuit distance, running anatomy, a specific vault interaction, hand contact with the obstacle, landing continuity, moving camera geometry, wet reflective metal, rain, sparks, and a rapidly moving city background.

---

Example 02 — Prompt Adherence, Materials, Reflection & Transparency

Focus: explicit state adherence, hand behavior, material differences, object ordering, reflection, transparency, and text.

Native MiniMax H3	+ Semantic Bridge (alpha=0.15)
View native video	View bridge video

Controlled prompt-following observation

A particularly useful instruction in this prompt is:

"Her right hand rests flat on the glass tabletop with all five fingers naturally separated and clearly visible."

In this A/B generation:

• Native H3 introduces an unrequested action: the right hand moves across the tabletop rather than remaining in the requested resting state.
• Semantic Bridge (alpha=0.15) keeps the hand resting on the glass surface, more closely preserving the explicitly requested state.

This observation is important because it is not a subjective claim that one result is simply "prettier." The prompt specifies a directly observable state — resting — and the two outputs behave differently with respect to that instruction.

It is still presented only as qualitative evidence from this controlled pair, not as a statistical claim that the adapter universally improves prompt adherence.

The exact full prompt is preserved in:

examples/02_glass_table_prompt_adherence/prompt.txt

The same prompt also stresses:

• anatomically coherent hands;
• a cup held specifically in the left hand;
• exactly three tabletop objects in a specified left-to-right order;
• the text NIGHT SHIFT;
• mirror correspondence;
• transparent glass;
• refractive bottle behavior;
• reflective metal;
• matte ceramic;
• warm/cool directional lighting interactions.

---

What Semantic Bridge is — and is not

Semantic Bridge is best described as:

a compact conditioning-space adapter distilled from a cross-architecture semantic mapping

It is not:

• a conventional LoRA;
• a checkpoint merge;
• a direct parameter graft;
• a copy of SenseNova weights;
• a second multimodal model running beside H3.

At inference time the released adapter operates only on H3 conditioning.

Conceptually:

Prompt
  ↓
H3 text conditioning
  ↓
Semantic Bridge
  ↓
learned semantic representation
  ↓
magnitude matching
  ↓
controlled residual blend
  ↓
MiniMax H3 video generation

SenseNova was used as a teacher during the research process only.

---

Scope: FL2VA / standard text-conditioned H3 only

This limitation is important.

The released student was distilled from the standard H3 conditioning path. It was not trained on the separate multimodal reference-conditioning distribution used by Ref2VA.

An experimental Ref2VA-compatible node was tested by modifying text-designated token positions while preserving visual-reference tokens. In reference-audio singing tests, this produced noticeably worse vocal articulation and stronger mumbling-like lip motion than native Ref2VA.

The practical conclusion for v1 is therefore:

Do not use this adapter for Ref2VA / reference-conditioned generation, especially reference-audio singing or lip-sync.

This does not establish that a semantic bridge can never work with Ref2VA. It suggests that a Ref2VA version should be trained separately on the multimodal conditioning regime it is intended to modify.

A useful lesson from the failed experiment is that matching tensor dimensionality does not guarantee matching conditioning semantics.

---

Research materials included

The repository intentionally includes more than the final adapter so others can inspect the experimental path.

research/datasets/

• bridge_prompts_480.json — historical filename; the preserved dataset contains 440 development prompts.
• bridge_ood_prompts_160.json — 160 strict OOD prompts.

research/raw_scripts/

Original research scripts are preserved largely as-run. They include local Windows paths and historical filenames. This is intentional: they are provided as a research snapshot rather than as a polished one-command training framework.

The scripts cover areas such as:

• architecture and tokenizer comparison;
• H3 and SenseNova hidden-state extraction;
• layer-pair screening;
• MoT diagnostic experiments;
• OOD dataset construction;
• full bridge evaluation;
• distillation screening;
• V2/V3 student training and evaluation.

research/reports/

Raw TXT/CSV outputs from the experiments are included, including negative and intermediate results.

This is deliberate. The unsuccessful directions are part of the research record and may help others avoid repeating the same experiments.

research/teacher_artifacts/

SN_L32_to_H3_L49_rank128.safetensors is preserved as an optional research artifact from the teacher-side bridge work.

It is not required to use the public Semantic Bridge.

Large extracted hidden-state .pt caches are not included. The prompt datasets and extraction scripts are provided so those intermediates can be regenerated.

---

Limitations

• This is an experimental research adapter, not a universal H3 enhancer.
• It may help some prompts, do little on others, or occasionally make a result worse.
• The reported cosine metrics evaluate agreement with the teacher-derived representation, not perceptual video quality.
• A small conditioning difference can produce a large downstream sampling difference.
• The current adapter is not validated for Ref2VA/reference-conditioned generation.
• The project has not yet been evaluated with a large standardized human-preference benchmark.
• The published A/B videos are illustrative controlled examples, not statistical proof.

---

Licensing and upstream terms

Please read this section before using or redistributing the model-derived artifacts.

MiniMax H3 is released under the MiniMax H3 Community License Agreement. The official agreement defines terms for MiniMax H3 and Model Derivatives, including distribution requirements and territorial restrictions. This repository uses license: other metadata and points directly to the upstream H3 agreement rather than relabeling the model-derived adapter as Apache/MIT.

Official H3 license:

https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE

The H3 agreement requires distributions to include a NOTICE. This repository includes NOTICE.txt.

SenseNova U1.5, used as the experimental teacher during development, is published under Apache License 2.0:

https://huggingface.co/sensenova/SenseNova-U1.5-8B-MoT

This repository does not redistribute the SenseNova checkpoint.

See LICENSE.md and UPSTREAM_LICENSES.md for repository-specific notes and direct upstream references.

This licensing summary is provided for transparency and is not legal advice. Users and redistributors should review the upstream terms themselves.

---

Full article

For the complete chronological research write-up, including failed grafting, representation screening, Full Bridge, OOD testing, early distillation, V2/V3 results, and the Ref2VA limitation, see:

RESEARCH_ARTICLE.md

---

Acknowledgements

This is an independent experimental project built around MiniMax H3 and teacher-assisted representation studies using SenseNova U1.5.

It is not an official MiniMax or SenseNova release.

The value of the project is not only the final adapter, but the possibility that useful semantic behavior may sometimes be transferred between incompatible architectures through representation alignment and distillation, even when direct parameter grafting is not meaningful.