tiny-random/sensenova-loop

This is a randomly initialized tiny model based on sensenova/SenseNova-Loop-9B-Preview for testing and debugging.

Feature	Source	This repo
Checkpoint tensor elements	243.29M BF16, 9.41B F32	9.19K BF16, 4.55M F32
Safetensors weight files	38.14 GB (9 files)	18.25 MB (2 files)
Distinct / executed decoder layers	32 / 56 (block 4-28 twice)	12 / 16 (block 4-7 twice)
Attention cadence	three linear then full, four-layer cycle	Same; four complete cycles
Hidden width / MLP width	4,096 / 12,288	8 / 32
Full Q heads / KV heads / head dim	16 / 4 / 256	8 / 4 / 32
Linear key/value heads x head dim	16x128 / 32x128	4x32 / 8x32
mHC streams / Sinkhorn iterations	2 / 20 over layers 4-28	2 / 20 over layers 4-7
MTP draft layers / precision	1 / BF16 (separate mtp.safetensors)	Same
Vision depth / hidden / heads	27 / 1,152 / 16	4 / 64 / 2
Patch / temporal patch / spatial merge	16 / 2 / 2	Same
Vision position buffer	2,304 entries	Same
Vocabulary rows / tying	248,320 / untied	Same
Backbone storage precision	FP32	Same
SSM/conv/mHC parameters	FP32	Same
Maximum context length	262,144	Same

The pinned Loop/mHC wrapper loads the native Qwen3.5 backbone; the shared block owns one weight set but one dynamic-cache slot per visit.

File path	Size
model.safetensors	18.2MB
mtp.safetensors	0.0MB

Example usage

Transformers (unverified)

Load the included pinned Loop/mHC implementation with trust_remote_code=True. The example sends one image and four predecoded video frames; the hybrid linear-attention path requires CUDA.

def run_transformers(model_id: str):
    import numpy as np
    import torch
    from PIL import Image
    from transformers import AutoModelForImageTextToText, AutoProcessor
    from transformers.video_utils import VideoMetadata

    model_id = str(model_id)
    processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
    model = (
        AutoModelForImageTextToText.from_pretrained(
            model_id, trust_remote_code=True, dtype=torch.bfloat16, attn_implementation="sdpa"
        )
        .eval()
        .cuda()
    )
    image = Image.fromarray(np.random.randint(0, 255, (256, 256, 3), dtype=np.uint8), "RGB")
    frames = np.stack(
        [
            np.asarray(Image.new("RGB", (256, 256), color))
            for color in ((220, 30, 30), (30, 220, 30), (30, 30, 220), (220, 220, 30))
        ]
    )
    messages = [
        {
            "role": "user",
            "content": [
                {"type": "image", "image": image},
                {"type": "video", "video": frames},
                {"type": "text", "text": "Describe the image and the video."},
            ],
        }
    ]
    inputs = processor.apply_chat_template(
        messages,
        add_generation_prompt=True,
        tokenize=True,
        return_dict=True,
        return_tensors="pt",
        enable_thinking=False,
        processor_kwargs={"video_metadata": [VideoMetadata(total_num_frames=4, fps=2, frames_indices=[0, 1, 2, 3])]},
    ).to(model.device)
    assert inputs["pixel_values"].shape[0] > 0 and inputs["pixel_values_videos"].shape[0] > 0
    with torch.no_grad():
        logits = model(**inputs).logits
        assert torch.isfinite(logits).all()
        output = model.generate(**inputs, do_sample=False, min_new_tokens=8, max_new_tokens=8)
    print(processor.decode(output[0, inputs["input_ids"].shape[1] :], skip_special_tokens=False))


if __name__ == "__main__":
    run_transformers(model_id="scratch/sensenova-loop")

Creating this repo

• Code: .meta/create_model.py
• Model structure: .meta/model.log
• Environment: .meta/env.json

Change log

• 2026-11-03: Initial version: 12 physical layers / 16 looped executions with two-stream FP32 mHC routers and a separate BF16 MTP draft.

This repo is generated by GPT-6.1-Sol.