NeoteAI/n0-vtla-base

$\mathcal{N}_0$-VTLA — pretrained base

Pretrained base weights for N0-VTLA, a vision–tactile–language–action policy: a π0.5-style VLA with a latent tactile pathway.

Files

model.safetensors   the full checkpoint (~8.25 GB): π0.5 VLM, flow-matching action expert,
                    latent tactile predictor, and frozen DINOv2, in one state dict
config.json         architecture summary

Load

from huggingface_hub import snapshot_download
import safetensors.torch as st
from n0vtla.training.config import get_config
from n0vtla.models_pytorch.n0vtla_policy import N0VTLAPolicy

ckpt = snapshot_download("NeoteAI/n0-vtla-base")
cfg = get_config("vtla_tactile_posttrain")
model = N0VTLAPolicy(cfg.model).eval().cuda()
missing, unexpected = st.load_model(model, f"{ckpt}/model.safetensors", strict=False, device="cuda")
assert not unexpected, unexpected

missing is expected to contain exactly paligemma_with_expert.paligemma.model.language_model.embed_tokens.weight: it is tied to lm_head.weight and safetensors stores tied weights once. unexpected must be empty.

In practice you do not call this yourself — point VTLA_PRETRAINED_CHECKPOINT at the download directory and use the post-training recipe:

hf download NeoteAI/n0-vtla-base --local-dir checkpoints/n0-vtla-base
export VTLA_PRETRAINED_CHECKPOINT="$PWD/checkpoints/n0-vtla-base"

See Post-training on your own robot data in the repository README.

Normalization statistics

Not included, by design. Normalization depends on your dataset, robot action layout, and delta-action convention, so it has to be computed on your own data:

python scripts/compute_canonical_norm.py \
  --repo-id /path/to/your/canonical_dataset \
  --robot flexiv \
  --train-config-name vtla_tactile_posttrain \
  --asset-id your_asset_id

The result lands in assets/vtla_tactile_posttrain/<asset-id>/norm_stats.json, and VTLA_ASSET_ID must match the --asset-id you used.

Action format

The policy outputs normalized element-wise delta actions. To run them on a robot:

1. De-normalize in the same element-wise (subtraction) space the model was trained in. The quantile inverse is x = (x_norm + 1) / 2 * (q99 - q01) + q01, applied to the active dims only — 10 for single-arm, 20 for dual-arm.
2. Add back to the current state, abs = state + delta, on the delta dims. Grippers and padding are already absolute. This is the standard AbsoluteActions step; there is no relative-rotation or rotation-matrix math, because the rotation delta is element-wise.
3. Execute the full chunk. The action horizon is 50 — execute all 50 steps before requesting the next prediction. Executing fewer clips the tail of the chunk, which is where the grasp-closing motion lives, and the gripper will look stuck open.

32-dim action/state layout

left arm  [0:10]:  xyz [0:3],   rot6d [3:9],   gripper [9]
right arm [10:20]: xyz [10:13], rot6d [13:19], gripper [19]
[20:32]:           reserved / unused

Rotation is rot6d, not quaternion or Euler. On disk the action field is the absolute end-effector pose; the training transforms form the delta target as action[t+k] - state[t].

License

These weights are derived from Google's PaliGemma/Gemma parameters and are therefore made available under the Gemma Terms of Use and the Gemma Prohibited Use Policy, not under the CC BY-SA 4.0 licence that covers the source code. This is inherited from the base model, not a restriction added on top.