OpenWAM-Stanford/OpenWAM-Pretraining

OpenWAM Pretraining

This is an OpenWAM video pretraining checkpoint. It is an intermediate research checkpoint for text-conditioned latent video prediction, trained with single-view video and synchronized multi view compositions. It is intended for video prediction research and subsequent adaptation; it is not an action-finetuned robot deployment policy.

Project: OpenWAM.

Released weights

File	Purpose	Size
model_state.pt	Model weights for inference or initialization	20,433,187,287 bytes
resolved_config.yaml	Portable reference configuration	13,556 bytes
SHA256SUMS	Integrity hashes for the weights and config	—
release_metadata.json	Source checkpoint provenance and release manifest	—

This is a weights-only release. Optimizer, scheduler, and resumable trainer states are not included. The original model weight file is preserved byte-for-byte.

Training stage

• Objective: latent video prediction, with task text conditioning.
• Architecture: OpenWAM/VPM causal video prediction pipeline; a 30-layer transformer with hidden size 3,072, 24 attention heads, and 48-channel Wan 2.2 VAE latents.
• Global batch size: 288 (36 per GPU × 8 GPUs, accumulation 1).
• Variable-length batching: bucket mode, with bucket pool size 1,152.
• Optimizer: AdamW; learning rate 1e-5 in this phase.
• Checkpoint interval: 1,000 optimizer steps.

The frozen data pool for this phase contained 1,053,709 single-view clip entries and 192,689 multi view clip entries. This describes the available pool, not a claim that every entry was consumed. Single-view sources include LIBERO, UMI, AgiBot, RoboMind, InternData, RoboCOIN, FastUMI, EgoExo4D RGB, and EgoExo4D SLAM. Multi view augmentation is added alongside single views; EgoExo4D RGB and SLAM are excluded from this augmentation.

During the final 1,000-step phase, the eight ranks recorded 288,000 training window presentations: 270,813 single-view and 17,187 multi view. These counts include repeated sampling and are not unique episode counts or the cumulative exposure over all earlier training stages. Data encoded or admitted after this phase is not included in the figures above.

Multi view inputs combine synchronized cameras before VAE encoding while preserving each view's aspect ratio. Current layouts include horizontal pairs for LIBERO and FastUMI, and head-over-wrists arrangements where applicable. The six-camera RoboMind subset uses camera_top, rotated 180 degrees, above the two wrist views. FastUMI retains full image height with limited outer black-border trimming.

Download and loading

from huggingface_hub import snapshot_download

release_dir = snapshot_download(
    repo_id="OpenWAM-Stanford/OpenWAM-Pretraining",
    local_dir="OpenWAM-Pretraining",
)

The native model payload can be read with PyTorch:

from pathlib import Path
import torch

payload = torch.load(
    Path(release_dir) / "model_state.pt",
    map_location="cpu",
    mmap=True,
    weights_only=True,
)
model_state_dict = payload["model_state_dict"]

Instantiate the matching OpenWAM/VPM pipeline before loading the state dict. This release uses the native model-state format and is not a standalone AutoModel.from_pretrained or Diffusers pipeline export. The reference config preserves the model architecture, diffusion settings, training objective, and batching settings. Asset, manifest, cache, and output locations are portable local placeholders; the previous resume path, cached-text fingerprint, and training stopping target are cleared. Source labels use project-neutral VPT-* IDs. Set these paths and training.num_steps before starting training. External VAE, text encoder, tokenizer, and training data are not bundled. Use the matching frontend assets when encoding or decoding video. The weights alone cannot restore optimizer or scheduler state.

Check download integrity from the release directory with sha256sum -c SHA256SUMS.

Evaluation and limitations

Fixed-sample visual checks cover seven sources and thirteen forecasts; these are diagnostics, not an independent held-out benchmark. Results are mixed, and lower MSE does not imply uniformly better visual quality. Long forecasts can stagnate, drift, deform objects, or blur details. RoboMind and RoboCOIN examples do not yet reliably follow the ground-truth manipulation. This checkpoint should be interpreted as an intermediate OpenWAM pretraining checkpoint.