ACT — flatten_tshirt (bimanual cloth flattening)

An ACT policy trained on the flatten_tshirt task of a bimanual deformable-object (cloth / bag) manipulation benchmark. The robot is a dual-arm Piper; the task is to flatten a crumpled t-shirt on a table.

Simulation uses a GPU cloth solver co-simulated with the robot in a single model, and observations are rendered with a photorealistic renderer.

Model


Architecture	ACT, ResNet-18 vision backbone
Observation	3 × RGB `720×1280` (`static_cam`, `left_hand_cam`, `right_hand_cam`) + 14-D joint state
Action	14-D (left 6 joints + gripper, right 6 joints + gripper)
Chunk size / action steps	100 / 100
`n_obs_steps`	1

Training


Dataset	`flatten_tshirt_200` — 200 episodes / 41,464 frames, LeRobot v3.0, 25 fps
Steps	30,000
Batch size	16 (single A100-80G)
Learning rate	1e-5
Seed	1000
Image augmentation	enabled, max 3 random transforms per sample

Augmentation follows a tuned recipe (brightness / contrast / saturation / hue / sharpness / small affine) rather than the LeRobot default, which is disabled. The brightness range is deliberately asymmetric toward the darker side: simulation lighting is idealized while real RealSense D435i footage tends to be darker, so biasing the augmentation toward darker samples is the right direction for sim-to-real transfer.

Status

⚠️ This checkpoint has not yet been formally evaluated. It has only been through a small smoke-level closed-loop run, not the benchmark's standard N=100 protocol. Success-rate numbers are deliberately not published here yet; they will be added once the full evaluation has been run. Treat this as a training artifact, not a reported result.

Usage

from lerobot.policies.act.modeling_act import ACTPolicy

policy = ACTPolicy.from_pretrained("hwk0809/act-flatten-tshirt")

The policy expects the three camera streams named exactly as listed above, plus a 14-D observation.state, and returns a 14-D action. It runs in-process (no policy server needed).