license: apache-2.0
language:
- en
- zh
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- embodied-ai
- robotics
- vision-language-model
- embodied-reasoning
- spatial-reasoning
- pointing
- vla
- qwen3-vl
base_model:
- Qwen/Qwen3-VL-8B-Instruct
Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models
🌐 Project Page | 💻 Code | 🧰 EmbodiedEvalKit | 🤗 Models & Datasets
Embodied-R1.5 is a unified Embodied Foundation Model (EFM) that integrates comprehensive embodied reasoning capabilities, spanning embodied cognition, task planning, correction, and pointing, within a single 8B-parameter architecture toward general physical intelligence.
Built on the paradigm of our prior work Embodied-R1, Embodied-R1.5 leaps from a pointing specialist to a comprehensive EFM that unifies three capability dimensions in one model. Leveraging three automated data construction pipelines, we build a large-scale data system of over 15B tokens and design a multi-task balanced RL recipe to alleviate heterogeneous task conflicts. We further introduce a Planner-Grounder-Corrector (PGC) closed-loop framework that enables a single model to autonomously execute and self-correct over long-horizon tasks.
Highlights
- Unified embodied capability system. A single 8B model unifies three capability dimensions: Cognition & Spatial Reasoning, Planning & Correction, and Pointing & Location.
- State-of-the-art performance. Achieves SOTA on 16 out of 24 embodied VLM benchmarks, with an average score of 70.4% across 21 main accuracy-based benchmarks, surpassing Gemini-Robotics-ER-1.5 and GPT-5.4 by 17.0% and 21.7% respectively.
- Closed-loop autonomy. The PGC framework lets one model serve as planner, grounder, and corrector simultaneously, completing long-horizon real-world tasks (e.g., making milk tea, sweeping garbage, stacking cups) without human intervention.
- Efficient adaptation to action. Because embodied reasoning is internalized upstream, the model can be fine-tuned into Embodied-R1.5-VLA with only a small amount of action data, outperforming strong VLA baselines such as $\pi_{0.5}$ across 4 popular manipulation benchmark suites (e.g., 92.4% on SimplerEnv Google Robot Visual Matching).
- Fully open-source. We release model weights, datasets, training code, and EmbodiedEvalKit, an evaluation framework tailored for embodied tasks.
Model Details
- Architecture: Qwen3-VL (
Qwen3VLForConditionalGeneration) - Parameters: ~8B
- Modality: Image / Video + Text → Text
- Output format: All outputs are plain-text token sequences. Coordinates are normalized to $[0, 1000]$, trajectories are ordered coordinate sequences, and reasoning is free-form text. The final decision is emitted within an
<answer>...</answer>tag.
Unified Capabilities
- Embodied Cognition & Spatial Reasoning — comprehends the semantic and spatial structure of the physical world, including static geometric relations and dynamic interaction possibilities.
- Embodied Planning & Correction — covers the full task life cycle: long-horizon task decomposition, next-step planning, process detection, error localization, and error correction.
- Embodied Pointing & Location — grounds high-level reasoning in coordinates and trajectories, covering referring expression grounding, region-level localization, functional (affordance) grounding, and visual trace generation.
Quick Start
from transformers import AutoModelForImageTextToText, AutoProcessor
from PIL import Image
model_id = "IffYuan/Embodied-R1.5"
model = AutoModelForImageTextToText.from_pretrained(
model_id, torch_dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained(model_id)
image = Image.open("scene.jpg")
messages = [
{
"role": "user",
"content": [
{"type": "image"},
{"type": "text", "text": "You are a robot performing manipulation tasks. "
"The task instruction is: move the blue cube on top of the yellow cube. "
"Use 2D points to mark the target location."},
],
}
]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
print(processor.batch_decode(out, skip_special_tokens=True)[0])
The model reasons over the visual observation and emits its final decision within an <answer> tag, e.g. <answer>[{"point_2d": [750, 748]}]</answer>.
Citation
@article{yuan2026embodiedr15,
title = {Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models},
author = {Yuan, Yifu and others},
year = {2026}
}
License
Released under the Apache 2.0 license.