Align model card with GitHub repo (arXiv, project page, demo, TODO, Apache-2.0)
Browse files
README.md
CHANGED
|
@@ -1,16 +1,17 @@
|
|
| 1 |
---
|
| 2 |
-
license:
|
| 3 |
language:
|
| 4 |
- multilingual
|
| 5 |
-
pipeline_tag:
|
| 6 |
tags:
|
|
|
|
| 7 |
- hunyuan
|
| 8 |
- hunyuan_vl_mot
|
| 9 |
- unified_mot
|
| 10 |
- vision-language
|
| 11 |
- Embodied
|
|
|
|
| 12 |
- image-to-text
|
| 13 |
-
- any-to-any
|
| 14 |
- MoT
|
| 15 |
- flow-matching
|
| 16 |
---
|
|
@@ -18,11 +19,14 @@ tags:
|
|
| 18 |
<div align="center">
|
| 19 |
<h1>RxBrain</h1>
|
| 20 |
<p><b>Embodied Cognition Foundation Model with Joint Language–Visual Reasoning and Imagination</b></p>
|
| 21 |
-
<p><i>Tencent Robotics X × Futian Laboratory × Tencent
|
| 22 |
|
| 23 |
-
<a href="https://
|
| 24 |
-
<a href="https://huggingface.co/tencent/Hy-Embodied-RxBrain-1.0"><img src="https://img.shields.io/badge/
|
| 25 |
-
<a href="https://
|
|
|
|
|
|
|
|
|
|
| 26 |
|
| 27 |
</div>
|
| 28 |
|
|
@@ -54,12 +58,11 @@ These capabilities are unified through **interleaved generation**: within a sing
|
|
| 54 |
* 🎨 **Flow-Matching Image Head:** Imagined frames are produced by a flow-matching head decoding into a frozen **FLUX** VAE latent space, enabling text-to-image, multi-frame world-model rollout, and goal-image planning.
|
| 55 |
* 🔗 **Interleaved Reasoning + Imagination:** Text reasoning and generated frames are emitted in one sequence, coupling symbolic plans with visual goals.
|
| 56 |
|
| 57 |
-
##
|
| 58 |
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
- [ ] Online Gradio Demo
|
| 63 |
|
| 64 |
## 🛠️ Dependencies and Installation
|
| 65 |
|
|
@@ -196,5 +199,5 @@ python interleave_inference.py \
|
|
| 196 |
|
| 197 |
## 📊 Evaluation
|
| 198 |
|
| 199 |
-
RxBrain is evaluated on embodied understanding, spatial reasoning, and imagination/generation benchmarks. For detailed metrics and methodology, please refer to our [technical report](https://
|
| 200 |
|
|
|
|
| 1 |
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
language:
|
| 4 |
- multilingual
|
| 5 |
+
pipeline_tag: any-to-any
|
| 6 |
tags:
|
| 7 |
+
- any-to-any
|
| 8 |
- hunyuan
|
| 9 |
- hunyuan_vl_mot
|
| 10 |
- unified_mot
|
| 11 |
- vision-language
|
| 12 |
- Embodied
|
| 13 |
+
- image-text-to-text
|
| 14 |
- image-to-text
|
|
|
|
| 15 |
- MoT
|
| 16 |
- flow-matching
|
| 17 |
---
|
|
|
|
| 19 |
<div align="center">
|
| 20 |
<h1>RxBrain</h1>
|
| 21 |
<p><b>Embodied Cognition Foundation Model with Joint Language–Visual Reasoning and Imagination</b></p>
|
| 22 |
+
<p><i>Tencent Robotics X × Futian Laboratory × Tencent Hy Team</i></p>
|
| 23 |
|
| 24 |
+
<a href="https://arxiv.org/abs/2607.14187"><img src="https://img.shields.io/badge/arXiv-2607.14187-b31b1b.svg" alt="arXiv"></a>
|
| 25 |
+
<a href="https://huggingface.co/tencent/Hy-Embodied-RxBrain-1.0"><img src="https://img.shields.io/badge/🤗%20HuggingFace-Model-FFD21E" alt="Model"></a>
|
| 26 |
+
<a href="https://tairos.tencent.com/openSourceModels/hy-embodied-rxbrain-1.0"><img src="https://img.shields.io/badge/🌐%20Project-Page-2ea44f" alt="Project Page"></a>
|
| 27 |
+
<a href="https://huggingface.co/spaces/hugging-apps/rxbrain-embodied-cognition"><img src="https://img.shields.io/badge/🤗%20HuggingFace-Demo-FFD21E" alt="Demo"></a>
|
| 28 |
+
<a href="https://github.com/Tencent-Hunyuan/Hy-Embodied-RxBrain-1.0"><img src="https://img.shields.io/badge/GitHub-Code-181717?logo=github&logoColor=white" alt="GitHub"></a>
|
| 29 |
+
<a href="./LICENSE"><img src="https://img.shields.io/badge/License-Apache_2.0-blue.svg" alt="License"></a>
|
| 30 |
|
| 31 |
</div>
|
| 32 |
|
|
|
|
| 58 |
* 🎨 **Flow-Matching Image Head:** Imagined frames are produced by a flow-matching head decoding into a frozen **FLUX** VAE latent space, enabling text-to-image, multi-frame world-model rollout, and goal-image planning.
|
| 59 |
* 🔗 **Interleaved Reasoning + Imagination:** Text reasoning and generated frames are emitted in one sequence, coupling symbolic plans with visual goals.
|
| 60 |
|
| 61 |
+
## 📋 TODO
|
| 62 |
|
| 63 |
+
* [ ] Open-source **RxBrain-Bench**, a comprehensive benchmark for embodied cognition.
|
| 64 |
+
* [ ] Open-source the fine-tuning code for **Visual Question Answering (VQA)**, **Multi Frames Generation**, and **Interleave Generation**.
|
| 65 |
+
* [x] Release the **technical report**, **inference code**, and **model weights**.
|
|
|
|
| 66 |
|
| 67 |
## 🛠️ Dependencies and Installation
|
| 68 |
|
|
|
|
| 199 |
|
| 200 |
## 📊 Evaluation
|
| 201 |
|
| 202 |
+
RxBrain is evaluated on embodied understanding, spatial reasoning, and imagination/generation benchmarks. For detailed metrics and methodology, please refer to our [technical report](https://arxiv.org/abs/2607.14187).
|
| 203 |
|