UNICA: A Unified Neural Framework for Controllable 3D Avatars
Paper • 2604.02799 • Published • 1
import torch
from diffusers import DiffusionPipeline
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("zjh21/UNICA", dtype=torch.bfloat16, device_map="cuda")
prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]UNICA: A Unified Neural Framework for Controllable 3D Avatars
Abstract
Controllable 3D human avatars have found widespread applications in 3D games, the metaverse, and AR/VR scenarios. The conventional approach to creating such a 3D avatar requires a lengthy, intricate pipeline encompassing appearance modeling, motion planning, rigging, and physical simulation. In this paper, we introduce UNICA (UNIfied neural Controllable Avatar), a skeleton-free generative model that unifies all avatar control components into a single neural framework. Given keyboard inputs akin to video game controls, UNICA generates the next frame of a 3D avatar's geometry through an action-conditioned diffusion model operating on 2D position maps. A point transformer then maps the resulting geometry to 3D Gaussian Splatting for high-fidelity free-view rendering. Our approach naturally captures hair and loose clothing dynamics without manually designed physical simulation, and supports extra-long autoregressive generation.
Resources
- Paper: UNICA: A Unified Neural Framework for Controllable 3D Avatars
- GitHub Repository: https://github.com/zjh21/UNICA
Installation and Usage
Please refer to the official GitHub repository for detailed installation instructions and inference scripts. The pipeline generally involves two stages:
- Geometry Generation: Using the action-conditioned diffusion model to generate position maps.
- Appearance Mapping: Mapping geometry to 3D Gaussian Splatting via a point transformer for rendering.
- Downloads last month
- -
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support