Instructions to use SuhaanCoding/Wan2.2-S2V-14B-Diffusers with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use SuhaanCoding/Wan2.2-S2V-14B-Diffusers with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("SuhaanCoding/Wan2.2-S2V-14B-Diffusers", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Wan-AI/Wan2.2-S2V-14B
|
| 4 |
+
pipeline_tag: image-to-video
|
| 5 |
+
---
|
| 6 |
+
|
| 7 |
+
# Wan2.2-S2V-14B (Diffusers layout)
|
| 8 |
+
|
| 9 |
+
This is [Wan-AI/Wan2.2-S2V-14B](https://huggingface.co/Wan-AI/Wan2.2-S2V-14B)
|
| 10 |
+
— a model that animates a picture of a person in sync with a voice recording —
|
| 11 |
+
repacked into the folder layout that
|
| 12 |
+
[FastVideo](https://github.com/hao-ai-lab/FastVideo) loads.
|
| 13 |
+
|
| 14 |
+
**No weights were changed.** The video model tensors are byte-identical copies
|
| 15 |
+
of the official release. The text encoder, VAE, tokenizer and scheduler come
|
| 16 |
+
from Wan-AI's own Diffusers release of Wan2.1 (the same weights S2V uses), and
|
| 17 |
+
the wav2vec2 speech encoder is the one bundled inside the official S2V repo.
|
| 18 |
+
|
| 19 |
+
Made with
|
| 20 |
+
[`scripts/checkpoint_conversion/wan_s2v_to_diffusers.py`](https://github.com/hao-ai-lab/FastVideo/pull/1683)
|
| 21 |
+
from FastVideo PR #1683.
|
| 22 |
+
|
| 23 |
+
## Verification status
|
| 24 |
+
|
| 25 |
+
- Checked: all 1260 video-model tensors load into FastVideo's model with the
|
| 26 |
+
correct names and shapes (the conversion script verifies this automatically
|
| 27 |
+
and refuses to produce output otherwise).
|
| 28 |
+
- Not yet checked: end-to-end video generation on a GPU. If you run it and
|
| 29 |
+
find problems, please comment on the PR above.
|
| 30 |
+
|
| 31 |
+
## Usage (with FastVideo, once PR #1683 is merged)
|
| 32 |
+
|
| 33 |
+
```python
|
| 34 |
+
from fastvideo import VideoGenerator
|
| 35 |
+
|
| 36 |
+
gen = VideoGenerator.from_pretrained("SuhaanCoding/Wan2.2-S2V-14B-Diffusers")
|
| 37 |
+
gen.generate_video(
|
| 38 |
+
prompt="...",
|
| 39 |
+
image_path="face.jpg",
|
| 40 |
+
audio_path="speech.wav",
|
| 41 |
+
output_path="out.mp4",
|
| 42 |
+
save_video=True,
|
| 43 |
+
)
|
| 44 |
+
```
|