Video-Text-to-Text
Transformers
Safetensors
English
moss_vl
feature-extraction
Realtime
Streaming
Video-Understanding
Image-Understanding
MOSS-VL
OpenMOSS
multimodal
video
vision-language
custom_code
Instructions to use OpenMOSS-Team/MOSS-VL-Realtime with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenMOSS-Team/MOSS-VL-Realtime with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("OpenMOSS-Team/MOSS-VL-Realtime", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -140,7 +140,7 @@ Common session operations:
|
|
| 140 |
|
| 141 |
`system_prompt` and `initial_prompt` are tokenized as the initial system/user turns before the first frame arrives. Subsequent user turns can be appended with `push_prompt(...)` while the same session continues observing frames.
|
| 142 |
|
| 143 |
-
For
|
| 144 |
|
| 145 |
```python
|
| 146 |
import time
|
|
@@ -349,8 +349,8 @@ print(texts)
|
|
| 349 |
| Model | Parameters | Context | Usage | Hugging Face |
|
| 350 |
| --- | ---: | ---: | --- | --- |
|
| 351 |
| MOSS-VL-Realtime | 11B | 256K | Realtime streaming video interaction | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Realtime |
|
| 352 |
-
| MOSS-VL-Instruct | 11B | 256K | Offline multimodal instruction following | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Instruct
|
| 353 |
-
| MOSS-VL-Base | 11B | 256K | Continued pretraining and fine-tuning | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Base
|
| 354 |
| MOSS-VL-Instruct-0408 | 11B | 256K | Previous instruction-tuned checkpoint | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Instruct-0408 |
|
| 355 |
| MOSS-VL-Base-0408 | 11B | 256K | Previous base checkpoint | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Base-0408 |
|
| 356 |
|
|
|
|
| 140 |
|
| 141 |
`system_prompt` and `initial_prompt` are tokenized as the initial system/user turns before the first frame arrives. Subsequent user turns can be appended with `push_prompt(...)` while the same session continues observing frames.
|
| 142 |
|
| 143 |
+
For complete real-time inference usage, including local-video replay and service deployment, see [`realtime_inference`](https://github.com/OpenMOSS/MOSS-VL/tree/main/realtime_inference) in the MOSS-VL GitHub repository.
|
| 144 |
|
| 145 |
```python
|
| 146 |
import time
|
|
|
|
| 349 |
| Model | Parameters | Context | Usage | Hugging Face |
|
| 350 |
| --- | ---: | ---: | --- | --- |
|
| 351 |
| MOSS-VL-Realtime | 11B | 256K | Realtime streaming video interaction | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Realtime |
|
| 352 |
+
| MOSS-VL-Instruct | 11B | 256K | Offline multimodal instruction following | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Instruct |
|
| 353 |
+
| MOSS-VL-Base | 11B | 256K | Continued pretraining and fine-tuning | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Base |
|
| 354 |
| MOSS-VL-Instruct-0408 | 11B | 256K | Previous instruction-tuned checkpoint | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Instruct-0408 |
|
| 355 |
| MOSS-VL-Base-0408 | 11B | 256K | Previous base checkpoint | https://huggingface.co/OpenMOSS-Team/MOSS-VL-Base-0408 |
|
| 356 |
|