Instructions to use SupraLabs/Supra-Router-51M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SupraLabs/Supra-Router-51M with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SupraLabs/Supra-Router-51M")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SupraLabs/Supra-Router-51M")
model = AutoModelForCausalLM.from_pretrained("SupraLabs/Supra-Router-51M")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SupraLabs/Supra-Router-51M with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SupraLabs/Supra-Router-51M"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SupraLabs/Supra-Router-51M",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/SupraLabs/Supra-Router-51M

SGLang

How to use SupraLabs/Supra-Router-51M with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SupraLabs/Supra-Router-51M" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SupraLabs/Supra-Router-51M",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SupraLabs/Supra-Router-51M" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SupraLabs/Supra-Router-51M",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use SupraLabs/Supra-Router-51M with Docker Model Runner:
```
docker model run hf.co/SupraLabs/Supra-Router-51M
```

Idea do modelo

by RexTRO111 - opened 1 day ago

Discussion

RexTRO111

1 day ago

Por favor, crie um modelo Supra-50M em português, chamado: Supra-50M-Portugues-Base (e Instruct), treinado em GigaVerbo, FineWeb2 e Wikipédia.

AxionLab-official

SupraLabs org 1 day ago

Vou pesquisar dados em portugues pra fazer

AxionLab-official changed discussion status to closed 1 day ago

RexTRO111

1 day ago

Espere um segundo, você vai usar CPT, finetune ou pré-treinar do zero?

QyrouNnet-AI

SupraLabs org about 24 hours ago

Todos os modelos da Supra são treinados do zero por nós!

E sim! Estamos trabalhando para tornar nossos modelos multilíngues. Por exemplo:

Inglês–Francês
Inglês–Português
Inglês–Japonês

E muitos outros...

Vamos construir tudo isso sobre os nossos modelos Supra-2 Base e disponibilizar os checkpoints como código aberto (open source) ❤️

QyrouNnet-AI changed discussion status to open about 24 hours ago

QyrouNnet-AI changed discussion status to closed about 24 hours ago

MishaGGG

about 14 hours ago

QyrouNnet-AI, please, somehow add the Russian language to the models ❤️

RexTRO111

about 6 hours ago

Certo, mas qual conjunto de dados você vai usar?

QyrouNnet-AI

SupraLabs org about 6 hours ago

Nós faremos o nosso próprio, e usaremos dados públicos de várias fontes com uma licença de código aberto.

RexTRO111

about 6 hours ago

Obrigado/a!

QyrouNnet-AI

SupraLabs org about 6 hours ago

@MishaGGG We are happy to look into this! Right now, our entire team is fully focused on the development of the SupraCode and Supra-2 models. However, we do plan to expand the upcoming Supra-2 models to support multiple languages. This process will take some time, so we truly appreciate your patience and support.

MishaGGG

about 6 hours ago

@QyrouNnet-AI Thanks for the reply. How many parameters will SupraCode have? And could you evaluate my model, VDrontGPT50m-Base (50 million parameters)? I was inspired by your Supra 50M model and decided to build my own from scratch.

QyrouNnet-AI

SupraLabs org about 4 hours ago

SupraCode is an app, but we will release specialize models from scratch and finetuned ones from qwen and gemma after we release the app.

And sure, I will look into your model and write a review as a discussion thread, @MishaGGG 😁

RexTRO111

about 4 hours ago

Srry. 5 tokens per parameter (250 millon tokens) 😟 Not comparable to supra 50m which was trained in 20b tokens (and supra 1.5 which was cpt'd in 3 billon tokens, so 23 billon tokens total for 1.5)

MishaGGG

about 3 hours ago

RexTRO111 supra 1.5 50m model, 23 bilion t, 460 tokens per param, wow, but according to Chinchilla optium there are 20 tokens per parameter?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment