Instructions to use IvmeLabs/Ivme-Conversate-v3-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use IvmeLabs/Ivme-Conversate-v3-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="IvmeLabs/Ivme-Conversate-v3-Base", trust_remote_code=True)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("IvmeLabs/Ivme-Conversate-v3-Base", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use IvmeLabs/Ivme-Conversate-v3-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "IvmeLabs/Ivme-Conversate-v3-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IvmeLabs/Ivme-Conversate-v3-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/IvmeLabs/Ivme-Conversate-v3-Base

SGLang

How to use IvmeLabs/Ivme-Conversate-v3-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "IvmeLabs/Ivme-Conversate-v3-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IvmeLabs/Ivme-Conversate-v3-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "IvmeLabs/Ivme-Conversate-v3-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IvmeLabs/Ivme-Conversate-v3-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use IvmeLabs/Ivme-Conversate-v3-Base with Docker Model Runner:
```
docker model run hf.co/IvmeLabs/Ivme-Conversate-v3-Base
```

Factorized embeddings and 2-pass recycling on micro-parameter budgets (Maba)

by AndrewThompson1233 - opened 4 days ago

Discussion

AndrewThompson1233

4 days ago

Hi IvmeLabs team,

Really enjoyed reading through the v3 card. Pushing past Chinchilla on 15B tokens with Muon on a single Blackwell workstation and digging into XSA (arXiv:2603.09078) is great engineering.

I noticed two specific constraints you hit while balancing the budget:

Vocab tax: Even after narrowing to 16k vocab, embeddings still eat 20.7% of your 24.8M parameters (~5.1M weights), locking capacity that could otherwise go into transformers blocks.

Depth vs width: You pushed to 16 layers for coherence, but were capped by total parameter overhead.

I've been working on an open architecture called Maba that tackles these exact bottlenecks on small compute budgets:

Factorized embeddings: Rank-projected lookup drops the vocab parameter tax to around 4.3% of the total budget, freeing up several million parameters strictly for hidden layers.

2-pass physical block recycling: Passes representations through the physical blocks twice with layer-index conditioning, giving 40 effective layers from a 20-block parameter footprint.

Sub-quadratic scaling: 75% GDN-2 linear recurrence (O(1) state memory) paired with 25% GQA pushes context past 1024 up to 32k-131k tokens without KV-cache explosion.

Native Muon + AdamW training support out of the box.

The PyTorch reference implementation, specs, and weights are here:
https://huggingface.co/AndrewThompson1233/maba-v1-architecture

If you run an exploratory v4 run or want to test low-rank factorized embeddings to claw back that 20.7% vocab overhead, take a look. Curious what you think of block recycling on micro models?

Best,

Andrew

ereniko

İvmeLabs org 3 days ago

Hi Andrew,

Your architecture is really interesting and I'm sure it could help our models so thank you for your interest and recommendations.

But as IvmeLabs we like to stay on more verified or proven architectures that we think are suitable for our models, and because of that I sadly have to turn down your recommendation.

Maybe we could test your architecture in our ExpIvme line someday but for now, we would like to stay on more stable and proven gains.

Best,

Eren

ereniko changed discussion status to closed 3 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment