Instructions to use utter-project/EuroVLM-9B-Preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use utter-project/EuroVLM-9B-Preview with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="utter-project/EuroVLM-9B-Preview")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("utter-project/EuroVLM-9B-Preview")
model = AutoModelForImageTextToText.from_pretrained("utter-project/EuroVLM-9B-Preview")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use utter-project/EuroVLM-9B-Preview with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "utter-project/EuroVLM-9B-Preview"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "utter-project/EuroVLM-9B-Preview",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/utter-project/EuroVLM-9B-Preview

SGLang

How to use utter-project/EuroVLM-9B-Preview with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "utter-project/EuroVLM-9B-Preview" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "utter-project/EuroVLM-9B-Preview",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "utter-project/EuroVLM-9B-Preview" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "utter-project/EuroVLM-9B-Preview",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use utter-project/EuroVLM-9B-Preview with Docker Model Runner:
```
docker model run hf.co/utter-project/EuroVLM-9B-Preview
```

Critical requirement for upcoming releases: Prioritization of European linguistic variants

by galisep - opened Feb 13

Discussion

galisep

Feb 13

Hi,

As you prepare for the upcoming model releases, I must highlight a critical issue observed in the current versions that needs to be addressed.

Currently, the models consistently default to Brazilian Portuguese (pt-BR) regarding syntax, vocabulary, and grammar.

It is crucial to emphasize that pt-BR is not an official language of the European Union. European Portuguese (pt-PT) is.

For a project explicitly branded as a "European LLM" with the mission of preserving European linguistic sovereignty, it is imperative that the models prioritize European linguistic variants (e.g., pt-PT) over non-European ones.

Please ensure that the training data and alignment processes for the new models are corrected to reflect this. The models should default to European Portuguese standards, as defaulting to a non-EU variant contradicts the core purpose of this initiative.

We look forward to seeing this corrected in the next release.

andre-martins

UTTER - Unified Transcription and Translation for Extended Reality org Feb 13

Dear @galisep

Thanks for your feedback. This is not a critical issue - European and Brazilian Portuguese are two variants of the same language (Portuguese) which is one of the 24 official EU languages. However, we agree that it is useful for the model for distinguish between language variants. In fact, we are working on a separate project whose aim is precisely tuning EuroLLM for European Portuguese. Stay tuned as we plan to release that model soon. In the meantime, we recommend that you request EuroLLM to answer in European Portuguese in the system prompt if this is your intended use, which doesn't completely solve but mitigates the issue you point out.

André

andre-martins changed discussion status to closed Feb 13

galisep

Feb 13

Hi André, thanks for the reply.

While I appreciate that a specific fine-tune is in the works, I must respectfully disagree that the current behavior is "not a critical issue" for a project named EuroLLM.

The problem isn't that the model knows Brazilian Portuguese (which is great); the problem is that an EU-funded/centric model treats a non-EU variant as the default.

Logic dictates that a model's alignment should reflect its origin and purpose. It is inconceivable that a South American LLM initiative (e.g., Mercosur) would release a foundational model that defaults to European Portuguese syntax and grammar. They would rightfully prioritize their own linguistic reality.

If this project aims to represent European digital sovereignty, the default variant for any pluricentric language should be the European one. European Portuguese shouldn't be relegated to a "separate project" or require specific prompting to override a non-European default—it should be the baseline standard for EuroLLM.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment