Text Generation
Transformers
Safetensors
qwen3
Merge
taskvector
math
think
conversational
text-generation-inference
Instructions to use Montalte/qwen4b-math-think-taskvector with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Montalte/qwen4b-math-think-taskvector with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Montalte/qwen4b-math-think-taskvector") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Montalte/qwen4b-math-think-taskvector") model = AutoModelForCausalLM.from_pretrained("Montalte/qwen4b-math-think-taskvector", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Montalte/qwen4b-math-think-taskvector with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Montalte/qwen4b-math-think-taskvector" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Montalte/qwen4b-math-think-taskvector", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Montalte/qwen4b-math-think-taskvector
- SGLang
How to use Montalte/qwen4b-math-think-taskvector with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Montalte/qwen4b-math-think-taskvector" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Montalte/qwen4b-math-think-taskvector", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Montalte/qwen4b-math-think-taskvector" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Montalte/qwen4b-math-think-taskvector", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Montalte/qwen4b-math-think-taskvector with Docker Model Runner:
docker model run hf.co/Montalte/qwen4b-math-think-taskvector
File size: 3,929 Bytes
49105af | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 | {
"schema": "ddc-atomic-checkpoint-commit/1",
"runner_id": "THINK_B_2EP_SIX_ARMS_RUNNER_V3",
"completed_optimizer_updates": 977,
"cumulative_realized_active_tokens": 63999854,
"bindings": {
"config_sha256": "3f2dccc79ebaff962851b13c3bfad37d248bc2da8c44418520ccf30eb828f097",
"runner_code_sha256": "fd6d1c6403810acc4460e2ce0fb5f4418881387b11f4bd21f7cd21067faa3a0a",
"config_schema_sha256": "d6d13eeee42bf04cf34fc68b2fa7a61b316c4543b77d7d779815b8eddf10c099",
"python_binary_sha256": "5700a6a7804feb44da6c963ebd1323982478b7ca57478e4e81981c47818453bd",
"environment_lock_sha256": "627d4fe088b8f64841b0520ae5bd9dc5c81e9dcd5969b4bd45c9a9910b2f3c01",
"base_composite_sha256": "26cbdb468ced4891fee1fffa2be7b7722e5d8d836d0bc986eecd97fa12cd1747",
"base_lock_sha256": "9a145eebf5d9df466a5f554c4850290be671acd284e0f703b7dfdb087555858b",
"frozen_rows_sha256": "a3872b6c83623ad098a605056257f8414c451bb2beb8ba1025d1ca395de5806b",
"ordered_ids_sha256": "e4b37fe0b96d39ed6435a7e607b6a7df82c09c655af90cb2cecda752ecb1f892",
"update_plan_sha256": "43812327be005fdc9fcb08e193e82d306aacef1b740168dbd303cb0ea00f0ea5",
"data_receipt_sha256": "e66346fb413557929ab53648bc656fb3102557cf3460abb49c7b1c6e7281420d",
"decontamination_receipt_sha256": "66c360871b06208ed20e473e57ad0f09c79e4dac1b3124a9461a5b729b487153",
"render_receipt_sha256": "e66346fb413557929ab53648bc656fb3102557cf3460abb49c7b1c6e7281420d",
"token_bake_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b",
"continuation_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b"
},
"trainer_state": {
"path": "trainer_state.pt",
"bytes": 1586264781,
"sha256": "a46ffd5c3a6e3365bcf3863def07830b9324553e5dbca648600c3c42f5437908"
},
"token_boundary_ledger": {
"path": "TOKEN_BOUNDARY_LEDGER.json",
"bytes": 971964,
"sha256": "152b22ac803dd21f9d9cf581168d2a6116ce0aad8016a8871febc41380fcc2b3"
},
"adapter_files": [
{
"path": "adapter/README.md",
"bytes": 5414,
"sha256": "5480222f959c7f360c56310c9bfb8de0acfb45aa67a64c1bc9cafa38a7662226"
},
{
"path": "adapter/adapter_config.json",
"bytes": 1398,
"sha256": "df298fe17d86f237af9418474ceb14756ff38aa1e0a1c550608dea0bbc8a69f3"
},
{
"path": "adapter/adapter_model.safetensors",
"bytes": 528611960,
"sha256": "3723bbe789bf7b07edadb84dddab2de9b1a13652d3c0778284c358608522e2d6"
}
],
"serialization_canary": {
"save_load_format": "safetensors",
"state_key_count": 506,
"token_state": {
"schema": "think-b-2ep-token-rows/1",
"tensors": [
{
"key": "base_model.model.lm_head.token_adapter.trainable_tokens_delta",
"sha256": "6095bc98d1398df335b62c3661eba9fee45d0ca519408b22a3c6692ea264c8bd",
"shape": [
3,
2560
],
"dtype": "torch.float32"
},
{
"key": "base_model.model.model.embed_tokens.token_adapter.trainable_tokens_delta",
"sha256": "4c41034fce978c114587c727a962fb4fe6f83e338651fc008db26d93810e7f15",
"shape": [
3,
2560
],
"dtype": "torch.float32"
}
]
}
},
"token_state": {
"schema": "think-b-2ep-token-rows/1",
"tensors": [
{
"key": "base_model.model.lm_head.token_adapter.trainable_tokens_delta",
"sha256": "6095bc98d1398df335b62c3661eba9fee45d0ca519408b22a3c6692ea264c8bd",
"shape": [
3,
2560
],
"dtype": "torch.float32"
},
{
"key": "base_model.model.model.embed_tokens.token_adapter.trainable_tokens_delta",
"sha256": "4c41034fce978c114587c727a962fb4fe6f83e338651fc008db26d93810e7f15",
"shape": [
3,
2560
],
"dtype": "torch.float32"
}
]
}
}
|