Instructions to use CodeXomics/CodeXomics-ToolAgent-4B-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use CodeXomics/CodeXomics-ToolAgent-4B-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="CodeXomics/CodeXomics-ToolAgent-4B-v1")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("CodeXomics/CodeXomics-ToolAgent-4B-v1")
model = AutoModelForMultimodalLM.from_pretrained("CodeXomics/CodeXomics-ToolAgent-4B-v1", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use CodeXomics/CodeXomics-ToolAgent-4B-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "CodeXomics/CodeXomics-ToolAgent-4B-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CodeXomics/CodeXomics-ToolAgent-4B-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/CodeXomics/CodeXomics-ToolAgent-4B-v1

SGLang

How to use CodeXomics/CodeXomics-ToolAgent-4B-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "CodeXomics/CodeXomics-ToolAgent-4B-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CodeXomics/CodeXomics-ToolAgent-4B-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "CodeXomics/CodeXomics-ToolAgent-4B-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CodeXomics/CodeXomics-ToolAgent-4B-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use CodeXomics/CodeXomics-ToolAgent-4B-v1 with Docker Model Runner:
```
docker model run hf.co/CodeXomics/CodeXomics-ToolAgent-4B-v1
```

CodeXomics-ToolAgent-4B-v1 / codexomics-v5-benchmark-comparison.svg

CodeXomics

Upload codexomics-v5-benchmark-comparison.svg with huggingface_hub

27f4aff verified 3 days ago

preview code

Raw

History Blame Contribute Delete

8.21 kB

	<?xml version="1.0" encoding="utf-8"?>
	<!-- Generator: Adobe Illustrator 24.0.1, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
	<svg version="1.1" id="图层_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
	viewBox="0 0 960 600" style="enable-background:new 0 0 960 600;" xml:space="preserve">
	<style type="text/css">
	.st0{fill:#111827;}
	.st1{font-family:'ArialMT';}
	.st2{font-size:22px;}
	.st3{fill:#4F8EF7;}
	.st4{fill:#1F2937;}
	.st5{font-size:14px;}
	.st6{fill:#10B981;}
	.st7{font-size:15px;}
	.st8{fill:none;stroke:#E5E7EB;}
	.st9{fill:#231815;}
	.st10{font-family:'Helvetica-Bold';}
	.st11{font-size:12px;}
	.st12{fill:none;stroke:#D1D5DB;stroke-width:1.5;}
	.st13{fill:none;stroke:#4F8EF7;stroke-width:1.5;}
	.st14{fill:none;stroke:#10B981;stroke-width:1.5;}
	.st15{fill:#FFFFFF;}
	</style>
	<title>CodeXomics Benchmark: Base vs. Fine-Tuned 4B</title>
	<desc>
	Two-panel figure. Left: CodeXomics Benchmark pass rates. Base Qwen3.5-4B: simple 139/143, complex 26/29, total 165/172. Fine-tuned CodeXomics-ToolAgent-4B-v1: simple 143/143, complex 29/29, total 172/172. Right: inference latency in seconds with error bars, mean plus or minus SD over three runs: simple 9.2 vs 8.6, complex 25.8 vs 21.5, overall 12.0 vs 10.7.
	</desc>
	<text transform="matrix(1 0 0 1 235.2446 53)" class="st0 st1 st2">CodeXomics Benchmark: Base vs. Fine-Tuned 4B</text>
	<g>
	<path class="st3" d="M309,69h6c1.7,0,3,1.3,3,3v6c0,1.7-1.3,3-3,3h-6c-1.7,0-3-1.3-3-3v-6C306,70.3,307.3,69,309,69z"/>
	<text transform="matrix(1 0 0 1 326 80)" class="st4 st1 st5">Base Qwen3.5-4B</text>
	<path class="st6" d="M499,69h6c1.7,0,3,1.3,3,3v6c0,1.7-1.3,3-3,3h-6c-1.7,0-3-1.3-3-3v-6C496,70.3,497.3,69,499,69z"/>
	<text transform="matrix(1 0 0 1 516 80)" class="st4 st1 st5">CodeXomics-ToolAgent-4B-v1 </text>
	</g>
	<text transform="matrix(1 0 0 1 186.2256 107)" class="st0 st1 st7">Benchmark accuracy (%)</text>
	<g>
	<g>
	<line class="st8" x1="70" y1="145" x2="470" y2="145"/>
	<text transform="matrix(1 0 0 1 29.3086 149)" class="st9 st10 st11">100%</text>
	<line class="st8" x1="70" y1="242.5" x2="470" y2="242.5"/>
	<text transform="matrix(1 0 0 1 35.9824 246.5)" class="st9 st10 st11">75%</text>
	<line class="st8" x1="70" y1="340" x2="470" y2="340"/>
	<text transform="matrix(1 0 0 1 35.9824 344)" class="st9 st10 st11">50%</text>
	<line class="st8" x1="70" y1="437.5" x2="470" y2="437.5"/>
	<text transform="matrix(1 0 0 1 35.9824 441.5)" class="st9 st10 st11">25%</text>
	<line class="st12" x1="70" y1="535" x2="470" y2="535"/>
	<text transform="matrix(1 0 0 1 42.6562 539)" class="st9 st10 st11">0%</text>
	</g>
	<g>
	<path class="st3" d="M110,155.9h38c1.7,0,3,1.3,3,3V532c0,1.7-1.3,3-3,3h-38c-1.7,0-3-1.3-3-3V158.9
	C107,157.2,108.3,155.9,110,155.9z"/>
	<text transform="matrix(1 0 0 1 107.3115 149.9)" class="st0 st10 st11">139/143</text>
	<path class="st6" d="M154,145h38c1.7,0,3,1.3,3,3v384c0,1.7-1.3,3-3,3h-38c-1.7,0-3-1.3-3-3V148C151,146.3,152.3,145,154,145z"/>
	<text transform="matrix(1 0 0 1 151.3115 139)" class="st0 st10 st11">143/143</text>
	<path class="st3" d="M243,185.2h38c1.7,0,3,1.3,3,3V532c0,1.7-1.3,3-3,3h-38c-1.7,0-3-1.3-3-3V188.2
	C240,186.5,241.3,185.2,243,185.2z"/>
	<text transform="matrix(1 0 0 1 246.9854 179.2)" class="st0 st10 st11">26/29</text>
	<path class="st6" d="M287,145h38c1.7,0,3,1.3,3,3v384c0,1.7-1.3,3-3,3h-38c-1.7,0-3-1.3-3-3V148C284,146.3,285.3,145,287,145z"/>
	<text transform="matrix(1 0 0 1 290.9854 139)" class="st0 st10 st11">29/29</text>
	<path class="st3" d="M376,161h38c1.7,0,3,1.3,3,3v368c0,1.7-1.3,3-3,3h-38c-1.7,0-3-1.3-3-3V164C373,162.3,374.3,161,376,161z"/>
	<text transform="matrix(1 0 0 1 373.3115 155)" class="st0 st10 st11">165/172</text>
	<path class="st6" d="M420,145h38c1.7,0,3,1.3,3,3v384c0,1.7-1.3,3-3,3h-38c-1.7,0-3-1.3-3-3V148C417,146.3,418.3,145,420,145z"/>
	<text transform="matrix(1 0 0 1 417.3115 139)" class="st0 st10 st11">172/172</text>
	</g>
	<g>
	<text transform="matrix(1 0 0 1 115.6035 557)" class="st4 st1 st5">Simple</text>
	<text transform="matrix(1 0 0 1 242.3794 557)" class="st4 st1 st5">Complex</text>
	<text transform="matrix(1 0 0 1 388.2139 557)" class="st4 st1 st5">Total</text>
	</g>
	</g>
	<text transform="matrix(1 0 0 1 625.4224 107)" class="st0 st1 st7">Inference latency (s, mean ± SD)</text>
	<g>
	<g>
	<line class="st8" x1="540" y1="145" x2="930" y2="145"/>
	<text transform="matrix(1 0 0 1 507.3184 149)" class="st9 st10 st11">30 s</text>
	<line class="st8" x1="540" y1="275" x2="930" y2="275"/>
	<text transform="matrix(1 0 0 1 507.3184 279)" class="st9 st10 st11">20 s</text>
	<line class="st8" x1="540" y1="405" x2="930" y2="405"/>
	<text transform="matrix(1 0 0 1 507.3184 409)" class="st9 st10 st11">10 s</text>
	<line class="st12" x1="540" y1="535" x2="930" y2="535"/>
	<text transform="matrix(1 0 0 1 513.9922 539)" class="st9 st10 st11">0 s</text>
	</g>
	<g>
	<path class="st3" d="M578,415.4h38c1.7,0,3,1.3,3,3V532c0,1.7-1.3,3-3,3h-38c-1.7,0-3-1.3-3-3V418.4
	C575,416.7,576.3,415.4,578,415.4z"/>
	<line class="st13" x1="597" y1="414.1" x2="597" y2="416.7"/>
	<line class="st13" x1="591" y1="414.1" x2="603" y2="414.1"/>
	<line class="st13" x1="591" y1="416.7" x2="603" y2="416.7"/>
	<text transform="matrix(1 0 0 1 588.6592 409.4)" class="st0 st10 st11">9.2</text>
	<path class="st6" d="M622,423.2h38c1.7,0,3,1.3,3,3V532c0,1.7-1.3,3-3,3h-38c-1.7,0-3-1.3-3-3V426.2
	C619,424.5,620.3,423.2,622,423.2z"/>
	<line class="st14" x1="641" y1="414.1" x2="641" y2="432.3"/>
	<line class="st14" x1="635" y1="414.1" x2="647" y2="414.1"/>
	<line class="st14" x1="635" y1="432.3" x2="647" y2="432.3"/>
	<text transform="matrix(1 0 0 1 632.6592 417.2)" class="st0 st10 st11">8.6</text>
	<path class="st3" d="M708,199.6h38c1.7,0,3,1.3,3,3V532c0,1.7-1.3,3-3,3h-38c-1.7,0-3-1.3-3-3V202.6
	C705,200.9,706.3,199.6,708,199.6z"/>
	<line class="st13" x1="727" y1="195.7" x2="727" y2="203.5"/>
	<line class="st13" x1="721" y1="195.7" x2="733" y2="195.7"/>
	<line class="st13" x1="721" y1="203.5" x2="733" y2="203.5"/>
	<text transform="matrix(1 0 0 1 715.3223 193.6)" class="st0 st10 st11">25.8</text>
	<path class="st6" d="M752,255.5h38c1.7,0,3,1.3,3,3V532c0,1.7-1.3,3-3,3h-38c-1.7,0-3-1.3-3-3V258.5
	C749,256.8,750.3,255.5,752,255.5z"/>
	<line class="st14" x1="771" y1="230.8" x2="771" y2="280.2"/>
	<line class="st14" x1="765" y1="230.8" x2="777" y2="230.8"/>
	<line class="st14" x1="765" y1="280.2" x2="777" y2="280.2"/>
	<text transform="matrix(1 0 0 1 759.3223 249.5)" class="st0 st10 st11">21.5</text>
	<path class="st3" d="M838,379h38c1.7,0,3,1.3,3,3v150c0,1.7-1.3,3-3,3h-38c-1.7,0-3-1.3-3-3V382C835,380.3,836.3,379,838,379z"/>
	<line class="st13" x1="857" y1="377.7" x2="857" y2="380.3"/>
	<line class="st13" x1="851" y1="377.7" x2="863" y2="377.7"/>
	<line class="st13" x1="851" y1="380.3" x2="863" y2="380.3"/>
	<text transform="matrix(1 0 0 1 845.3223 373)" class="st0 st10 st11">12.0</text>
	<path class="st6" d="M882,395.9h38c1.7,0,3,1.3,3,3V532c0,1.7-1.3,3-3,3h-38c-1.7,0-3-1.3-3-3V398.9
	C879,397.2,880.3,395.9,882,395.9z"/>
	<line class="st14" x1="901" y1="384.2" x2="901" y2="407.6"/>
	<line class="st14" x1="895" y1="384.2" x2="907" y2="384.2"/>
	<line class="st14" x1="895" y1="407.6" x2="907" y2="407.6"/>
	<text transform="matrix(1 0 0 1 889.3223 389.9)" class="st0 st10 st11">10.7</text>
	</g>
	<g>
	<text transform="matrix(1 0 0 1 583.6035 557)" class="st4 st1 st5">Simple</text>
	<text transform="matrix(1 0 0 1 707.3794 557)" class="st4 st1 st5">Complex</text>
	<text transform="matrix(1 0 0 1 842.8276 557)" class="st4 st1 st5">Overall</text>
	</g>
	</g>
	<text transform="matrix(1 0 0 1 112.6035 327.6742)" class="st15 st10 st11">97.2%</text>
	<text transform="matrix(1 0 0 1 245.6035 327.6742)" class="st15 st10 st11">89.7%</text>
	<text transform="matrix(1 0 0 1 152.3402 289.3988)" class="st15 st10 st11">100.0%</text>
	<text transform="matrix(1 0 0 1 285.0032 289.3983)" class="st15 st10 st11">100.0%</text>
	<text transform="matrix(1 0 0 1 419.0032 289.3983)" class="st15 st10 st11">100.0%</text>
	<text transform="matrix(1 0 0 1 377.3402 328.3988)" class="st15 st10 st11">95.9%</text>
	</svg>