Instructions to use experimentalmachines/QwenGrad-DPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use experimentalmachines/QwenGrad-DPO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="experimentalmachines/QwenGrad-DPO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("experimentalmachines/QwenGrad-DPO")
model = AutoModelForCausalLM.from_pretrained("experimentalmachines/QwenGrad-DPO", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use experimentalmachines/QwenGrad-DPO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "experimentalmachines/QwenGrad-DPO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "experimentalmachines/QwenGrad-DPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/experimentalmachines/QwenGrad-DPO

SGLang

How to use experimentalmachines/QwenGrad-DPO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "experimentalmachines/QwenGrad-DPO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "experimentalmachines/QwenGrad-DPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "experimentalmachines/QwenGrad-DPO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "experimentalmachines/QwenGrad-DPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use experimentalmachines/QwenGrad-DPO with Docker Model Runner:
```
docker model run hf.co/experimentalmachines/QwenGrad-DPO
```

QwenGrad-DPO — M1 DPO calibration on selected M0-final-v2

Direct Preference Optimization applied to the healthy selected M0-final-v2 checkpoint of the OpenGrad tool-use study — not to the base model, and not to a historical DPO checkpoint.

Research artifact, not a production model.

This repository is published by Experimental Machines. It contains the promoted dpo-checkpoint-30 at the repository root, in standard transformers layout.


Original release	`arrochi112/OpenGrad-Qwen3.5-2B-M1-DPO-CanonicalV2-Final-v2` at revision `f33d20308982f37deb459076f489e794d5521ee3` (also holds checkpoints 60/90/120)
Base model	`Qwen/Qwen3.5-2B` at revision `15852e8c16360a2fea060d615a32b45270f8a8fc`
Selection	checkpoint 30 of 120, frozen DEV balanced selector
Promotion	`PROMOTED` under the prospective `tool_use_promotion_v4` policy

Results (pre-registered internal confirmatory partition, 1,277 examples)

The chart reports the exact values for call_f1, precision, recall, over_call, clarification, and unsupported. Higher is better for every metric except over_call.

M1 preserves the M0 calibrated frontier and makes a small improvement in call F1 and recall. It does not materially reduce over-calling; this is calibration retention and slight improvement, not a large frontier movement.

Evaluation and promotion

Checkpoint selection used the frozen DEV partition (2,373 examples, fingerprint 88a56821…). Checkpoints 30/60 were within the pre-registered 0.01 macro tolerance, so the earlier checkpoint 30 was selected. The confirmatory partition (fingerprint d6d1e394…) was then scored exactly once on checkpoint 30.

The prospective tool_use_promotion_v4 policy passed: precision, recall, and macro floors; over-call ceiling; clarification and unsupported floors; parse validity; and M0-relative regression checks. Checkpoint 30 is PROMOTED. This policy does not compare recall to B0's degenerate always-call recall.

The confirmatory partition is pre-registered internal evidence, not an untouched external test. The evaluation population has no ANSWER examples, so no_call_accuracy is NA. Tool-selection accuracy, argument validity, and schema validity are not computed by the current evaluator and are not treated as satisfied.

Frozen lineage

Parent experiment: m0_sft_canonical_v2_final
Parent checkpoint: checkpoint-1800
Parent model hash: 7144579aeecec8b4de25f193ab63085efdf8d9d76b85ed915352291b0152277a
Preference dataset: 481 local calibration pairs, hash d39168948d09fc3c355cd83f9f0857f310086322b0968fd2e7d78125150faef4 (m1_calibration_preference_pairs_v1)
DPO: beta 0.05, learning rate 5e-7, cosine schedule, 120 steps, seed 42, bfloat16
Training commit: bb15c7e0181c7f526875cd1426672431ae14bd83

The preference set combines deterministic base/M0 disagreements on Canonical-v2 training prompts with a bounded curated When2Call training slice. Frozen behavioral evaluation IDs were excluded; no paid external API was used. All pair origins and input hashes are recorded in the OpenGrad repository.

Intended use

Research artifact. Not safety-tuned, not aligned, and not intended for autonomous tool use. It inherits the limitations of its public training sources and of its evaluator's unmeasured dimensions. The base model's own license and terms continue to apply to these weights.

Provenance

The model card text above mirrors the original OpenGrad release card for this checkpoint; only the repository-level framing (mirror, Experimental Machines) has been added. The evidence behind every number is in the OpenGrad repository:

Execution: reports/M1_DPO_EXECUTION_REPORT.md
Evaluation: reports/M1_DPO_EVALUATION.md
Machine-readable status: docs/EXPERIMENT_STATUS.md
Next-stage decision: reports/M2_DECISION.md

Published by Experimental Machines.

Downloads last month: 497

Safetensors

Model size

2B params

Tensor type

BF16

Model tree for experimentalmachines/QwenGrad-DPO

Base model

Qwen/Qwen3.5-2B-Base

Finetuned

Qwen/Qwen3.5-2B

Finetuned

(372)

this model

experimentalmachines
/

QwenGrad-DPO