Model card: quickstart (vLLM + OpenAI client + transformers), serving gotchas, training details, limitations

README.md

@@ -1,28 +1,199 @@
 ---
 license: apache-2.0
 base_model: Qwen/Qwen3.6-27B
+base_model_relation: finetune
+pipeline_tag: text-generation
+library_name: transformers
+language:
+- en
+- vi
 tags:
 - function-calling
 - tool-use
+- tool-calling
 - agent
+- agentic
+- rl
 - grpo
+- lora
 - intuitor
 - self-certainty
-- qwen3_5
-pipeline_tag: text-generation
-library_name: transformers
+- vllm
+- qwen3
+- vietnamese
+- enterprise-automation
 ---
 
-# Opera
+# Opera — a 27B agent tuned for multi-turn tool calling
+
+**Opera** is an RL fine-tune of **Qwen3.6-27B** for *agentic* tool-calling: multi-turn
+episodes where the model has to look up a policy, read messy free-text input, compute an
+exact number, and then call the right tool with the right arguments — not just emit one
+well-formed function call.
+
+It was trained with **GRPO + LoRA** and a composite reward, including
+[Intuitor self-certainty](https://arxiv.org/abs/2505.19590) as an intrinsic signal
+alongside rule-based correctness.
 
-RL fine-tune of **Qwen3.6-27B** for agentic tool-calling, trained with GRPO + LoRA
-and a composite reward: rule-based (function name + argument-subset partial credit),
-structural tool-call validity, and **Intuitor self-certainty** (arXiv:2505.19590),
-rule and self-certainty weighted 1.0 each (normalized separately, then combined).
+**Use it if you need:** an open-weight agent backend for business automation
+(accounting, HR, operations) that holds up over long tool-use episodes, and that works on
+Vietnamese input out of the box.
 
+| | |
+|---|---|
+| Base | `Qwen/Qwen3.6-27B` (dense, VL architecture) |
+| Method | GRPO + LoRA (r32 / α64), merged into full weights |
+| Precision / size | bf16, 11 shards (~54 GB) |
+| Context | 16K validated (base supports more) |
+| Reasoning | thinking **on** — this is how it was trained and evaluated |
+| Tool-call format | Qwen3.5-style XML (`<function=...>`), **not** JSON — see gotchas |
+| License | Apache-2.0 |
+
+---
+
+## Quickstart
+
+### Serve with vLLM (recommended)
 
-## Serve (vLLM >= 0.25, no --trust-remote-code)
 ```bash
-vllm serve beyoru/Opera --served-model-name opera -tp 4 --max-model-len 16384 \
+vllm serve beyoru/Opera --served-model-name opera \
+  -tp 4 --max-model-len 16384 \
   --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3
-```
+```
+
+Do **not** pass `--trust-remote-code`: vLLM registers its own `Qwen3_5MoeConfig`/`qwen3_5`
+config class only when remote code is off. With the flag on you get
+`Invalid type of HuggingFace config`.
+
+Fits on 1×H200, or 2×A100-80G / 4×A100-40G with `-tp`.
+
+### Call it like any OpenAI-compatible endpoint
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
+
+tools = [{
+    "type": "function",
+    "function": {
+        "name": "create_penalty_voucher",
+        "description": "Lập chứng từ lãi phạt chậm thanh toán cho một hóa đơn.",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "invoice_no": {"type": "string"},
+                "amount": {"type": "integer", "description": "Số tiền lãi phạt (VND)"},
+            },
+            "required": ["invoice_no", "amount"],
+        },
+    },
+}]
+
+resp = client.chat.completions.create(
+    model="opera",
+    messages=[{"role": "user", "content":
+               "Hóa đơn HD-1042 của Cty TNHH An Phát: dư nợ 120 triệu, quá hạn 45 ngày. "
+               "Tra chính sách rồi lập chứng từ lãi phạt."}],
+    tools=tools,
+    temperature=0.7,
+    max_tokens=4096,
+)
+print(resp.choices[0].message.tool_calls)
+```
+
+### Transformers
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+tok = AutoTokenizer.from_pretrained("beyoru/Opera")
+model = AutoModelForCausalLM.from_pretrained("beyoru/Opera", dtype="auto", device_map="auto")
+
+msgs = [{"role": "user", "content": "Tính lãi phạt cho hóa đơn quá hạn 45 ngày, dư nợ 120 triệu."}]
+ids = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True, return_tensors="pt")
+out = model.generate(ids.to(model.device), max_new_tokens=1024)
+print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))
+```
+
+---
+
+## Results
+
+Held-out validation: the 48 hardest multi-turn tool-calling episodes from a private
+business-workflow set, scored every 5 steps during training.
+
+| step | score |
+|---|---|
+| 0 (base) | 0.564 |
+| 5 | 0.570 |
+| 10 | 0.567 |
+| 15 | **0.600** (peak, not checkpointed) |
+| **20 (this checkpoint)** | 0.574 |
+
+Only the final checkpoint was saved, so the released weights are **not** the peak of that
+curve, and the measured gain of this checkpoint over the base on that set is small
+(+0.010). Treat this as a targeted nudge on top of an already strong base — the value
+here is a merged, ready-to-serve agentic checkpoint plus the serving recipe below, not a
+large capability jump.
+
+---
+
+## Gotchas worth knowing
+
+These cost real debugging time. They apply to the whole Qwen3.5/3.6 family, not just Opera.
+
+1. **Tool calls are XML, not JSON.** The model emits
+   `<tool_call><function=NAME><parameter=KEY>VALUE</parameter></function></tool_call>`.
+   A Hermes/JSON parser silently returns *zero* tool calls — which looks exactly like a
+   broken environment or a model that refuses to act. Use `--tool-call-parser qwen3_coder`,
+   or parse the XML yourself.
+2. **No `--trust-remote-code`** when serving with vLLM (see above).
+3. **Keep thinking on.** Training and evaluation both ran with reasoning enabled;
+   disabling it costs the most on exact-sequence style metrics.
+4. **Vision is inherited, untouched, untested.** The checkpoint keeps the base VL
+   architecture and vision tower. RL training was text-only and the multimodal path was
+   never evaluated here.
+
+---
+
+## Training details
+
+| | |
+|---|---|
+| Algorithm | GRPO, 20 steps × batch 32 (640 prompts), 4 rollouts/prompt |
+| Adapter | LoRA r32 / α64, lr 5e-6, merged into base at export |
+| Data | xLAM + APIGen-MT function-calling traces, pre-filtered to ≤8192 tokens |
+| Reward | rule-based (function name + argument-subset partial credit) + tool-call structural validity + Intuitor self-certainty, rule and self-certainty z-normalized separately then weighted 1.0 / 1.0, applied at EOS |
+| Framework | verl |
+
+Self-certainty is computed over full-vocabulary logits. At 27B that has to be **chunked**
+(2048-token chunks, mirroring entropy chunking) or the run hangs during log-prob
+recomputation with the GPU at 0% and no OOM message.
+
+Training data is English function-calling; **Vietnamese ability comes from the base
+model** — it is not something this RL stage taught.
+
+## Limitations
+
+- Short RL run (20 steps). This is a targeted nudge on top of a strong base, not a
+  from-scratch agent.
+- Validation set is small (n=48) and private, so differences of a few points are within
+  noise and cannot be independently reproduced. No public-leaderboard numbers are claimed.
+- Released checkpoint is step 20, not the step-15 peak.
+- Not evaluated on: long-context (>16K), multimodal input, non-agentic chat quality,
+  safety. Assume base-model behaviour on all of those.
+
+## Citation
+
+```bibtex
+@misc{opera2026,
+  title  = {Opera: GRPO + self-certainty RL for agentic tool-calling},
+  author = {beyoru},
+  year   = {2026},
+  url    = {https://huggingface.co/beyoru/Opera}
+}
+```
+
+Built on [Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B).
+Self-certainty reward from [Intuitor](https://arxiv.org/abs/2505.19590).