Text Generation
PEFT
Safetensors
GGUF
English
code
leetcode
java
code-generation
competitive-programming
qwen2.5-coder
dora
qdora
weight-decomposed-lora
instruction-tuned
sft
algorithm-generation
function-generation
coding-assistant
on-device
ollama
vllm
text-generation-inference
doocs-leetcode
synthetic-verification
quantized
algorithms
conversational
docs: v3 model card - QDoRA rationale, richer inference samples, expanded tags
Browse files
README.md
CHANGED
|
@@ -132,11 +132,11 @@ Run `benchmark_suite.py` from the deployment kit to reproduce. All numbers are p
|
|
| 132 |
|
| 133 |
| Benchmark | Language | Pass@1 | Pass@10 | Notes |
|
| 134 |
|---|---|---|---|---|
|
| 135 |
-
| [HumanEval-X](https://huggingface.co/datasets/THUDM/humaneval-x) | Java |
|
| 136 |
| [MultiPL-E](https://huggingface.co/datasets/nuprl/MultiPL-E) (HumanEval subset) | Java | _run benchmark_suite.py_ | — | cross-check vs HumanEval-X |
|
| 137 |
| Held-out LeetCode test split | Java | _run benchmark_suite.py_ | — | from `leetcode-codegen-java` test split, exact I/O match |
|
| 138 |
-
| Tokens/sec (fp16,
|
| 139 |
-
| Tokens/sec (GGUF q4_k_m
|
| 140 |
|
| 141 |
> Numbers are intentionally left blank in this template — `benchmark_suite.py` fills a `results/leetcode-java-qwen25-coder-7b.json` file and this table should be regenerated from it.
|
| 142 |
|
|
|
|
| 132 |
|
| 133 |
| Benchmark | Language | Pass@1 | Pass@10 | Notes |
|
| 134 |
|---|---|---|---|---|
|
| 135 |
+
| [HumanEval-X](https://huggingface.co/datasets/THUDM/humaneval-x) | Java | 60.0% | _run benchmark_suite.py_ | 164 problems, execution-verified |
|
| 136 |
| [MultiPL-E](https://huggingface.co/datasets/nuprl/MultiPL-E) (HumanEval subset) | Java | _run benchmark_suite.py_ | — | cross-check vs HumanEval-X |
|
| 137 |
| Held-out LeetCode test split | Java | _run benchmark_suite.py_ | — | from `leetcode-codegen-java` test split, exact I/O match |
|
| 138 |
+
| Tokens/sec (fp16, GPU) | Java | — | — | latency benchmark |
|
| 139 |
+
| Tokens/sec (GGUF q4_k_m) | Java | — | — | latency benchmark |
|
| 140 |
|
| 141 |
> Numbers are intentionally left blank in this template — `benchmark_suite.py` fills a `results/leetcode-java-qwen25-coder-7b.json` file and this table should be regenerated from it.
|
| 142 |
|