coding-agents
tool-selection
paintedwolf-ai commited on
Commit
39012d0
·
verified ·
1 Parent(s): 4c74ef2

Release open1-b5-e4-bialy

Browse files

Prepared Bialy weights and training artifacts; checksums anchored in paintedwolf-ai/bialy.

Signed-off-by: Chris Beckman <chris@paintedwolf.ai>

LICENSE ADDED
@@ -0,0 +1,218 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ Apache License
3
+ Version 2.0, January 2004
4
+ http://www.apache.org/licenses/
5
+
6
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7
+
8
+ 1. Definitions.
9
+
10
+ "License" shall mean the terms and conditions for use, reproduction,
11
+ and distribution as defined by Sections 1 through 9 of this document.
12
+
13
+ "Licensor" shall mean the copyright owner or entity authorized by
14
+ the copyright owner that is granting the License.
15
+
16
+ "Legal Entity" shall mean the union of the acting entity and all
17
+ other entities that control, are controlled by, or are under common
18
+ control with that entity. For the purposes of this definition,
19
+ "control" means (i) the power, direct or indirect, to cause the
20
+ direction or management of such entity, whether by contract or
21
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
22
+ outstanding shares, or (iii) beneficial ownership of such entity.
23
+
24
+ "You" (or "Your") shall mean an individual or Legal Entity
25
+ exercising permissions granted by this License.
26
+
27
+ "Source" form shall mean the preferred form for making modifications,
28
+ including but not limited to software source code, documentation
29
+ source, and configuration files.
30
+
31
+ "Object" form shall mean any form resulting from mechanical
32
+ transformation or translation of a Source form, including but
33
+ not limited to compiled object code, generated documentation,
34
+ and conversions to other media types.
35
+
36
+ "Work" shall mean the work of authorship, whether in Source or
37
+ Object form, made available under the License, as indicated by a
38
+ copyright notice that is included in or attached to the work
39
+ (an example is provided in the Appendix below).
40
+
41
+ "Derivative Works" shall mean any work, whether in Source or Object
42
+ form, that is based on (or derived from) the Work and for which the
43
+ editorial revisions, annotations, elaborations, or other modifications
44
+ represent, as a whole, an original work of authorship. For the purposes
45
+ of this License, Derivative Works shall not include works that remain
46
+ separable from, or merely link (or bind by name) to the interfaces of,
47
+ the Work and Derivative Works thereof.
48
+
49
+ "Contribution" shall mean any work of authorship, including
50
+ the original version of the Work and any modifications or additions
51
+ to that Work or Derivative Works thereof, that is intentionally
52
+ submitted to Licensor for inclusion in the Work by the copyright owner
53
+ or by an individual or Legal Entity authorized to submit on behalf of
54
+ the copyright owner. For the purposes of this definition, "submitted"
55
+ means any form of electronic, verbal, or written communication sent
56
+ to the Licensor or its representatives, including but not limited to
57
+ communication on electronic mailing lists, source code control systems,
58
+ and issue tracking systems that are managed by, or on behalf of, the
59
+ Licensor for the purpose of discussing and improving the Work, but
60
+ excluding communication that is conspicuously marked or otherwise
61
+ designated in writing by the copyright owner as "Not a Contribution."
62
+
63
+ "Contributor" shall mean Licensor and any individual or Legal Entity
64
+ on behalf of whom a Contribution has been received by Licensor and
65
+ subsequently incorporated within the Work.
66
+
67
+ 2. Grant of Copyright License. Subject to the terms and conditions of
68
+ this License, each Contributor hereby grants to You a perpetual,
69
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70
+ copyright license to reproduce, prepare Derivative Works of,
71
+ publicly display, publicly perform, sublicense, and distribute the
72
+ Work and such Derivative Works in Source or Object form.
73
+
74
+ 3. Grant of Patent License. Subject to the terms and conditions of
75
+ this License, each Contributor hereby grants to You a perpetual,
76
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77
+ (except as stated in this section) patent license to make, have made,
78
+ use, offer to sell, sell, import, and otherwise transfer the Work,
79
+ where such license applies only to those patent claims licensable
80
+ by such Contributor that are necessarily infringed by their
81
+ Contribution(s) alone or by combination of their Contribution(s)
82
+ with the Work to which such Contribution(s) was submitted. If You
83
+ institute patent litigation against any entity (including a
84
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
85
+ or a Contribution incorporated within the Work constitutes direct
86
+ or contributory patent infringement, then any patent licenses
87
+ granted to You under this License for that Work shall terminate
88
+ as of the date such litigation is filed.
89
+
90
+ 4. Redistribution. You may reproduce and distribute copies of the
91
+ Work or Derivative Works thereof in any medium, with or without
92
+ modifications, and in Source or Object form, provided that You
93
+ meet the following conditions:
94
+
95
+ (a) You must give any other recipients of the Work or
96
+ Derivative Works a copy of this License; and
97
+
98
+ (b) You must cause any modified files to carry prominent notices
99
+ stating that You changed the files; and
100
+
101
+ (c) You must retain, in the Source form of any Derivative Works
102
+ that You distribute, all copyright, patent, trademark, and
103
+ attribution notices from the Source form of the Work,
104
+ excluding those notices that do not pertain to any part of
105
+ the Derivative Works; and
106
+
107
+ (d) If the Work includes a "NOTICE" text file as part of its
108
+ distribution, then any Derivative Works that You distribute must
109
+ include a readable copy of the attribution notices contained
110
+ within such NOTICE file, excluding those notices that do not
111
+ pertain to any part of the Derivative Works, in at least one
112
+ of the following places: within a NOTICE text file distributed
113
+ as part of the Derivative Works; within the Source form or
114
+ documentation, if provided along with the Derivative Works; or,
115
+ within a display generated by the Derivative Works, if and
116
+ wherever such third-party notices normally appear. The contents
117
+ of the NOTICE file are for informational purposes only and
118
+ do not modify the License. You may add Your own attribution
119
+ notices within Derivative Works that You distribute, alongside
120
+ or as an addendum to the NOTICE text from the Work, provided
121
+ that such additional attribution notices cannot be construed
122
+ as modifying the License.
123
+
124
+ You may add Your own copyright statement to Your modifications and
125
+ may provide additional or different license terms and conditions
126
+ for use, reproduction, or distribution of Your modifications, or
127
+ for any such Derivative Works as a whole, provided Your use,
128
+ reproduction, and distribution of the Work otherwise complies with
129
+ the conditions stated in this License.
130
+
131
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
132
+ any Contribution intentionally submitted for inclusion in the Work
133
+ by You to the Licensor shall be under the terms and conditions of
134
+ this License, without any additional terms or conditions.
135
+ Notwithstanding the above, nothing herein shall supersede or modify
136
+ the terms of any separate license agreement you may have executed
137
+ with Licensor regarding such Contributions.
138
+
139
+ 6. Trademarks. This License does not grant permission to use the trade
140
+ names, trademarks, service marks, or product names of the Licensor,
141
+ except as required for reasonable and customary use in describing the
142
+ origin of the Work and reproducing the content of the NOTICE file.
143
+
144
+ 7. Disclaimer of Warranty. Unless required by applicable law or
145
+ agreed to in writing, Licensor provides the Work (and each
146
+ Contributor provides its Contributions) on an "AS IS" BASIS,
147
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148
+ implied, including, without limitation, any warranties or conditions
149
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150
+ PARTICULAR PURPOSE. You are solely responsible for determining the
151
+ appropriateness of using or redistributing the Work and assume any
152
+ risks associated with Your exercise of permissions under this License.
153
+
154
+ 8. Limitation of Liability. In no event and under no legal theory,
155
+ whether in tort (including negligence), contract, or otherwise,
156
+ unless required by applicable law (such as deliberate and grossly
157
+ negligent acts) or agreed to in writing, shall any Contributor be
158
+ liable to You for damages, including any direct, indirect, special,
159
+ incidental, or consequential damages of any character arising as a
160
+ result of this License or out of the use or inability to use the
161
+ Work (including but not limited to damages for loss of goodwill,
162
+ work stoppage, computer failure or malfunction, or any and all
163
+ other commercial damages or losses), even if such Contributor
164
+ has been advised of the possibility of such damages.
165
+
166
+ 9. Accepting Warranty or Additional Liability. While redistributing
167
+ the Work or Derivative Works thereof, You may choose to offer,
168
+ and charge a fee for, acceptance of support, warranty, indemnity,
169
+ or other liability obligations and/or rights consistent with this
170
+ License. However, in accepting such obligations, You may act only
171
+ on Your own behalf and on Your sole responsibility, not on behalf
172
+ of any other Contributor, and only if You agree to indemnify,
173
+ defend, and hold each Contributor harmless for any liability
174
+ incurred by, or claims asserted against, such Contributor by reason
175
+ of your accepting any such warranty or additional liability.
176
+
177
+ END OF TERMS AND CONDITIONS
178
+
179
+ APPENDIX: How to apply the Apache License to your work.
180
+
181
+ To apply the Apache License to your work, attach the following
182
+ boilerplate notice, with the fields enclosed by brackets "[]"
183
+ replaced with your own identifying information. (Don't include
184
+ the brackets!) The text should be enclosed in the appropriate
185
+ comment syntax for the file format. We also recommend that a
186
+ file or class name and description of purpose be included on the
187
+ same "printed page" as the copyright notice for easier
188
+ identification within third-party archives.
189
+
190
+ Copyright 2026 Painted Wolf LLC
191
+
192
+ Licensed under the Apache License, Version 2.0 (the "License");
193
+ you may not use this file except in compliance with the License.
194
+ You may obtain a copy of the License at
195
+
196
+ http://www.apache.org/licenses/LICENSE-2.0
197
+
198
+ Unless required by applicable law or agreed to in writing, software
199
+ distributed under the License is distributed on an "AS IS" BASIS,
200
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201
+ See the License for the specific language governing permissions and
202
+ limitations under the License.
203
+
204
+ ---
205
+ NOTICE
206
+
207
+ Trademarks: Painted Wolf is a trademark of Painted Wolf LLC, along with the
208
+ product name Painted Wolf Code and the project's logos, mascots, and brand
209
+ artwork. This license grants copyright permissions for source code; it
210
+ does not grant trademark rights. What redistributors may do with the marks
211
+ is in docs/trademarks.md. That policy gives themes limited permission to alter
212
+ the app's visual identity, including the fixed Painted Wolf Code identity's
213
+ theme-controlled colors and logo-glyph visibility, through the built-in theme
214
+ system.
215
+
216
+ Other licenses: Bundled third-party engines and vendored gate rules retain
217
+ their respective upstream licenses. Lycaon-authored rule YAML is licensed
218
+ separately under CC-BY-4.0. See docs/licensing.md.
NOTICE ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Bialy decision heads
2
+ Copyright 2026 Painted Wolf LLC
3
+
4
+ Licensed under the Apache License, Version 2.0. See LICENSE.
5
+
6
+ These heads run over a backbone they do not include:
7
+
8
+ - Laya multilingual (convaiinnovations/laya-multilingual, revision
9
+ e4e9ddf21a7b1903b7acffd8814ad4307bf63a67), by Convai Innovations, under
10
+ the Apache License, Version 2.0: https://huggingface.co/convaiinnovations/laya-multilingual
11
+ - mmBERT-base (jhu-clsp/mmBERT-base), the encoder Laya multilingual builds on,
12
+ by the Johns Hopkins University Center for Language and Speech Processing,
13
+ under the MIT License: https://huggingface.co/jhu-clsp/mmBERT-base
14
+
15
+ "Painted Wolf" and "Painted Wolf Code" are trademarks of Painted Wolf LLC.
16
+ This license grants no trademark rights.
PROVENANCE.json ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "schema": "pw-decide-heads-evidence/1",
3
+ "version": "open1-b5-e4",
4
+ "model_repo": "paintedwolfcode/bialy",
5
+ "code_repo": "https://github.com/paintedwolf-ai/bialy",
6
+ "backbone": {
7
+ "model": "convaiinnovations/laya-multilingual",
8
+ "revision": "e4e9ddf21a7b1903b7acffd8814ad4307bf63a67"
9
+ },
10
+ "heads": [
11
+ {
12
+ "file": "turn-load.safetensors",
13
+ "sha256": "550f30b94d5c6a8680d2cb2a607ebc9ee33e709c7679f72606d1e9fda17f2e90",
14
+ "bytes": 59881652,
15
+ "metadata": {
16
+ "by_type": "{\"loss\": {\"tools\": 0.0732}, \"tools\": {\"precision\": 0.417, \"recall\": 0.691, \"positives\": 285, \"options\": 30083}, \"tools@coordinator\": {\"precision\": 0.422, \"recall\": 0.691, \"positives\": 285, \"options\": 27810}, \"tools@worker\": {\"precision\": 0.0, \"recall\": 0.0, \"positives\": 0, \"options\": 2273}}",
17
+ "head_max_len": "512",
18
+ "format": "pw-decide-head/1",
19
+ "label": "open1-turn-load-B5-release-independent",
20
+ "max_len": "1024",
21
+ "tool_option_words": "60",
22
+ "corpus": "dc3bf9215aac8877",
23
+ "val_acc": "0.9879333843034271",
24
+ "turn_rows": "engine-off",
25
+ "train_rows": "2746",
26
+ "rank_levels": "blended",
27
+ "val_loss": "0.07324849329615148",
28
+ "pos_weight": "6.0",
29
+ "tool_weight": "none",
30
+ "seed": "11",
31
+ "state_tokens": "{\"tools\": {\"median\": 86, \"under_16\": 0.0, \"request_kept\": 1.0}}",
32
+ "tool_negatives": "24",
33
+ "backbone": "jhu-clsp/mmBERT-base",
34
+ "model": "convaiinnovations/laya-multilingual",
35
+ "holdout": "[]",
36
+ "tool_truth": "consensus",
37
+ "tool_encoding": "independent"
38
+ }
39
+ },
40
+ {
41
+ "file": "unit-rank.safetensors",
42
+ "sha256": "c62674993eabba94660eccfafbb6e04d1549adc7335656c0ddefa7f068fa58f7",
43
+ "bytes": 59881268,
44
+ "metadata": {
45
+ "val_loss": "0.58553760010303",
46
+ "format": "pw-decide-head/1",
47
+ "label": "open1-unit-rank-e4-dense1",
48
+ "seed": "11",
49
+ "holdout": "[]",
50
+ "turn_rows": "engine-off",
51
+ "pos_weight": "6.0",
52
+ "head_max_len": "512",
53
+ "val_acc": "0.7880391514072964",
54
+ "train_rows": "3792",
55
+ "backbone": "jhu-clsp/mmBERT-base",
56
+ "by_type": "{\"score\": 0.788, \"score_within_one\": 0.921, \"loss\": {\"requests\": 0.5928, \"skills\": 0.5827}}",
57
+ "max_len": "1024",
58
+ "model": "convaiinnovations/laya-multilingual",
59
+ "corpus": "dc3bf9215aac8877",
60
+ "tool_weight": "none",
61
+ "tool_truth": "consensus",
62
+ "rank_levels": "skills-blended",
63
+ "state_tokens": "{}"
64
+ }
65
+ },
66
+ {
67
+ "file": "code-rank.safetensors",
68
+ "sha256": "abe235c347a223e957a0f8b7b979e3a584aebab76ee239e5f62117e839b9c40a",
69
+ "bytes": 59880964,
70
+ "metadata": {
71
+ "label": "open1-code-rank",
72
+ "model": "convaiinnovations/laya-multilingual",
73
+ "val_loss": "0.734294557281827",
74
+ "train_examples": "251360",
75
+ "backbone": "jhu-clsp/mmBERT-base",
76
+ "max_len": "512",
77
+ "format": "pw-decide-head/1",
78
+ "val_acc": "0.684346701164295",
79
+ "val_mae": "0.7976714370407317"
80
+ }
81
+ }
82
+ ],
83
+ "release_manifest_sha256": "404f04b03fcf6b757330eda9c0841835248b1f8dc105fb3b61f01dfa9657c872",
84
+ "dataset_version": "open1-b5-e4-bialy",
85
+ "dataset_status": "A companion dataset bundle is prepared separately. This statement does not assert that either bundle has been published.",
86
+ "training_sources": {
87
+ "turn-load": {
88
+ "trainer_commit": "3d79a91c92",
89
+ "factory_commit": "bf0ee33",
90
+ "training_rows_sha256": "2ec8ce650fd19b0e22bbc6b1673be699165fa258e98bbf5f963e6c9354ba80bd",
91
+ "validation_rows_sha256": "67c4c86e3385f0241048f2633d97a4aad78a53a3b7a540592937a20eba669fd3",
92
+ "corpus_sha256": "c77abd8b40965c05cc43a7545e8c2db2405bdcfc18a73bad254632d523b67bcd"
93
+ },
94
+ "unit-rank": {
95
+ "trainer_commit": "946aea8506c9440a662f87a8d5479963f27735d4",
96
+ "training_factory_commit": "b60c15122b4dcddaee75bc45b2e9413a0fe3fa8e",
97
+ "evaluation_factory_commit": "5f2b0b0751030dba9fe45d08f3c2e40f27cab4f9"
98
+ },
99
+ "code-rank": {
100
+ "status": "Selected open1 weights retained; training metadata is embedded in the head. No standalone code-rank benchmark report is asserted by this bundle."
101
+ }
102
+ },
103
+ "source_availability": "The separate companion dataset contains training inputs, sanitized conversation exports, trainer snapshots and provenance. Private originals and additional operational artifacts remain archived separately. Historical commits are not promised to be reachable in a flattened repository.",
104
+ "evidence": [
105
+ "evidence/tool-validation.json",
106
+ "evidence/unit-rank-historical-acceptance.json",
107
+ "evidence/host-policy-context.json"
108
+ ],
109
+ "replay_compatibility": "Evidence retains the original scorer semantics. It is not the legacy replay_eval overall schema consumed by bialy audit heads.",
110
+ "assembly": "Copied selected weights without modification. Public reports are aggregate projections of existing evidence; no training, inference, or tests were run during packaging.",
111
+ "public_name": "Bialy",
112
+ "package_version": "open1-b5-e4-bialy",
113
+ "dataset_repo": "paintedwolfcode/bialy-dataset"
114
+ }
README.md CHANGED
@@ -1,3 +1,115 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ base_model: convaiinnovations/laya-multilingual
4
+ tags:
5
+ - safetensors
6
+ - coding-agents
7
+ - tool-selection
8
+ - decision-heads
9
  ---
10
+
11
+ # Bialy
12
+
13
+ Package `open1-b5-e4-bialy` preserves model release `open1-b5-e4` and contains three trained decision heads used by Painted Wolf Code:
14
+
15
+ - `turn-load`: B5 independently scores candidate tools for preloading.
16
+ - `unit-rank`: E4 dense1 scores skills and requested tools for relevance.
17
+ - `code-rank`: open1 scores candidate code units for retrieval.
18
+
19
+ These are head weights, not a standalone language model. They require the
20
+ Laya multilingual backbone and the compatible `bialy` runtime. This repository
21
+ does not bundle the backbone. Training inputs and sanitized session exports are
22
+ prepared separately in the companion dataset `paintedwolfcode/bialy-dataset`.
23
+ Preparation does not assert that either bundle has been published.
24
+
25
+ ## Files and identity
26
+
27
+ | File | Label | SHA-256 |
28
+ |---|---|---|
29
+ | `turn-load.safetensors` | `open1-turn-load-B5-release-independent` | `550f30b94d5c6a8680d2cb2a607ebc9ee33e709c7679f72606d1e9fda17f2e90` |
30
+ | `unit-rank.safetensors` | `open1-unit-rank-e4-dense1` | `c62674993eabba94660eccfafbb6e04d1549adc7335656c0ddefa7f068fa58f7` |
31
+ | `code-rank.safetensors` | `open1-code-rank` | `abe235c347a223e957a0f8b7b979e3a584aebab76ee239e5f62117e839b9c40a` |
32
+
33
+ `release.json` pins the weights, backbone revision, and tool option vocabulary.
34
+ `PROVENANCE.json` records the available training identities and evidence scope.
35
+ `SHA256SUMS` covers every public file; the source repository's
36
+ `releases/heads-open1-b5-e4-bialy.json` anchors that checksum list.
37
+
38
+ ## Loading the heads
39
+
40
+ Use the Painted Wolf Code runtime and the backbone
41
+ `convaiinnovations/laya-multilingual` at revision
42
+ `e4e9ddf21a7b1903b7acffd8814ad4307bf63a67`. Its encoder is `jhu-clsp/mmBERT-base`.
43
+
44
+ When building Painted Wolf Code, set `BIALY_HEADS_DIR` to the directory containing
45
+ these three `.safetensors` files. The app's committed decision release manifest
46
+ must match their hashes; the staging process verifies them before packaging.
47
+ The B5 head uses independent tool encoding, a 1024-token maximum length, and a
48
+ 512-token head budget. A generic Transformers pipeline is not provided.
49
+
50
+ ## Tool validation
51
+
52
+ The existing validation set contains 445 complete turns. Consensus labels and
53
+ observed tool calls are separate evaluation targets. B2 is the previous trained
54
+ tool head, not an untuned backbone baseline. B5 selection used this validation
55
+ distribution; these results are not fresh final acceptance or end-to-end task success.
56
+
57
+ This compares two complete configurations: B2 uses joint encoding with six-word
58
+ tool options; B5 uses independent scoring with up to 60-word options. Their corpus
59
+ hashes intentionally differ. Each candidate was bound to its own corpus, prediction
60
+ metadata, head hash and predictions; their validation-row hash is shared.
61
+
62
+ | Head | Cutoff | Consensus precision | Consensus recall | Observed-call precision | Observed-call recall | Loads per turn |
63
+ |---|---|---|---|---|---|---|
64
+ | B2 | 0.89 | 0.757 | 0.295 | 0.517 | 0.149 | 0.339 |
65
+ | B5 | 0.95 | 0.822 | 0.291 | 0.547 | 0.143 | 0.308 |
66
+ | B5 | 0.94 | 0.798 | 0.319 | 0.559 | 0.172 | 0.362 |
67
+
68
+ The 0.94 and 0.95 rows describe two evaluated cutoffs for the same B5 weights.
69
+ The archived host closeout uses tool preload 0.94, at most one automatic skill at
70
+ 3.4, and free-text skill lookup 3.2. Those settings do not change the weights.
71
+ Later host tests and startup checks establish implementation correctness within
72
+ their recorded scope, not model task quality or fresh end-to-end acceptance.
73
+ App policy controls cutoffs independently of head files. The evidence file
74
+ retains all archived aggregate threshold rows, input hashes, and report hashes.
75
+ No benchmark was rerun for this publication bundle.
76
+
77
+ ## Historical unit-rank evidence
78
+
79
+ E4's archived acceptance report covers 789 requests, of which 660 have a relevant
80
+ skill. Under its historical listing and preload policy, a relevant skill appeared
81
+ in the first six for 92.1% of relevant requests; 122 of 123 automatic preloads were
82
+ relevant. The report contains no observed request-tool needs.
83
+
84
+ The strict skills verdict was **failed**: the common-skill criterion did not pass.
85
+ The other listed skill criteria passed. This evidence does not establish acceptance
86
+ for current free-text lookup or current
87
+ automatic skill preloading. The original policy values and verdict booleans are
88
+ included in `evidence/unit-rank-historical-acceptance.json`.
89
+
90
+ ## Limits and evidence access
91
+
92
+ Only aggregate evaluation evidence is published here. Per-example requests,
93
+ training rows, session stores, raw prediction text, machine paths, and operational
94
+ logs are excluded. Original report and input hashes preserve evidence identities;
95
+ they do not make omitted inputs publicly reproducible. Inputs, source snapshots,
96
+ predictions and training artifacts are archived privately. That private archive
97
+ is distinct from what a reader can reproduce from this weights-only public bundle.
98
+ The companion dataset package `open1-b5-e4-bialy` provides training inputs,
99
+ sanitized conversation exports and historical trainer snapshots separately.
100
+ Historical source commits are recorded without claiming they are reachable in
101
+ a flattened public repository.
102
+ No standalone code-rank benchmark or full-current-policy acceptance claim is made.
103
+
104
+ `bialy verify heads` can check this bundle against its committed checksum anchor.
105
+ The included reports use the tool-validation and ranking scorer formats, so the
106
+ legacy `bialy audit heads` replay interface does not support this evidence bundle.
107
+ It should not be presented as a successful replay audit.
108
+
109
+ ## Attribution
110
+
111
+ The heads use [Laya](https://github.com/NandhaKishorM/laya), by Convai Innovations,
112
+ and its [multilingual checkpoint](https://huggingface.co/convaiinnovations/laya-multilingual),
113
+ over [mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base).
114
+ Head weights are provided under Apache-2.0. See `LICENSE` and `NOTICE` for
115
+ attribution and the separately licensed backbone dependencies.
SHA256SUMS ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 3c2307bbbbb88f79560a165b098c600da8f20fb1ec024586d4be192ebc9c6f8e LICENSE
2
+ b4cdc8ad8e0ec0cc6def7b0c1ad077e5bbd8f473f0ba8bc6a66dd5a46a71abee NOTICE
3
+ ce81c8201ed2afb855c95d99d70ccc107ec8577937ea98ef3506e29eb1d520a6 PROVENANCE.json
4
+ d6001f48ac599ef2361a0b1026bf2780b62c6565dab2eb1f7c24b8c05e3988fd README.md
5
+ abe235c347a223e957a0f8b7b979e3a584aebab76ee239e5f62117e839b9c40a code-rank.safetensors
6
+ 0f338ab310d2746541ca08799fcbb76ab25dbdc724a8e41731ea874091ff0549 evidence/host-policy-context.json
7
+ 45ea0d69658a114dc73f3984766e529c66a87f6f35fc2deb536d26b84313cd9d evidence/tool-validation.json
8
+ 5b97c4f57274870cdcd11a2912baa83c69dc8e40f74df9409f628c076d89a20e evidence/unit-rank-historical-acceptance.json
9
+ 404f04b03fcf6b757330eda9c0841835248b1f8dc105fb3b61f01dfa9657c872 release.json
10
+ 550f30b94d5c6a8680d2cb2a607ebc9ee33e709c7679f72606d1e9fda17f2e90 turn-load.safetensors
11
+ c62674993eabba94660eccfafbb6e04d1549adc7335656c0ddefa7f068fa58f7 unit-rank.safetensors
code-rank.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:abe235c347a223e957a0f8b7b979e3a584aebab76ee239e5f62117e839b9c40a
3
+ size 59880964
evidence/host-policy-context.json ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "scope": "Archived host policy closeout; historical implementation and startup checks do not establish current model task quality.",
3
+ "source_report_sha256": "bc2c913cf0db2d60a7808bf0554a32a2fab91a120c8277d447402d6752e9a880",
4
+ "source_deployment_sha256": "65e2d0f6b6e0c3e15c68642837f9d64a962d8189af47d617ffcf5dd46f24ed6b",
5
+ "host_commit": "28d781197b38568d543d459a0702eb38e57348b6",
6
+ "factory_commit": "5b757e846c6108f834fdb14a4c045497b2139df6",
7
+ "policy": {
8
+ "tool_preload_threshold": 0.94,
9
+ "skill_preload_threshold": 3.4,
10
+ "maximum_automatic_skills": 1,
11
+ "skill_lookup_threshold": 3.2,
12
+ "full_skill_catalog_in_prompt": false
13
+ },
14
+ "model_heads_changed": false
15
+ }
evidence/tool-validation.json ADDED
@@ -0,0 +1,1141 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "scope": "Existing validation distribution; threshold selection, not fresh final acceptance.",
3
+ "comparison": "Two complete configurations on the same validation rows: B2 joint encoding with six-word tool options; B5 independent encoding with up to 60-word options. Corpus hashes intentionally differ.",
4
+ "transformation": "Per-tool breakdowns omitted; all aggregate threshold rows retained without recalculation.",
5
+ "prediction_metadata_sha256": "de8a6974d1ca4a615f319089b2a6b023e5e57ede9bc03e52de5aa9524bde11c9",
6
+ "reports": {
7
+ "B2": {
8
+ "original_report_sha256": "4b705b7e02a4307e3d23898691c6e82fc72fcfbe9d4e94c0c0341838af21d206",
9
+ "input_sha256": {
10
+ "predictions": "b20f3ddf176f4a5905ac1a4a3073e41a1c4833a930292c942abbba974c10c9b1",
11
+ "rows": "67c4c86e3385f0241048f2633d97a4aad78a53a3b7a540592937a20eba669fd3",
12
+ "corpus": "51e3a65258fc34581bc8cfaae0f7c97f444f0a50f3dda906c77b6554fd5c66d3"
13
+ },
14
+ "prediction_metadata_sha256": "e8d91777999dded4eafe02cf46d7e99a3a89b0bd376c7f6922519e33f7e360dd",
15
+ "encoding": "joint",
16
+ "tool_option_words": 6,
17
+ "head_sha256": "bc432ea7497e9300e39230ee9850de8f42e3f9a3cbbf372259263c49c77011e4",
18
+ "truth": {
19
+ "consensus": [
20
+ {
21
+ "threshold": 0.5,
22
+ "counts": {
23
+ "out_of_catalog_labels": 31,
24
+ "out_of_catalog_positives": 0,
25
+ "turns": 445,
26
+ "loads": 935,
27
+ "empty": 128,
28
+ "needed_turns": 216,
29
+ "any_needed_loaded": 173,
30
+ "all_needed_loaded": 149,
31
+ "tn": 29289,
32
+ "fp": 509,
33
+ "tp": 203,
34
+ "unknown_loaded": 223,
35
+ "fn": 82
36
+ },
37
+ "precision": 0.2851123595505618,
38
+ "recall": 0.712280701754386,
39
+ "f0.5": 0.3239706351739547,
40
+ "loads_per_turn": 2.101123595505618,
41
+ "empty_turn_rate": 0.2876404494382023
42
+ },
43
+ {
44
+ "threshold": 0.6,
45
+ "counts": {
46
+ "out_of_catalog_labels": 31,
47
+ "out_of_catalog_positives": 0,
48
+ "turns": 445,
49
+ "loads": 734,
50
+ "empty": 158,
51
+ "needed_turns": 216,
52
+ "any_needed_loaded": 159,
53
+ "all_needed_loaded": 137,
54
+ "tn": 29438,
55
+ "fp": 360,
56
+ "tp": 186,
57
+ "unknown_loaded": 188,
58
+ "fn": 99
59
+ },
60
+ "precision": 0.34065934065934067,
61
+ "recall": 0.6526315789473685,
62
+ "f0.5": 0.37667071688942894,
63
+ "loads_per_turn": 1.649438202247191,
64
+ "empty_turn_rate": 0.3550561797752809
65
+ },
66
+ {
67
+ "threshold": 0.7,
68
+ "counts": {
69
+ "out_of_catalog_labels": 31,
70
+ "out_of_catalog_positives": 0,
71
+ "turns": 445,
72
+ "loads": 533,
73
+ "empty": 189,
74
+ "needed_turns": 216,
75
+ "any_needed_loaded": 147,
76
+ "all_needed_loaded": 122,
77
+ "tn": 29576,
78
+ "fp": 222,
79
+ "tp": 166,
80
+ "unknown_loaded": 145,
81
+ "fn": 119
82
+ },
83
+ "precision": 0.42783505154639173,
84
+ "recall": 0.5824561403508772,
85
+ "f0.5": 0.4518236254763201,
86
+ "loads_per_turn": 1.1977528089887641,
87
+ "empty_turn_rate": 0.4247191011235955
88
+ },
89
+ {
90
+ "threshold": 0.75,
91
+ "counts": {
92
+ "out_of_catalog_labels": 31,
93
+ "out_of_catalog_positives": 0,
94
+ "turns": 445,
95
+ "loads": 419,
96
+ "empty": 220,
97
+ "needed_turns": 216,
98
+ "any_needed_loaded": 131,
99
+ "all_needed_loaded": 105,
100
+ "tn": 29640,
101
+ "fp": 158,
102
+ "tp": 147,
103
+ "unknown_loaded": 114,
104
+ "fn": 138
105
+ },
106
+ "precision": 0.4819672131147541,
107
+ "recall": 0.5157894736842106,
108
+ "f0.5": 0.4883720930232558,
109
+ "loads_per_turn": 0.9415730337078652,
110
+ "empty_turn_rate": 0.4943820224719101
111
+ },
112
+ {
113
+ "threshold": 0.8,
114
+ "counts": {
115
+ "out_of_catalog_labels": 31,
116
+ "out_of_catalog_positives": 0,
117
+ "turns": 445,
118
+ "loads": 301,
119
+ "empty": 262,
120
+ "needed_turns": 216,
121
+ "any_needed_loaded": 108,
122
+ "all_needed_loaded": 83,
123
+ "tn": 29703,
124
+ "fp": 95,
125
+ "tp": 117,
126
+ "unknown_loaded": 89,
127
+ "fn": 168
128
+ },
129
+ "precision": 0.5518867924528302,
130
+ "recall": 0.4105263157894737,
131
+ "f0.5": 0.5163283318623124,
132
+ "loads_per_turn": 0.6764044943820224,
133
+ "empty_turn_rate": 0.5887640449438202
134
+ },
135
+ {
136
+ "threshold": 0.85,
137
+ "counts": {
138
+ "out_of_catalog_labels": 31,
139
+ "out_of_catalog_positives": 0,
140
+ "turns": 445,
141
+ "loads": 211,
142
+ "empty": 299,
143
+ "needed_turns": 216,
144
+ "any_needed_loaded": 93,
145
+ "all_needed_loaded": 72,
146
+ "tn": 29745,
147
+ "fp": 53,
148
+ "tp": 100,
149
+ "unknown_loaded": 58,
150
+ "fn": 185
151
+ },
152
+ "precision": 0.6535947712418301,
153
+ "recall": 0.3508771929824561,
154
+ "f0.5": 0.5574136008918618,
155
+ "loads_per_turn": 0.47415730337078654,
156
+ "empty_turn_rate": 0.6719101123595506
157
+ },
158
+ {
159
+ "threshold": 0.89,
160
+ "counts": {
161
+ "out_of_catalog_labels": 31,
162
+ "out_of_catalog_positives": 0,
163
+ "turns": 445,
164
+ "loads": 151,
165
+ "empty": 326,
166
+ "needed_turns": 216,
167
+ "any_needed_loaded": 80,
168
+ "all_needed_loaded": 60,
169
+ "tn": 29771,
170
+ "fp": 27,
171
+ "fn": 201,
172
+ "unknown_loaded": 40,
173
+ "tp": 84
174
+ },
175
+ "precision": 0.7567567567567568,
176
+ "recall": 0.29473684210526313,
177
+ "f0.5": 0.5761316872427984,
178
+ "loads_per_turn": 0.3393258426966292,
179
+ "empty_turn_rate": 0.7325842696629213
180
+ },
181
+ {
182
+ "threshold": 0.9,
183
+ "counts": {
184
+ "out_of_catalog_labels": 31,
185
+ "out_of_catalog_positives": 0,
186
+ "turns": 445,
187
+ "loads": 138,
188
+ "empty": 336,
189
+ "needed_turns": 216,
190
+ "any_needed_loaded": 74,
191
+ "all_needed_loaded": 56,
192
+ "tn": 29773,
193
+ "fp": 25,
194
+ "fn": 207,
195
+ "unknown_loaded": 35,
196
+ "tp": 78
197
+ },
198
+ "precision": 0.7572815533980582,
199
+ "recall": 0.2736842105263158,
200
+ "f0.5": 0.5595408895265424,
201
+ "loads_per_turn": 0.3101123595505618,
202
+ "empty_turn_rate": 0.755056179775281
203
+ },
204
+ {
205
+ "threshold": 0.95,
206
+ "counts": {
207
+ "out_of_catalog_labels": 31,
208
+ "out_of_catalog_positives": 0,
209
+ "turns": 445,
210
+ "loads": 84,
211
+ "empty": 373,
212
+ "needed_turns": 216,
213
+ "any_needed_loaded": 51,
214
+ "all_needed_loaded": 43,
215
+ "tn": 29789,
216
+ "fn": 230,
217
+ "unknown_loaded": 20,
218
+ "tp": 55,
219
+ "fp": 9
220
+ },
221
+ "precision": 0.859375,
222
+ "recall": 0.19298245614035087,
223
+ "f0.5": 0.5083179297597042,
224
+ "loads_per_turn": 0.18876404494382024,
225
+ "empty_turn_rate": 0.8382022471910112
226
+ },
227
+ {
228
+ "threshold": 0.97,
229
+ "counts": {
230
+ "out_of_catalog_labels": 31,
231
+ "out_of_catalog_positives": 0,
232
+ "turns": 445,
233
+ "loads": 62,
234
+ "empty": 391,
235
+ "needed_turns": 216,
236
+ "any_needed_loaded": 41,
237
+ "all_needed_loaded": 34,
238
+ "tn": 29793,
239
+ "fn": 241,
240
+ "unknown_loaded": 13,
241
+ "tp": 44,
242
+ "fp": 5
243
+ },
244
+ "precision": 0.8979591836734694,
245
+ "recall": 0.1543859649122807,
246
+ "f0.5": 0.4573804573804574,
247
+ "loads_per_turn": 0.1393258426966292,
248
+ "empty_turn_rate": 0.8786516853932584
249
+ },
250
+ {
251
+ "threshold": 0.99,
252
+ "counts": {
253
+ "out_of_catalog_labels": 31,
254
+ "out_of_catalog_positives": 0,
255
+ "turns": 445,
256
+ "loads": 29,
257
+ "empty": 417,
258
+ "needed_turns": 216,
259
+ "any_needed_loaded": 21,
260
+ "all_needed_loaded": 15,
261
+ "tn": 29795,
262
+ "fn": 264,
263
+ "unknown_loaded": 5,
264
+ "tp": 21,
265
+ "fp": 3
266
+ },
267
+ "precision": 0.875,
268
+ "recall": 0.07368421052631578,
269
+ "f0.5": 0.2755905511811024,
270
+ "loads_per_turn": 0.0651685393258427,
271
+ "empty_turn_rate": 0.9370786516853933
272
+ }
273
+ ],
274
+ "called": [
275
+ {
276
+ "threshold": 0.5,
277
+ "counts": {
278
+ "out_of_catalog_labels": 31,
279
+ "out_of_catalog_positives": 0,
280
+ "turns": 445,
281
+ "loads": 935,
282
+ "empty": 128,
283
+ "needed_turns": 228,
284
+ "any_needed_loaded": 182,
285
+ "all_needed_loaded": 91,
286
+ "tn": 29888,
287
+ "fp": 659,
288
+ "tp": 276,
289
+ "fn": 248
290
+ },
291
+ "precision": 0.29518716577540105,
292
+ "recall": 0.5267175572519084,
293
+ "f0.5": 0.3236397748592871,
294
+ "loads_per_turn": 2.101123595505618,
295
+ "empty_turn_rate": 0.2876404494382023
296
+ },
297
+ {
298
+ "threshold": 0.6,
299
+ "counts": {
300
+ "out_of_catalog_labels": 31,
301
+ "out_of_catalog_positives": 0,
302
+ "turns": 445,
303
+ "loads": 734,
304
+ "empty": 158,
305
+ "needed_turns": 228,
306
+ "any_needed_loaded": 166,
307
+ "all_needed_loaded": 76,
308
+ "tn": 30053,
309
+ "fp": 494,
310
+ "tp": 240,
311
+ "fn": 284
312
+ },
313
+ "precision": 0.32697547683923706,
314
+ "recall": 0.4580152671755725,
315
+ "f0.5": 0.3468208092485549,
316
+ "loads_per_turn": 1.649438202247191,
317
+ "empty_turn_rate": 0.3550561797752809
318
+ },
319
+ {
320
+ "threshold": 0.7,
321
+ "counts": {
322
+ "out_of_catalog_labels": 31,
323
+ "out_of_catalog_positives": 0,
324
+ "turns": 445,
325
+ "loads": 533,
326
+ "empty": 189,
327
+ "needed_turns": 228,
328
+ "any_needed_loaded": 149,
329
+ "all_needed_loaded": 59,
330
+ "tn": 30208,
331
+ "tp": 194,
332
+ "fp": 339,
333
+ "fn": 330
334
+ },
335
+ "precision": 0.36397748592870544,
336
+ "recall": 0.3702290076335878,
337
+ "f0.5": 0.36521084337349397,
338
+ "loads_per_turn": 1.1977528089887641,
339
+ "empty_turn_rate": 0.4247191011235955
340
+ },
341
+ {
342
+ "threshold": 0.75,
343
+ "counts": {
344
+ "out_of_catalog_labels": 31,
345
+ "out_of_catalog_positives": 0,
346
+ "turns": 445,
347
+ "loads": 419,
348
+ "empty": 220,
349
+ "needed_turns": 228,
350
+ "any_needed_loaded": 133,
351
+ "all_needed_loaded": 46,
352
+ "tn": 30286,
353
+ "fn": 366,
354
+ "fp": 261,
355
+ "tp": 158
356
+ },
357
+ "precision": 0.37708830548926014,
358
+ "recall": 0.3015267175572519,
359
+ "f0.5": 0.35909090909090907,
360
+ "loads_per_turn": 0.9415730337078652,
361
+ "empty_turn_rate": 0.4943820224719101
362
+ },
363
+ {
364
+ "threshold": 0.8,
365
+ "counts": {
366
+ "out_of_catalog_labels": 31,
367
+ "out_of_catalog_positives": 0,
368
+ "turns": 445,
369
+ "loads": 301,
370
+ "empty": 262,
371
+ "needed_turns": 228,
372
+ "any_needed_loaded": 106,
373
+ "all_needed_loaded": 38,
374
+ "tn": 30368,
375
+ "fn": 402,
376
+ "fp": 179,
377
+ "tp": 122
378
+ },
379
+ "precision": 0.4053156146179402,
380
+ "recall": 0.23282442748091603,
381
+ "f0.5": 0.35300925925925924,
382
+ "loads_per_turn": 0.6764044943820224,
383
+ "empty_turn_rate": 0.5887640449438202
384
+ },
385
+ {
386
+ "threshold": 0.85,
387
+ "counts": {
388
+ "out_of_catalog_labels": 31,
389
+ "out_of_catalog_positives": 0,
390
+ "turns": 445,
391
+ "loads": 211,
392
+ "empty": 299,
393
+ "needed_turns": 228,
394
+ "any_needed_loaded": 83,
395
+ "all_needed_loaded": 32,
396
+ "tn": 30430,
397
+ "fn": 430,
398
+ "fp": 117,
399
+ "tp": 94
400
+ },
401
+ "precision": 0.44549763033175355,
402
+ "recall": 0.17938931297709923,
403
+ "f0.5": 0.3435672514619883,
404
+ "loads_per_turn": 0.47415730337078654,
405
+ "empty_turn_rate": 0.6719101123595506
406
+ },
407
+ {
408
+ "threshold": 0.89,
409
+ "counts": {
410
+ "out_of_catalog_labels": 31,
411
+ "out_of_catalog_positives": 0,
412
+ "turns": 445,
413
+ "loads": 151,
414
+ "empty": 326,
415
+ "needed_turns": 228,
416
+ "any_needed_loaded": 72,
417
+ "all_needed_loaded": 24,
418
+ "tn": 30474,
419
+ "fn": 446,
420
+ "fp": 73,
421
+ "tp": 78
422
+ },
423
+ "precision": 0.5165562913907285,
424
+ "recall": 0.14885496183206107,
425
+ "f0.5": 0.34574468085106386,
426
+ "loads_per_turn": 0.3393258426966292,
427
+ "empty_turn_rate": 0.7325842696629213
428
+ },
429
+ {
430
+ "threshold": 0.9,
431
+ "counts": {
432
+ "out_of_catalog_labels": 31,
433
+ "out_of_catalog_positives": 0,
434
+ "turns": 445,
435
+ "loads": 138,
436
+ "empty": 336,
437
+ "needed_turns": 228,
438
+ "any_needed_loaded": 65,
439
+ "all_needed_loaded": 22,
440
+ "tn": 30479,
441
+ "fn": 454,
442
+ "fp": 68,
443
+ "tp": 70
444
+ },
445
+ "precision": 0.5072463768115942,
446
+ "recall": 0.13358778625954199,
447
+ "f0.5": 0.3252788104089219,
448
+ "loads_per_turn": 0.3101123595505618,
449
+ "empty_turn_rate": 0.755056179775281
450
+ },
451
+ {
452
+ "threshold": 0.95,
453
+ "counts": {
454
+ "out_of_catalog_labels": 31,
455
+ "out_of_catalog_positives": 0,
456
+ "turns": 445,
457
+ "loads": 84,
458
+ "empty": 373,
459
+ "needed_turns": 228,
460
+ "any_needed_loaded": 42,
461
+ "all_needed_loaded": 15,
462
+ "tn": 30508,
463
+ "fn": 479,
464
+ "tp": 45,
465
+ "fp": 39
466
+ },
467
+ "precision": 0.5357142857142857,
468
+ "recall": 0.08587786259541985,
469
+ "f0.5": 0.2616279069767442,
470
+ "loads_per_turn": 0.18876404494382024,
471
+ "empty_turn_rate": 0.8382022471910112
472
+ },
473
+ {
474
+ "threshold": 0.97,
475
+ "counts": {
476
+ "out_of_catalog_labels": 31,
477
+ "out_of_catalog_positives": 0,
478
+ "turns": 445,
479
+ "loads": 62,
480
+ "empty": 391,
481
+ "needed_turns": 228,
482
+ "any_needed_loaded": 32,
483
+ "all_needed_loaded": 9,
484
+ "tn": 30520,
485
+ "fn": 489,
486
+ "tp": 35,
487
+ "fp": 27
488
+ },
489
+ "precision": 0.5645161290322581,
490
+ "recall": 0.06679389312977099,
491
+ "f0.5": 0.2266839378238342,
492
+ "loads_per_turn": 0.1393258426966292,
493
+ "empty_turn_rate": 0.8786516853932584
494
+ },
495
+ {
496
+ "threshold": 0.99,
497
+ "counts": {
498
+ "out_of_catalog_labels": 31,
499
+ "out_of_catalog_positives": 0,
500
+ "turns": 445,
501
+ "loads": 29,
502
+ "empty": 417,
503
+ "needed_turns": 228,
504
+ "any_needed_loaded": 15,
505
+ "all_needed_loaded": 5,
506
+ "tn": 30533,
507
+ "fn": 509,
508
+ "tp": 15,
509
+ "fp": 14
510
+ },
511
+ "precision": 0.5172413793103449,
512
+ "recall": 0.02862595419847328,
513
+ "f0.5": 0.1171875,
514
+ "loads_per_turn": 0.0651685393258427,
515
+ "empty_turn_rate": 0.9370786516853933
516
+ }
517
+ ]
518
+ }
519
+ },
520
+ "B5": {
521
+ "original_report_sha256": "18a25333e07dfb5b79c6709cf7c99855517ce07e51e3cd80ccc8e15e8a656a51",
522
+ "input_sha256": {
523
+ "predictions": "e70357d58524a0714882aef5c5de98a6c4cf74a3ad82ae23ee1ed9f97789d1c0",
524
+ "rows": "67c4c86e3385f0241048f2633d97a4aad78a53a3b7a540592937a20eba669fd3",
525
+ "corpus": "c77abd8b40965c05cc43a7545e8c2db2405bdcfc18a73bad254632d523b67bcd"
526
+ },
527
+ "prediction_metadata_sha256": "de8a6974d1ca4a615f319089b2a6b023e5e57ede9bc03e52de5aa9524bde11c9",
528
+ "encoding": "independent",
529
+ "tool_option_words": 60,
530
+ "head_sha256": "550f30b94d5c6a8680d2cb2a607ebc9ee33e709c7679f72606d1e9fda17f2e90",
531
+ "truth": {
532
+ "consensus": [
533
+ {
534
+ "threshold": 0.5,
535
+ "counts": {
536
+ "out_of_catalog_labels": 31,
537
+ "out_of_catalog_positives": 0,
538
+ "turns": 445,
539
+ "loads": 639,
540
+ "empty": 176,
541
+ "needed_turns": 216,
542
+ "any_needed_loaded": 166,
543
+ "all_needed_loaded": 141,
544
+ "tn": 29523,
545
+ "fp": 275,
546
+ "tp": 197,
547
+ "unknown_loaded": 167,
548
+ "fn": 88
549
+ },
550
+ "precision": 0.4173728813559322,
551
+ "recall": 0.6912280701754386,
552
+ "f0.5": 0.45329038196042337,
553
+ "loads_per_turn": 1.4359550561797754,
554
+ "empty_turn_rate": 0.3955056179775281
555
+ },
556
+ {
557
+ "threshold": 0.6,
558
+ "counts": {
559
+ "out_of_catalog_labels": 31,
560
+ "out_of_catalog_positives": 0,
561
+ "turns": 445,
562
+ "loads": 539,
563
+ "empty": 191,
564
+ "needed_turns": 216,
565
+ "any_needed_loaded": 164,
566
+ "all_needed_loaded": 138,
567
+ "tn": 29594,
568
+ "fp": 204,
569
+ "tp": 192,
570
+ "unknown_loaded": 143,
571
+ "fn": 93
572
+ },
573
+ "precision": 0.48484848484848486,
574
+ "recall": 0.6736842105263158,
575
+ "f0.5": 0.5136436597110754,
576
+ "loads_per_turn": 1.2112359550561798,
577
+ "empty_turn_rate": 0.42921348314606744
578
+ },
579
+ {
580
+ "threshold": 0.7,
581
+ "counts": {
582
+ "out_of_catalog_labels": 31,
583
+ "out_of_catalog_positives": 0,
584
+ "turns": 445,
585
+ "loads": 429,
586
+ "empty": 222,
587
+ "needed_turns": 216,
588
+ "any_needed_loaded": 145,
589
+ "all_needed_loaded": 118,
590
+ "tn": 29653,
591
+ "fp": 145,
592
+ "tp": 167,
593
+ "unknown_loaded": 117,
594
+ "fn": 118
595
+ },
596
+ "precision": 0.5352564102564102,
597
+ "recall": 0.5859649122807018,
598
+ "f0.5": 0.5446836268754077,
599
+ "loads_per_turn": 0.9640449438202248,
600
+ "empty_turn_rate": 0.49887640449438203
601
+ },
602
+ {
603
+ "threshold": 0.75,
604
+ "counts": {
605
+ "out_of_catalog_labels": 31,
606
+ "out_of_catalog_positives": 0,
607
+ "turns": 445,
608
+ "loads": 371,
609
+ "empty": 238,
610
+ "needed_turns": 216,
611
+ "any_needed_loaded": 136,
612
+ "all_needed_loaded": 108,
613
+ "tn": 29690,
614
+ "fp": 108,
615
+ "tp": 156,
616
+ "unknown_loaded": 107,
617
+ "fn": 129
618
+ },
619
+ "precision": 0.5909090909090909,
620
+ "recall": 0.5473684210526316,
621
+ "f0.5": 0.5816554809843401,
622
+ "loads_per_turn": 0.8337078651685393,
623
+ "empty_turn_rate": 0.5348314606741573
624
+ },
625
+ {
626
+ "threshold": 0.8,
627
+ "counts": {
628
+ "out_of_catalog_labels": 31,
629
+ "out_of_catalog_positives": 0,
630
+ "turns": 445,
631
+ "loads": 307,
632
+ "empty": 261,
633
+ "needed_turns": 216,
634
+ "any_needed_loaded": 126,
635
+ "all_needed_loaded": 100,
636
+ "tn": 29720,
637
+ "fp": 78,
638
+ "tp": 142,
639
+ "unknown_loaded": 87,
640
+ "fn": 143
641
+ },
642
+ "precision": 0.6454545454545455,
643
+ "recall": 0.4982456140350877,
644
+ "f0.5": 0.6094420600858369,
645
+ "loads_per_turn": 0.6898876404494382,
646
+ "empty_turn_rate": 0.5865168539325842
647
+ },
648
+ {
649
+ "threshold": 0.85,
650
+ "counts": {
651
+ "out_of_catalog_labels": 31,
652
+ "out_of_catalog_positives": 0,
653
+ "turns": 445,
654
+ "loads": 267,
655
+ "empty": 278,
656
+ "needed_turns": 216,
657
+ "any_needed_loaded": 118,
658
+ "all_needed_loaded": 91,
659
+ "tn": 29734,
660
+ "fp": 64,
661
+ "tp": 131,
662
+ "unknown_loaded": 72,
663
+ "fn": 154
664
+ },
665
+ "precision": 0.6717948717948717,
666
+ "recall": 0.45964912280701753,
667
+ "f0.5": 0.6150234741784038,
668
+ "loads_per_turn": 0.6,
669
+ "empty_turn_rate": 0.6247191011235955
670
+ },
671
+ {
672
+ "threshold": 0.89,
673
+ "counts": {
674
+ "out_of_catalog_labels": 31,
675
+ "out_of_catalog_positives": 0,
676
+ "turns": 445,
677
+ "loads": 222,
678
+ "empty": 295,
679
+ "needed_turns": 216,
680
+ "any_needed_loaded": 104,
681
+ "all_needed_loaded": 78,
682
+ "tn": 29754,
683
+ "fp": 44,
684
+ "tp": 115,
685
+ "unknown_loaded": 63,
686
+ "fn": 170
687
+ },
688
+ "precision": 0.7232704402515723,
689
+ "recall": 0.40350877192982454,
690
+ "f0.5": 0.6243213897937026,
691
+ "loads_per_turn": 0.49887640449438203,
692
+ "empty_turn_rate": 0.6629213483146067
693
+ },
694
+ {
695
+ "threshold": 0.9,
696
+ "counts": {
697
+ "out_of_catalog_labels": 31,
698
+ "out_of_catalog_positives": 0,
699
+ "turns": 445,
700
+ "loads": 216,
701
+ "empty": 297,
702
+ "needed_turns": 216,
703
+ "any_needed_loaded": 101,
704
+ "all_needed_loaded": 75,
705
+ "tn": 29757,
706
+ "fp": 41,
707
+ "tp": 112,
708
+ "unknown_loaded": 63,
709
+ "fn": 173
710
+ },
711
+ "precision": 0.7320261437908496,
712
+ "recall": 0.3929824561403509,
713
+ "f0.5": 0.6243032329988851,
714
+ "loads_per_turn": 0.4853932584269663,
715
+ "empty_turn_rate": 0.6674157303370787
716
+ },
717
+ {
718
+ "threshold": 0.95,
719
+ "counts": {
720
+ "out_of_catalog_labels": 31,
721
+ "out_of_catalog_positives": 0,
722
+ "turns": 445,
723
+ "loads": 137,
724
+ "empty": 338,
725
+ "needed_turns": 216,
726
+ "any_needed_loaded": 77,
727
+ "all_needed_loaded": 53,
728
+ "tn": 29780,
729
+ "fp": 18,
730
+ "fn": 202,
731
+ "unknown_loaded": 36,
732
+ "tp": 83
733
+ },
734
+ "precision": 0.8217821782178217,
735
+ "recall": 0.2912280701754386,
736
+ "f0.5": 0.602322206095791,
737
+ "loads_per_turn": 0.30786516853932583,
738
+ "empty_turn_rate": 0.7595505617977528
739
+ },
740
+ {
741
+ "threshold": 0.97,
742
+ "counts": {
743
+ "out_of_catalog_labels": 31,
744
+ "out_of_catalog_positives": 0,
745
+ "turns": 445,
746
+ "loads": 85,
747
+ "empty": 374,
748
+ "needed_turns": 216,
749
+ "any_needed_loaded": 51,
750
+ "all_needed_loaded": 37,
751
+ "tn": 29788,
752
+ "fp": 10,
753
+ "fn": 231,
754
+ "unknown_loaded": 21,
755
+ "tp": 54
756
+ },
757
+ "precision": 0.84375,
758
+ "recall": 0.18947368421052632,
759
+ "f0.5": 0.49907578558225507,
760
+ "loads_per_turn": 0.19101123595505617,
761
+ "empty_turn_rate": 0.8404494382022472
762
+ },
763
+ {
764
+ "threshold": 0.99,
765
+ "counts": {
766
+ "out_of_catalog_labels": 31,
767
+ "out_of_catalog_positives": 0,
768
+ "turns": 445,
769
+ "loads": 12,
770
+ "empty": 433,
771
+ "needed_turns": 216,
772
+ "any_needed_loaded": 9,
773
+ "all_needed_loaded": 5,
774
+ "tn": 29798,
775
+ "fn": 276,
776
+ "unknown_loaded": 3,
777
+ "tp": 9
778
+ },
779
+ "precision": 1.0,
780
+ "recall": 0.031578947368421054,
781
+ "f0.5": 0.14018691588785046,
782
+ "loads_per_turn": 0.02696629213483146,
783
+ "empty_turn_rate": 0.9730337078651685
784
+ }
785
+ ],
786
+ "called": [
787
+ {
788
+ "threshold": 0.5,
789
+ "counts": {
790
+ "out_of_catalog_labels": 31,
791
+ "out_of_catalog_positives": 0,
792
+ "turns": 445,
793
+ "loads": 639,
794
+ "empty": 176,
795
+ "needed_turns": 228,
796
+ "any_needed_loaded": 170,
797
+ "all_needed_loaded": 74,
798
+ "tn": 30152,
799
+ "fn": 280,
800
+ "fp": 395,
801
+ "tp": 244
802
+ },
803
+ "precision": 0.38184663536776214,
804
+ "recall": 0.46564885496183206,
805
+ "f0.5": 0.3961038961038961,
806
+ "loads_per_turn": 1.4359550561797754,
807
+ "empty_turn_rate": 0.3955056179775281
808
+ },
809
+ {
810
+ "threshold": 0.6,
811
+ "counts": {
812
+ "out_of_catalog_labels": 31,
813
+ "out_of_catalog_positives": 0,
814
+ "turns": 445,
815
+ "loads": 539,
816
+ "empty": 191,
817
+ "needed_turns": 228,
818
+ "any_needed_loaded": 165,
819
+ "all_needed_loaded": 68,
820
+ "tn": 30235,
821
+ "fn": 297,
822
+ "fp": 312,
823
+ "tp": 227
824
+ },
825
+ "precision": 0.42115027829313545,
826
+ "recall": 0.43320610687022904,
827
+ "f0.5": 0.42350746268656714,
828
+ "loads_per_turn": 1.2112359550561798,
829
+ "empty_turn_rate": 0.42921348314606744
830
+ },
831
+ {
832
+ "threshold": 0.7,
833
+ "counts": {
834
+ "out_of_catalog_labels": 31,
835
+ "out_of_catalog_positives": 0,
836
+ "turns": 445,
837
+ "loads": 429,
838
+ "empty": 222,
839
+ "needed_turns": 228,
840
+ "any_needed_loaded": 148,
841
+ "all_needed_loaded": 57,
842
+ "tn": 30312,
843
+ "fn": 330,
844
+ "fp": 235,
845
+ "tp": 194
846
+ },
847
+ "precision": 0.4522144522144522,
848
+ "recall": 0.3702290076335878,
849
+ "f0.5": 0.4330357142857143,
850
+ "loads_per_turn": 0.9640449438202248,
851
+ "empty_turn_rate": 0.49887640449438203
852
+ },
853
+ {
854
+ "threshold": 0.75,
855
+ "counts": {
856
+ "out_of_catalog_labels": 31,
857
+ "out_of_catalog_positives": 0,
858
+ "turns": 445,
859
+ "loads": 371,
860
+ "empty": 238,
861
+ "needed_turns": 228,
862
+ "any_needed_loaded": 141,
863
+ "all_needed_loaded": 51,
864
+ "tn": 30354,
865
+ "fn": 346,
866
+ "fp": 193,
867
+ "tp": 178
868
+ },
869
+ "precision": 0.4797843665768194,
870
+ "recall": 0.33969465648854963,
871
+ "f0.5": 0.44322709163346613,
872
+ "loads_per_turn": 0.8337078651685393,
873
+ "empty_turn_rate": 0.5348314606741573
874
+ },
875
+ {
876
+ "threshold": 0.8,
877
+ "counts": {
878
+ "out_of_catalog_labels": 31,
879
+ "out_of_catalog_positives": 0,
880
+ "turns": 445,
881
+ "loads": 307,
882
+ "empty": 261,
883
+ "needed_turns": 228,
884
+ "any_needed_loaded": 129,
885
+ "all_needed_loaded": 45,
886
+ "tn": 30395,
887
+ "fn": 369,
888
+ "fp": 152,
889
+ "tp": 155
890
+ },
891
+ "precision": 0.504885993485342,
892
+ "recall": 0.2958015267175573,
893
+ "f0.5": 0.442351598173516,
894
+ "loads_per_turn": 0.6898876404494382,
895
+ "empty_turn_rate": 0.5865168539325842
896
+ },
897
+ {
898
+ "threshold": 0.85,
899
+ "counts": {
900
+ "out_of_catalog_labels": 31,
901
+ "out_of_catalog_positives": 0,
902
+ "turns": 445,
903
+ "loads": 267,
904
+ "empty": 278,
905
+ "needed_turns": 228,
906
+ "any_needed_loaded": 116,
907
+ "all_needed_loaded": 36,
908
+ "tn": 30417,
909
+ "fn": 387,
910
+ "fp": 130,
911
+ "tp": 137
912
+ },
913
+ "precision": 0.5131086142322098,
914
+ "recall": 0.26145038167938933,
915
+ "f0.5": 0.43027638190954776,
916
+ "loads_per_turn": 0.6,
917
+ "empty_turn_rate": 0.6247191011235955
918
+ },
919
+ {
920
+ "threshold": 0.89,
921
+ "counts": {
922
+ "out_of_catalog_labels": 31,
923
+ "out_of_catalog_positives": 0,
924
+ "turns": 445,
925
+ "loads": 222,
926
+ "empty": 295,
927
+ "needed_turns": 228,
928
+ "any_needed_loaded": 105,
929
+ "all_needed_loaded": 31,
930
+ "tn": 30444,
931
+ "fn": 405,
932
+ "fp": 103,
933
+ "tp": 119
934
+ },
935
+ "precision": 0.536036036036036,
936
+ "recall": 0.22709923664122136,
937
+ "f0.5": 0.4213881019830028,
938
+ "loads_per_turn": 0.49887640449438203,
939
+ "empty_turn_rate": 0.6629213483146067
940
+ },
941
+ {
942
+ "threshold": 0.9,
943
+ "counts": {
944
+ "out_of_catalog_labels": 31,
945
+ "out_of_catalog_positives": 0,
946
+ "turns": 445,
947
+ "loads": 216,
948
+ "empty": 297,
949
+ "needed_turns": 228,
950
+ "any_needed_loaded": 103,
951
+ "all_needed_loaded": 31,
952
+ "tn": 30448,
953
+ "fn": 407,
954
+ "fp": 99,
955
+ "tp": 117
956
+ },
957
+ "precision": 0.5416666666666666,
958
+ "recall": 0.22328244274809161,
959
+ "f0.5": 0.4214697406340058,
960
+ "loads_per_turn": 0.4853932584269663,
961
+ "empty_turn_rate": 0.6674157303370787
962
+ },
963
+ {
964
+ "threshold": 0.95,
965
+ "counts": {
966
+ "out_of_catalog_labels": 31,
967
+ "out_of_catalog_positives": 0,
968
+ "turns": 445,
969
+ "loads": 137,
970
+ "empty": 338,
971
+ "needed_turns": 228,
972
+ "any_needed_loaded": 71,
973
+ "all_needed_loaded": 19,
974
+ "tn": 30485,
975
+ "fn": 449,
976
+ "fp": 62,
977
+ "tp": 75
978
+ },
979
+ "precision": 0.5474452554744526,
980
+ "recall": 0.1431297709923664,
981
+ "f0.5": 0.3498134328358209,
982
+ "loads_per_turn": 0.30786516853932583,
983
+ "empty_turn_rate": 0.7595505617977528
984
+ },
985
+ {
986
+ "threshold": 0.97,
987
+ "counts": {
988
+ "out_of_catalog_labels": 31,
989
+ "out_of_catalog_positives": 0,
990
+ "turns": 445,
991
+ "loads": 85,
992
+ "empty": 374,
993
+ "needed_turns": 228,
994
+ "any_needed_loaded": 45,
995
+ "all_needed_loaded": 11,
996
+ "tn": 30511,
997
+ "fn": 475,
998
+ "fp": 36,
999
+ "tp": 49
1000
+ },
1001
+ "precision": 0.5764705882352941,
1002
+ "recall": 0.09351145038167939,
1003
+ "f0.5": 0.2835648148148148,
1004
+ "loads_per_turn": 0.19101123595505617,
1005
+ "empty_turn_rate": 0.8404494382022472
1006
+ },
1007
+ {
1008
+ "threshold": 0.99,
1009
+ "counts": {
1010
+ "out_of_catalog_labels": 31,
1011
+ "out_of_catalog_positives": 0,
1012
+ "turns": 445,
1013
+ "loads": 12,
1014
+ "empty": 433,
1015
+ "needed_turns": 228,
1016
+ "any_needed_loaded": 7,
1017
+ "all_needed_loaded": 1,
1018
+ "tn": 30542,
1019
+ "fn": 517,
1020
+ "fp": 5,
1021
+ "tp": 7
1022
+ },
1023
+ "precision": 0.5833333333333334,
1024
+ "recall": 0.013358778625954198,
1025
+ "f0.5": 0.06118881118881119,
1026
+ "loads_per_turn": 0.02696629213483146,
1027
+ "empty_turn_rate": 0.9730337078651685
1028
+ }
1029
+ ]
1030
+ }
1031
+ },
1032
+ "B5-094": {
1033
+ "original_report_sha256": "35c09c36938053b2eb285f4ca8700e2b61b0bb2ba96f1a63d1bbe2004e929bcf",
1034
+ "input_sha256": {
1035
+ "predictions": "e70357d58524a0714882aef5c5de98a6c4cf74a3ad82ae23ee1ed9f97789d1c0",
1036
+ "rows": "67c4c86e3385f0241048f2633d97a4aad78a53a3b7a540592937a20eba669fd3",
1037
+ "corpus": "c77abd8b40965c05cc43a7545e8c2db2405bdcfc18a73bad254632d523b67bcd"
1038
+ },
1039
+ "prediction_metadata_sha256": "de8a6974d1ca4a615f319089b2a6b023e5e57ede9bc03e52de5aa9524bde11c9",
1040
+ "encoding": "independent",
1041
+ "tool_option_words": 60,
1042
+ "head_sha256": "550f30b94d5c6a8680d2cb2a607ebc9ee33e709c7679f72606d1e9fda17f2e90",
1043
+ "truth": {
1044
+ "consensus": [
1045
+ {
1046
+ "threshold": 0.95,
1047
+ "counts": {
1048
+ "out_of_catalog_labels": 31,
1049
+ "out_of_catalog_positives": 0,
1050
+ "turns": 445,
1051
+ "loads": 137,
1052
+ "empty": 338,
1053
+ "needed_turns": 216,
1054
+ "any_needed_loaded": 77,
1055
+ "all_needed_loaded": 53,
1056
+ "tn": 29780,
1057
+ "fp": 18,
1058
+ "fn": 202,
1059
+ "unknown_loaded": 36,
1060
+ "tp": 83
1061
+ },
1062
+ "precision": 0.8217821782178217,
1063
+ "recall": 0.2912280701754386,
1064
+ "f0.5": 0.602322206095791,
1065
+ "loads_per_turn": 0.30786516853932583,
1066
+ "empty_turn_rate": 0.7595505617977528
1067
+ },
1068
+ {
1069
+ "threshold": 0.94,
1070
+ "counts": {
1071
+ "out_of_catalog_labels": 31,
1072
+ "out_of_catalog_positives": 0,
1073
+ "turns": 445,
1074
+ "loads": 161,
1075
+ "empty": 326,
1076
+ "needed_turns": 216,
1077
+ "any_needed_loaded": 83,
1078
+ "all_needed_loaded": 58,
1079
+ "tn": 29775,
1080
+ "fp": 23,
1081
+ "fn": 194,
1082
+ "unknown_loaded": 47,
1083
+ "tp": 91
1084
+ },
1085
+ "precision": 0.7982456140350878,
1086
+ "recall": 0.3192982456140351,
1087
+ "f0.5": 0.6140350877192983,
1088
+ "loads_per_turn": 0.36179775280898874,
1089
+ "empty_turn_rate": 0.7325842696629213
1090
+ }
1091
+ ],
1092
+ "called": [
1093
+ {
1094
+ "threshold": 0.95,
1095
+ "counts": {
1096
+ "out_of_catalog_labels": 31,
1097
+ "out_of_catalog_positives": 0,
1098
+ "turns": 445,
1099
+ "loads": 137,
1100
+ "empty": 338,
1101
+ "needed_turns": 228,
1102
+ "any_needed_loaded": 71,
1103
+ "all_needed_loaded": 19,
1104
+ "tn": 30485,
1105
+ "fn": 449,
1106
+ "fp": 62,
1107
+ "tp": 75
1108
+ },
1109
+ "precision": 0.5474452554744526,
1110
+ "recall": 0.1431297709923664,
1111
+ "f0.5": 0.3498134328358209,
1112
+ "loads_per_turn": 0.30786516853932583,
1113
+ "empty_turn_rate": 0.7595505617977528
1114
+ },
1115
+ {
1116
+ "threshold": 0.94,
1117
+ "counts": {
1118
+ "out_of_catalog_labels": 31,
1119
+ "out_of_catalog_positives": 0,
1120
+ "turns": 445,
1121
+ "loads": 161,
1122
+ "empty": 326,
1123
+ "needed_turns": 228,
1124
+ "any_needed_loaded": 83,
1125
+ "all_needed_loaded": 21,
1126
+ "tn": 30476,
1127
+ "fn": 434,
1128
+ "fp": 71,
1129
+ "tp": 90
1130
+ },
1131
+ "precision": 0.5590062111801242,
1132
+ "recall": 0.1717557251908397,
1133
+ "f0.5": 0.3852739726027397,
1134
+ "loads_per_turn": 0.36179775280898874,
1135
+ "empty_turn_rate": 0.7325842696629213
1136
+ }
1137
+ ]
1138
+ }
1139
+ }
1140
+ }
1141
+ }
evidence/unit-rank-historical-acceptance.json ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "scope": "Historical E4 acceptance under the earlier listing and preload policy; not current free-text lookup or current preload acceptance.",
3
+ "head_sha256": "c62674993eabba94660eccfafbb6e04d1549adc7335656c0ddefa7f068fa58f7",
4
+ "original_report_sha256": "aa6910d6a8c4769fea3aeb987fe351c735772780a189b06c837d678e6af334df",
5
+ "original_verdict_sha256": "17d97a6e41623c55e5f2d678cf4f442097ee3ea2095e419522fcefbc2cf3815b",
6
+ "head_binding_record_sha256": "769aaafde400eb9b77238ff6c1fbf1136761b58131f8a2554ba86234f52d811f",
7
+ "policy": {
8
+ "preload_at": 3.99,
9
+ "list_at": 0.5,
10
+ "request": {
11
+ "load_at": 1.95,
12
+ "max_loads": 4,
13
+ "nearest_loads": 1
14
+ }
15
+ },
16
+ "skills": {
17
+ "rows": 789,
18
+ "no_skill_rows": 50,
19
+ "generated_none_rows": 57,
20
+ "generated_none_agreed_negative": 23,
21
+ "relevant_rows": 660,
22
+ "relevant_preloaded": 122,
23
+ "any_raw@6": 0.9212121212121213,
24
+ "any_visible@6": 0.9212121212121213,
25
+ "all_visible@6": 0.7666666666666667,
26
+ "any_raw@8": 0.943939393939394,
27
+ "any_visible@8": 0.943939393939394,
28
+ "all_visible@8": 0.8090909090909091,
29
+ "preloads": 123,
30
+ "preloads_unscored": 0,
31
+ "preloads_correct": 122,
32
+ "no_skill_preloads": 0,
33
+ "generated_none_preloads": 0,
34
+ "preloads_possible": 1,
35
+ "preload_precision": 0.991869918699187,
36
+ "no_skill_preload_rate": 0.0,
37
+ "bands": {
38
+ "rare": {
39
+ "support": 572,
40
+ "visible@6": 0.8041958041958042,
41
+ "macro_support3": 0.8244539343687479
42
+ },
43
+ "mid": {
44
+ "support": 254,
45
+ "visible@6": 0.84251968503937,
46
+ "macro_support3": 0.8774649925511994
47
+ },
48
+ "common": {
49
+ "support": 257,
50
+ "visible@6": 0.8793774319066148,
51
+ "macro_support3": 0.8631465825713925
52
+ }
53
+ }
54
+ },
55
+ "requests": {
56
+ "needs": 0,
57
+ "top1": null,
58
+ "served": null,
59
+ "precision": null,
60
+ "f1": 0,
61
+ "loads_per_need": null
62
+ },
63
+ "verdict": {
64
+ "checks": {
65
+ "rare_pooled": true,
66
+ "rare_macro": true,
67
+ "mid_pooled": true,
68
+ "mid_macro": true,
69
+ "common_pooled": false,
70
+ "no_category_collapse": true,
71
+ "preload_precision": true,
72
+ "negative_false_preloads": true,
73
+ "preload_coverage": true
74
+ },
75
+ "skills_passed": false
76
+ },
77
+ "transformation": "Aggregate policy, skills and request metrics retained; per-skill and redundant strata breakdowns omitted. Verdict booleans retained."
78
+ }
release.json ADDED
@@ -0,0 +1,149 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "version": 1,
3
+ "release": "open1-b5-e4",
4
+ "backbone": {
5
+ "model": "convaiinnovations/laya-multilingual",
6
+ "revision": "e4e9ddf21a7b1903b7acffd8814ad4307bf63a67"
7
+ },
8
+ "heads": {
9
+ "turn-load": {
10
+ "sha256": "550f30b94d5c6a8680d2cb2a607ebc9ee33e709c7679f72606d1e9fda17f2e90",
11
+ "label": "open1-turn-load-B5-release-independent"
12
+ },
13
+ "unit-rank": {
14
+ "sha256": "c62674993eabba94660eccfafbb6e04d1549adc7335656c0ddefa7f068fa58f7",
15
+ "label": "open1-unit-rank-e4-dense1"
16
+ },
17
+ "code-rank": {
18
+ "sha256": "abe235c347a223e957a0f8b7b979e3a584aebab76ee239e5f62117e839b9c40a",
19
+ "label": "open1-code-rank"
20
+ }
21
+ },
22
+ "preload": {
23
+ "catalog_revision": "dc3bf9215aac8877",
24
+ "encoding": "independent",
25
+ "option_words": 60,
26
+ "max_len": 1024,
27
+ "head_tokens": 512,
28
+ "options": {
29
+ "answer_decision": "Answer a worker's needs_decision: pick one of its offered options; the worker resumes on its same session with your choice.",
30
+ "ask_user": "Structured question for the user \u2014 this tool is the only way to ask one, and assistant prose is not an answer. Host parks until the user resolves; next wake returns this tool result. Prefer choices[] when present. Parallel while pending \u2192 ASK_USER_ALREADY_PENDING. Fill args per field docs below.",
31
+ "capture_page": "One-shot open\u2192act\u2192snapshot\u2192close; prefer page_* for live/authenticated state. Use project_dir for a static tree or url plus process_handle for a loopback app. Targets are exclusive. Returns structure, requests, errors, console, and a screenshot; use filmstrip for settled sequence claims and timeline for anything that happens while the page is painting.",
32
+ "chmod": "Change approval bits on repo-relative paths.",
33
+ "chown": "Change file ownership within write scope (never use command chown).",
34
+ "code_rewrite": "Rewrite a code shape across path or paths[]. $VAR matches one node; $$$VAR a sequence. Narrow bare `$A | $B` patterns (CODE_REWRITE_PATTERN_TOO_BROAD). Parse-breaking rewrites are refused (MUTATION_BROKE_PARSE). For search, use grep structural: true. Track changed, blocked (code/details), failed, unobserved_files and truncated separately: blocked is not no-match, and successful siblings may already be saved. Address each cause without repeating saved mutations",
35
+ "command": "Needs `command` or `pipeline` (not both). Runs as argv without a shell: `&&`/`||`/`;` sequence commands, `pipeline` carries `|`, `>`/`>>`/`2>` redirect, globs and `@scratch/<path>` expand, `$()`/backticks reject. Leading `NAME=value` assignments scope to their command; shared variables belong in `env`. Pass cwd; `cd` is rejected. Omit output paths to return output tails.",
36
+ "command_output": "Read command job output since cursor and current host-observed network facts. Inspect the handle of the process making a failed connection before attributing an application error to the sandbox. `network` records observed destinations and allow/deny decisions; `boundary_refusal` reports host attribution. Allowed access does not imply a successful upstream response.",
37
+ "command_stop": "Request termination of a command job process tree. stop_requested acknowledges the request; running stays true until exit is observed. Request once, then wait on the handle or inspect command_output rather than repeating stop.",
38
+ "complete_leg": "Finish this worker leg. Call once when the assignment is done or blocked. The host builds the parent report from these fields plus disk proof.",
39
+ "copy": "Copy files within write scope (files only; never use command cp).",
40
+ "delegate_decompose": "Decompose an approved plan into delegation legs",
41
+ "delegate_dispatch": "Dispatch a delegation leg to the worker queue",
42
+ "delegate_init": "Create a delegation departure for the current coordinator session",
43
+ "delegate_status": "Return delegation departure status and legs",
44
+ "delete": "Remove files and empty directories within write scope; the person can undo each removal.",
45
+ "diff": "Unified diff between two repo-relative file paths (read-scope only; never use command diff).",
46
+ "edit": "Replace an exact span in a write-scope file. Batch independent edits to distinct files. Read in an earlier response (same-batch read+edit rejects: EDITOR_DOCUMENT_READ_REQUIRED). Copy old_string after line-number tab; unique unless replace_all. Parse-breaking splices: MUTATION_BROKE_PARSE. New files: write.",
47
+ "extend_worker_budget": "Grant a running worker a higher tool-round ceiling, usually in answer to its request_budget; the worker is told on its next round and its request closes.",
48
+ "extract_archive": "Unpack .zip or .tar.gz into a write-scoped directory (replaces unzip/tar xzf; HITL ask by default).",
49
+ "fanout_plan": "Stamp a planned parallel fan-out (1\u2013N read-only legs). Call on fanout_planned phases, then workflow_advance. Stamping records the plan only; it does not enqueue workers. In the worker phase, dispatch each leg with task and its host-issued workflow_work_id. Recover incomplete legs within the plan's attempt allowance.",
50
+ "fetch_url": "Read a web page, documentation site, or other online resource. mode=text (default) extracts readable page text (HTML\u2192markdown); large bodies return a head + symbol map \u2014 page with offset+limit (cached). mode=raw returns the verbatim resource: textual MIME (HTML/CSS/JS/JSON/SVG/\u2026) as text with the same paging; image MIME types without dest automatically ingest into the session visual store and emit visual captures",
51
+ "find": "Bounded recursive directory walk \u2014 canonical for ** name_glob. Default type=any returns files and directories together. Wide unscoped walk at path=\".\" (2000+ paths) returns view digest (selected=0) with distribution + FIND_OVERFLOW_NARROW \u2014 pass name_glob when you can name a filename pattern. A concrete path (not \".\") or name_glob returns literal results[] (same as list_dir targeted). Explicit max_results or offset also",
52
+ "git_blame": "Per-line blame JSON (`available`, `lines` with commit/author/date/content).",
53
+ "git_branches": "List local branches JSON (`available`, `branches` with name/current).",
54
+ "git_checkout": "Switch to an existing local branch, or explicitly create and switch to a new one. Rehearses and reviews file changes before applying. Preserves unrelated local edits; refuses conflicting edits. Reports the actual branch and HEAD even after failure.",
55
+ "git_commit": "Stage and commit paths with a message; amend=true replaces HEAD with the selected paths, including new files, leaving unrelated staged changes alone. Success JSON: available, hash, paths, and uncommitted_paths/uncommitted_count for dirty files this commit left alone. Failure JSON: available=false, error, optional hook_failed, output_tail (last hook/CLI lines), and hint. Empty paths stages only files this session wrote; name paths explicitly to",
56
+ "git_compare": "Compare frozen commit IDs: ahead/behind counts for head_ref relative to base_ref, plus all merge bases.",
57
+ "git_diff": "Git diff JSON, one page of files in git's order (`available`, `files` with per-file insertions/deletions and, unless `stat`, that file's hunks in `diff`; `available:false` on non-git projects). Pages hold up to 80 files and 5 KiB of hunks by default: read `files_total`, use `next_offset` for more files when needed; exhaust the selection for tasks requiring complete coverage. `wire_spill_path` retains the",
58
+ "git_log": "Read paged commit history from a branch, tag, commit, or revision range. Use ref for revisions and path only for a file. main..feature lists commits unique to feature; main...feature lists both sides. Follow next_offset to continue.",
59
+ "git_merge": "Merge a revision into the current branch. Rehearses and reviews file effects. Conflicts are explicit and retain the merge state: edit resolutions, then continue with explicit paths; abort only when asked to discard merge work. Check status and after state, not diagnostics, for success.",
60
+ "git_ref": "Resolve refs to SHAs JSON (`available`, `refs` with ref/sha/peeled).",
61
+ "git_restore": "Restore explicit paths in the Git worktree, index, or both. Defaults to Git's ordinary worktree-from-index behavior. This discards selected state \u2014 use only when the user explicitly asked to revert named paths.",
62
+ "git_show": "Show commit or file at ref JSON (`available`, `show` with kind diff/blob).",
63
+ "git_stash": "Save explicit repository paths temporarily, apply a saved stash, or explicitly drop one. Save preserves unrelated edits and index entries. Apply retains the stash even on success; use drop separately after verifying restoration. Conflicts and partial effects are reported in structured state.",
64
+ "git_stash_list": "List stashes with immutable IDs and current selectors for apply/drop.",
65
+ "git_status": "Git working tree status JSON (`available`, `branch`, `head_short`, `upstream`, `ahead`/`behind`, `dirty`, staged/unstaged/untracked counts, `files` with porcelain XY status one entry per file \u2014 untracked directories are expanded; `available:false` on non-git projects). Pages of 80 files: read `files_total`, use `next_offset` to continue when more coverage is needed; exhaust pages when the task requires the complete selection. `paths` narrows the selection to",
66
+ "grep": "Search files with RE2 text patterns or tree-sitter structural patterns. Recurses under path with optional path_glob, searches gitignored files, and skips engine/VCS metadata. Wide results may return a view digest.",
67
+ "handoff_health": "Report reservation counts for a handoff session plane.",
68
+ "handoff_init": "Bind the parent session handoff plane for sibling worker coordination.",
69
+ "handoff_release": "Release one or more path reservations held by the caller agent.",
70
+ "handoff_release_all": "Release every path reservation held by the caller agent on the session plane.",
71
+ "handoff_reserve": "Record repo-relative path reservations on the parent session plane (informational; merge resolves overlaps).",
72
+ "held_result": "Read a held call by its handle, running or settled with its original outcome, codes, and result. wait_ms waits up to 30 s for a running call.",
73
+ "held_stop": "Cancel a running held call. Request once, then wait on the handle or read call_result.",
74
+ "http_request": "Send one bounded request to an API, webhook, or service endpoint and return its status, headers, and raw body.",
75
+ "jq": "Query JSON, YAML, or TOML via sandboxed jq. Pass limit/offset for literal values[].",
76
+ "jq_edit": "Update a JSON, YAML, or TOML document with a jq program and write the result in the same format, in place or to dest. The program returns the whole document (`.a.b = 1`, `del(.x)`, `.items |= map(...)`). Unchanged values keep their literals, key order, and YAML comments; syntax a rewrite would lose is refused (JQ_EDIT_LOSSY).",
77
+ "list_dir": "Locate repository areas with {\"path\":\".\"}. A named directory or explicit listing controls return entries instead. Use summarize to explain the area.",
78
+ "measure_page": "Measure selector geometry in CSS pixels, and whether the pointer can reach each element or what covers or clips it. Use id for an existing live page, or one exclusive project_dir/loopback url target. annotate adds boxes; page_geometry remains the numeric evidence.",
79
+ "mkdir": "Create directories within write scope (never use command mkdir).",
80
+ "move": "Rename files within write scope (never use command mv); a replaced destination can be undone.",
81
+ "pack_board": "When the board is stale, workers changed, or you need a scan/layout pulse \u2014 return the board snapshot. Not a substitute for summarize or a known-file read.",
82
+ "page_act": "Drive a held page with trusted input. Each step aims where its target receives the pointer; the result reports that point plus the requests, errors, and console output the drive caused. With record, the drive is also a timeline seen as a labeled contact sheet.",
83
+ "page_close": "Tear down a held page; optional final snapshot.",
84
+ "page_open": "Open a live page; returns an id held across turns. The page has its own cookies and storage, which persist until page_close. Up to max_pages stay open per session; re-opening a held target reloads it with the given viewport and routes. Prefer capture_page for a single check. Bindings match capture_page (project_dir or url).",
85
+ "page_snapshot": "Universal {state, snapshot, log, artifact} for a held page (does not close it).",
86
+ "parse_decomposition": "Validate decomposition.",
87
+ "parse_delegation_plan": "Validate delegation plan.",
88
+ "parse_evaluation": "Validate evaluation.",
89
+ "parse_extract_json": "Extract JSON.",
90
+ "parse_validate": "Validate JSON.",
91
+ "plan_append_review_evidence": "Append advisory plan review evidence to critic.jsonl",
92
+ "preview_overlay": "Merge preview \u2014 same plan as promote_overlay without landing. overlay_id == job_id. Returns path_status and ready_resolutions; spill_path is host-data-relative (promote-spills/<id>.json). Example: {\"overlay_id\":\"<job_id>\"}.",
93
+ "process_list": "Inspect host processes after approval. Returns names, executable paths, UID, parent PID, and opaque instance references; no argv or environment. Use pid for a known target. Use next_after_pid for another page of the changing snapshot. Unavailable metadata does not mean exited. Use command_output and command_stop for jobs you launched with command.",
94
+ "process_signal": "Send a signal to exact host process instances after approval. Pass opaque references returned by process_list in this task. PID reuse or changed identity rejects; names, bare PIDs, groups, and descendants are not targets. TERM requests graceful termination; KILL can discard unsaved work. Delivered means the signal was accepted, not that exit was observed. Inspect the result for each target.",
95
+ "promote_overlay": "Atomically promote a worker overlay (overlay_id=job_id), considering all observed changes. Conflicts return a menu without writes; resolve all via resolutions/drop.",
96
+ "read": "Read a file and return its content or structure in JSON; unscoped large files return a structure digest. For literal content, set mode / offset / limit / ranges / symbol.",
97
+ "recall": "Look up what was already observed instead of observing it again. Reach for it when a finished worker leg's envelope omits a detail the leg saw, when a continuation record or a `[compacted \u2026]` banner names a handle whose body is gone from context, or \u2014 from a root session \u2014 when an earlier session on this project may already",
98
+ "record_finding": "Post peer intent for parallel siblings \u2014 shared paths, registry slots, merge-relevant facts. Cite a real path/handle.",
99
+ "reject_overlay": "Abort overlay without landing. overlay_id == job_id. Wrong/superseded only \u2014 not landable conflicts. Example: {\"overlay_id\":\"<job_id>\",\"reason\":\"Superseded by corrected dispatch\"}.",
100
+ "render_view": "Rasterize SVG or HTML into an authored visual: UI/product mockups (especially before building), diagrams, comparisons, timelines, or charts. Use when spatial layout improves clarity; never as runtime evidence \u2014 use capture_page for observed UI. Host injects an offline design kit (OFL fonts, Lucide-adapted icons, theme tokens, spacing/type scales, media placeholders). No CDN links, scripts, or external https assets \u2014 use",
101
+ "replace_lines": "Atomically replace or reindent line ranges. Use direct fields once, or operations for coupled non-overlapping seams in one snapshot. Read existing editable files in an earlier response before applying changes; concurrent edits rebase against that read. EDITOR_DOCUMENT_READ_REQUIRED or EDITOR_DOCUMENT_CHANGING requires read and recomputation.",
102
+ "request_budget": "Ask your coordinator for more tool rounds when this leg's goal needs more than remain. Name the rounds and the concrete work they cover. You keep working while the coordinator decides; a granted ceiling arrives as a host notice on your next round. If none arrives, finish with an honest partial report at your current ceiling. One request can be",
103
+ "request_decision": "Escalate an ambiguity you can't resolve in scope \u2014 question + 2+ options, the accepted call suspends your run. The coordinator answers and you resume with the same workspace. Do not also call complete_leg.",
104
+ "request_tools": "Load more tool schemas for the next model call by describing what the work needs",
105
+ "restore_version": "Restore a file to a previously retained source version as a forward mutation in the source ledger. Use source_history to inspect available versions or diffs.",
106
+ "scan_compare": "Diff two complete scans \u2014 returns the findings introduced, resolved, and unchanged since the baseline. After a fix, pass the post-fix scan as new_scan_id and omit old_scan_id: the host resolves the previous complete scan in the same category. Use it to prove a fix landed; a board Scan: tail is not proof.",
107
+ "scan_list": "List recent code scans for the active project (read-only). Returns a slim id roster with no guidance bodies \u2014 use it to resolve a scan_id for scan_summary, scan_query, or scan_compare when the board does not already show one.",
108
+ "scan_pack": "Run the host's configured code scanners over the active project and return one scan_pack. Use it when the ask is security or code-quality work and the board shows no complete scan at the current head. Returns per_scan[].scan_id \u2014 pass that exact array to scan_summary or scan_query for a cross-engine aggregate, or set completion=summary to receive the aggregate when this call",
109
+ "scan_query": "Page findings for one or more scans. One id queries one scan; the exact id array returned by scan_pack queries that pack as a cross-engine aggregate. Narrow with rule_id, path, code, or level rather than reading every finding. When a result is truncated, page it with offset \u2014 do not re-run the scan.",
110
+ "scan_summary": "Read status and guidance for one or more scans. One id reads one scan; the exact id array returned by scan_pack reads that pack as a single cross-engine aggregate, and a pass_id reads a full pass, including scanners that have not started yet. This is the first call after a scan completes \u2014 start here rather than grepping the repo",
111
+ "secret_generate": "Generate random secret material and return an opaque {{paintedwolf-secret:\u2026}} reference for tool execution.",
112
+ "secret_list": "List visible managed secret references and metadata, never values.",
113
+ "secret_revoke": "Revoke a managed secret reference while retaining provider-screening evidence.",
114
+ "skills_read": "Read a skill or resource by name, find skills by describing the work, or omit arguments to list every loaded skill.",
115
+ "source_history": "Query the source ledger for file effects (mode=effects), current line authorship (mode=lines), this session's paths (mode=mine), historical version content (mode=version), or version diffs (mode=diff). Includes uncommitted state. Actors: you / agent / user / external / mixed / unknown. Missing history is reported explicitly.",
116
+ "stat": "File metadata (mode, size, mtime, entry_count for directories)",
117
+ "state_close": "Exit the active workflow run for the session",
118
+ "state_query": "Query active workflow run and scaffold vars",
119
+ "state_start": "Propose a workflow run on the current session. The host records the proposal; an explicit human action starts it.",
120
+ "state_update": "Update model-authored scaffold data on the active workflow run",
121
+ "submit_verdict": "Record the phase's review_loop verdict per verdict_schema. Terminal verdicts satisfy its evidence_passed gate; other gates and advancement rules still apply. Non-terminal verdicts re-loop within the cap. Terminal verdicts must cite observed evidence paths/handles in cited_evidence or fetched URLs in cited_urls. Invented references reject; non-terminal verdicts need no citations.",
122
+ "summarize": "Explain a named file or area with a bounded, source-cited briefing. Path sets scope; task ranks within it. Use next_actions only when more evidence is needed.",
123
+ "surface_note": "Post a concise, grounded mid-turn note for the user, then keep working \u2014 this does not end the turn. Use it for meaningful progress or expectation changes. Say what happened and what comes next. Cite at least one cited_evidence or cited_urls entry. Carries artifact_ids, which is how a still reaches the user mid-run. For a question use ask_user; the final",
124
+ "survey_repo": "One catalog map (Tier 1 digest, selected=0), not a briefing. Call when you still cannot name a destination and the ask is unknown layout (layout_overview), duplicated env/config/stores (ssot_drift), or where HTTP routes/handlers live (api_routes). Then summarize the destination. For how/what on a named file/package/dir, use summarize(path=\u2026) \u2014 never survey_repo for depth on a path you can already say.",
125
+ "task": "Start background workers for parallel execution across concurrent legs and return control immediately. Dispatch ready legs together in the same turn or launch companion legs while work runs. For single-thread, localized changes, execute inline directly. Match the worker profile and scope to the assignment.",
126
+ "terminal_close": "Tear down a held pty (graceful then kill). It returns lifecycle acknowledgement only; use terminal_read or terminal_snapshot before closing when output matters.",
127
+ "terminal_open": "Live host terminal (pty): run a command-class argv command. Prefer command when you only need exit code/tail for yourself. Use this tool when the session must be held or the user should see the screen \u2014 then terminal_snapshot. Returns an id held across turns plus the requested observation (screen by default). Never supply known plaintext via later terminal_send \u2014 use",
128
+ "terminal_read": "Read incremental pty output since the last read (idle/timeout bounded).",
129
+ "terminal_send": "Write literal text and/or named control tokens ({Enter}, {Ctrl-C}, \u2026) to a held pty, then return the settled vt grid. Never send known plaintext passwords, tokens, or passphrases. An opaque {{paintedwolf-secret:\u2026}} reference is allowed when the interactive prompt is the consuming boundary; the host resolves and screens it.",
130
+ "terminal_snapshot": "User-visible photograph of the held terminal session (CLI output or TUI), parallel to capture_page for pages. Returns one bounded VT screen and a receipt, never a duplicate main-stream log. Use for any held session \u2014 not only full-screen TUIs. Settles on output quiescence; does not advance terminal_read's cursor.",
131
+ "update_progress": "Set session progress checklist (markdown). Replaces the whole checklist \u2014 see content for markers and caps.",
132
+ "verify": "Run a verification check and report its terminal outcome. Bare verify runs the selected project command. With no selected command, supply the documented check appropriate to the task. A different command or pipeline is rejected when a project command is selected; use command(verification: true) for targeted checks. Selecting a command does not require it after every edit. outcome is passed",
133
+ "view_image": "Inspect an image file (.svg, .png, .jpg, .jpeg, .webp, .gif) or a session visual and perceive its rendered pixels and extracted text. Returns format metadata and extracted text, and emits a visual capture for multimodal models.",
134
+ "view_video": "See moments of a screen recording or other video (.mp4, .mov, .webm), including an attached one. Returns one sheet of frames, each labeled with its timestamp, plus the video's duration and size. Without times_ms it spreads count frames across start_ms to end_ms (default: the whole video); name exact times_ms to look closer at a moment, and crop to read a",
135
+ "wait": "Park without a worker slot until a condition is satisfied or timeout expires. Use until_complete for exact process handles without periodic timer wakes; omit conditions for a timer.",
136
+ "wc": "Count lines, bytes, and words in files or recursively in directories. bytes is the true file size; truncated marks counts taken from a capped scan.",
137
+ "web_search": "Search the web by time window. The host selects Direct search or custom providers from Settings \u2192 Web research.",
138
+ "worker_cancel": "Cancel a pending or running worker job; running workers get a prose-only cancellation report before the leg ends",
139
+ "workflow_advance": "Leave the current coordinator-controlled phase when gates pass. Returns phase_gate_unmet with failed_leaves when blocked; already_advanced if host auto-advanced first. Unavailable on host-auto phases.",
140
+ "workflow_catalog_summaries": "List bundled, session-composed, and template workflow ids.",
141
+ "workflow_compose": "Compose a session workflow from YAML. Returns effective_summary; host replays coordinator_brief each Prompt.",
142
+ "workflow_compose_from_template": "Compose from a bundled template. Returns effective_summary; read prepended coordinator_brief before proposing with state_start.",
143
+ "workflow_persist": "Save a session-composed workflow to .paintedwolf/workflows/<workflow_id>/workflow.yaml (requires user confirmation).",
144
+ "workflow_transition": "Fire choice leave by transition_id (Phase exit). Pending input latches block; human_approval_awaiting does not.",
145
+ "workflow_user_feedback": "Pending user feedback for the active run. Wait for the host card \u2014 do not infer answers.",
146
+ "write": "Create/replace files in write scope (append: true appends). Batch independent writes to distinct files; stage same-file or dependent writes. New or previously clean supported source must parse (MUTATION_BROKE_PARSE); broken files must improve and finish clean. Scope violation: WRITE_SCOPE_DENIED. Read existing editable files in an earlier response; concurrent edits rebase against it. EDITOR_DOCUMENT_READ_REQUIRED or EDITOR_DOCUMENT_CHANGING requires rereading and recomputatio"
147
+ }
148
+ }
149
+ }
turn-load.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:550f30b94d5c6a8680d2cb2a607ebc9ee33e709c7679f72606d1e9fda17f2e90
3
+ size 59881652
unit-rank.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c62674993eabba94660eccfafbb6e04d1549adc7335656c0ddefa7f068fa58f7
3
+ size 59881268