Release open1-b5-e4-bialy
Browse filesPrepared Bialy weights and training artifacts; checksums anchored in paintedwolf-ai/bialy.
Signed-off-by: Chris Beckman <chris@paintedwolf.ai>
- LICENSE +218 -0
- NOTICE +16 -0
- PROVENANCE.json +114 -0
- README.md +112 -0
- SHA256SUMS +11 -0
- code-rank.safetensors +3 -0
- evidence/host-policy-context.json +15 -0
- evidence/tool-validation.json +1141 -0
- evidence/unit-rank-historical-acceptance.json +78 -0
- release.json +149 -0
- turn-load.safetensors +3 -0
- unit-rank.safetensors +3 -0
LICENSE
ADDED
|
@@ -0,0 +1,218 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
|
| 2 |
+
Apache License
|
| 3 |
+
Version 2.0, January 2004
|
| 4 |
+
http://www.apache.org/licenses/
|
| 5 |
+
|
| 6 |
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
| 7 |
+
|
| 8 |
+
1. Definitions.
|
| 9 |
+
|
| 10 |
+
"License" shall mean the terms and conditions for use, reproduction,
|
| 11 |
+
and distribution as defined by Sections 1 through 9 of this document.
|
| 12 |
+
|
| 13 |
+
"Licensor" shall mean the copyright owner or entity authorized by
|
| 14 |
+
the copyright owner that is granting the License.
|
| 15 |
+
|
| 16 |
+
"Legal Entity" shall mean the union of the acting entity and all
|
| 17 |
+
other entities that control, are controlled by, or are under common
|
| 18 |
+
control with that entity. For the purposes of this definition,
|
| 19 |
+
"control" means (i) the power, direct or indirect, to cause the
|
| 20 |
+
direction or management of such entity, whether by contract or
|
| 21 |
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
| 22 |
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
| 23 |
+
|
| 24 |
+
"You" (or "Your") shall mean an individual or Legal Entity
|
| 25 |
+
exercising permissions granted by this License.
|
| 26 |
+
|
| 27 |
+
"Source" form shall mean the preferred form for making modifications,
|
| 28 |
+
including but not limited to software source code, documentation
|
| 29 |
+
source, and configuration files.
|
| 30 |
+
|
| 31 |
+
"Object" form shall mean any form resulting from mechanical
|
| 32 |
+
transformation or translation of a Source form, including but
|
| 33 |
+
not limited to compiled object code, generated documentation,
|
| 34 |
+
and conversions to other media types.
|
| 35 |
+
|
| 36 |
+
"Work" shall mean the work of authorship, whether in Source or
|
| 37 |
+
Object form, made available under the License, as indicated by a
|
| 38 |
+
copyright notice that is included in or attached to the work
|
| 39 |
+
(an example is provided in the Appendix below).
|
| 40 |
+
|
| 41 |
+
"Derivative Works" shall mean any work, whether in Source or Object
|
| 42 |
+
form, that is based on (or derived from) the Work and for which the
|
| 43 |
+
editorial revisions, annotations, elaborations, or other modifications
|
| 44 |
+
represent, as a whole, an original work of authorship. For the purposes
|
| 45 |
+
of this License, Derivative Works shall not include works that remain
|
| 46 |
+
separable from, or merely link (or bind by name) to the interfaces of,
|
| 47 |
+
the Work and Derivative Works thereof.
|
| 48 |
+
|
| 49 |
+
"Contribution" shall mean any work of authorship, including
|
| 50 |
+
the original version of the Work and any modifications or additions
|
| 51 |
+
to that Work or Derivative Works thereof, that is intentionally
|
| 52 |
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
| 53 |
+
or by an individual or Legal Entity authorized to submit on behalf of
|
| 54 |
+
the copyright owner. For the purposes of this definition, "submitted"
|
| 55 |
+
means any form of electronic, verbal, or written communication sent
|
| 56 |
+
to the Licensor or its representatives, including but not limited to
|
| 57 |
+
communication on electronic mailing lists, source code control systems,
|
| 58 |
+
and issue tracking systems that are managed by, or on behalf of, the
|
| 59 |
+
Licensor for the purpose of discussing and improving the Work, but
|
| 60 |
+
excluding communication that is conspicuously marked or otherwise
|
| 61 |
+
designated in writing by the copyright owner as "Not a Contribution."
|
| 62 |
+
|
| 63 |
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
| 64 |
+
on behalf of whom a Contribution has been received by Licensor and
|
| 65 |
+
subsequently incorporated within the Work.
|
| 66 |
+
|
| 67 |
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
| 68 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 69 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 70 |
+
copyright license to reproduce, prepare Derivative Works of,
|
| 71 |
+
publicly display, publicly perform, sublicense, and distribute the
|
| 72 |
+
Work and such Derivative Works in Source or Object form.
|
| 73 |
+
|
| 74 |
+
3. Grant of Patent License. Subject to the terms and conditions of
|
| 75 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 76 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 77 |
+
(except as stated in this section) patent license to make, have made,
|
| 78 |
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
| 79 |
+
where such license applies only to those patent claims licensable
|
| 80 |
+
by such Contributor that are necessarily infringed by their
|
| 81 |
+
Contribution(s) alone or by combination of their Contribution(s)
|
| 82 |
+
with the Work to which such Contribution(s) was submitted. If You
|
| 83 |
+
institute patent litigation against any entity (including a
|
| 84 |
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
| 85 |
+
or a Contribution incorporated within the Work constitutes direct
|
| 86 |
+
or contributory patent infringement, then any patent licenses
|
| 87 |
+
granted to You under this License for that Work shall terminate
|
| 88 |
+
as of the date such litigation is filed.
|
| 89 |
+
|
| 90 |
+
4. Redistribution. You may reproduce and distribute copies of the
|
| 91 |
+
Work or Derivative Works thereof in any medium, with or without
|
| 92 |
+
modifications, and in Source or Object form, provided that You
|
| 93 |
+
meet the following conditions:
|
| 94 |
+
|
| 95 |
+
(a) You must give any other recipients of the Work or
|
| 96 |
+
Derivative Works a copy of this License; and
|
| 97 |
+
|
| 98 |
+
(b) You must cause any modified files to carry prominent notices
|
| 99 |
+
stating that You changed the files; and
|
| 100 |
+
|
| 101 |
+
(c) You must retain, in the Source form of any Derivative Works
|
| 102 |
+
that You distribute, all copyright, patent, trademark, and
|
| 103 |
+
attribution notices from the Source form of the Work,
|
| 104 |
+
excluding those notices that do not pertain to any part of
|
| 105 |
+
the Derivative Works; and
|
| 106 |
+
|
| 107 |
+
(d) If the Work includes a "NOTICE" text file as part of its
|
| 108 |
+
distribution, then any Derivative Works that You distribute must
|
| 109 |
+
include a readable copy of the attribution notices contained
|
| 110 |
+
within such NOTICE file, excluding those notices that do not
|
| 111 |
+
pertain to any part of the Derivative Works, in at least one
|
| 112 |
+
of the following places: within a NOTICE text file distributed
|
| 113 |
+
as part of the Derivative Works; within the Source form or
|
| 114 |
+
documentation, if provided along with the Derivative Works; or,
|
| 115 |
+
within a display generated by the Derivative Works, if and
|
| 116 |
+
wherever such third-party notices normally appear. The contents
|
| 117 |
+
of the NOTICE file are for informational purposes only and
|
| 118 |
+
do not modify the License. You may add Your own attribution
|
| 119 |
+
notices within Derivative Works that You distribute, alongside
|
| 120 |
+
or as an addendum to the NOTICE text from the Work, provided
|
| 121 |
+
that such additional attribution notices cannot be construed
|
| 122 |
+
as modifying the License.
|
| 123 |
+
|
| 124 |
+
You may add Your own copyright statement to Your modifications and
|
| 125 |
+
may provide additional or different license terms and conditions
|
| 126 |
+
for use, reproduction, or distribution of Your modifications, or
|
| 127 |
+
for any such Derivative Works as a whole, provided Your use,
|
| 128 |
+
reproduction, and distribution of the Work otherwise complies with
|
| 129 |
+
the conditions stated in this License.
|
| 130 |
+
|
| 131 |
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
| 132 |
+
any Contribution intentionally submitted for inclusion in the Work
|
| 133 |
+
by You to the Licensor shall be under the terms and conditions of
|
| 134 |
+
this License, without any additional terms or conditions.
|
| 135 |
+
Notwithstanding the above, nothing herein shall supersede or modify
|
| 136 |
+
the terms of any separate license agreement you may have executed
|
| 137 |
+
with Licensor regarding such Contributions.
|
| 138 |
+
|
| 139 |
+
6. Trademarks. This License does not grant permission to use the trade
|
| 140 |
+
names, trademarks, service marks, or product names of the Licensor,
|
| 141 |
+
except as required for reasonable and customary use in describing the
|
| 142 |
+
origin of the Work and reproducing the content of the NOTICE file.
|
| 143 |
+
|
| 144 |
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
| 145 |
+
agreed to in writing, Licensor provides the Work (and each
|
| 146 |
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
| 147 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
| 148 |
+
implied, including, without limitation, any warranties or conditions
|
| 149 |
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
| 150 |
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
| 151 |
+
appropriateness of using or redistributing the Work and assume any
|
| 152 |
+
risks associated with Your exercise of permissions under this License.
|
| 153 |
+
|
| 154 |
+
8. Limitation of Liability. In no event and under no legal theory,
|
| 155 |
+
whether in tort (including negligence), contract, or otherwise,
|
| 156 |
+
unless required by applicable law (such as deliberate and grossly
|
| 157 |
+
negligent acts) or agreed to in writing, shall any Contributor be
|
| 158 |
+
liable to You for damages, including any direct, indirect, special,
|
| 159 |
+
incidental, or consequential damages of any character arising as a
|
| 160 |
+
result of this License or out of the use or inability to use the
|
| 161 |
+
Work (including but not limited to damages for loss of goodwill,
|
| 162 |
+
work stoppage, computer failure or malfunction, or any and all
|
| 163 |
+
other commercial damages or losses), even if such Contributor
|
| 164 |
+
has been advised of the possibility of such damages.
|
| 165 |
+
|
| 166 |
+
9. Accepting Warranty or Additional Liability. While redistributing
|
| 167 |
+
the Work or Derivative Works thereof, You may choose to offer,
|
| 168 |
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
| 169 |
+
or other liability obligations and/or rights consistent with this
|
| 170 |
+
License. However, in accepting such obligations, You may act only
|
| 171 |
+
on Your own behalf and on Your sole responsibility, not on behalf
|
| 172 |
+
of any other Contributor, and only if You agree to indemnify,
|
| 173 |
+
defend, and hold each Contributor harmless for any liability
|
| 174 |
+
incurred by, or claims asserted against, such Contributor by reason
|
| 175 |
+
of your accepting any such warranty or additional liability.
|
| 176 |
+
|
| 177 |
+
END OF TERMS AND CONDITIONS
|
| 178 |
+
|
| 179 |
+
APPENDIX: How to apply the Apache License to your work.
|
| 180 |
+
|
| 181 |
+
To apply the Apache License to your work, attach the following
|
| 182 |
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
| 183 |
+
replaced with your own identifying information. (Don't include
|
| 184 |
+
the brackets!) The text should be enclosed in the appropriate
|
| 185 |
+
comment syntax for the file format. We also recommend that a
|
| 186 |
+
file or class name and description of purpose be included on the
|
| 187 |
+
same "printed page" as the copyright notice for easier
|
| 188 |
+
identification within third-party archives.
|
| 189 |
+
|
| 190 |
+
Copyright 2026 Painted Wolf LLC
|
| 191 |
+
|
| 192 |
+
Licensed under the Apache License, Version 2.0 (the "License");
|
| 193 |
+
you may not use this file except in compliance with the License.
|
| 194 |
+
You may obtain a copy of the License at
|
| 195 |
+
|
| 196 |
+
http://www.apache.org/licenses/LICENSE-2.0
|
| 197 |
+
|
| 198 |
+
Unless required by applicable law or agreed to in writing, software
|
| 199 |
+
distributed under the License is distributed on an "AS IS" BASIS,
|
| 200 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 201 |
+
See the License for the specific language governing permissions and
|
| 202 |
+
limitations under the License.
|
| 203 |
+
|
| 204 |
+
---
|
| 205 |
+
NOTICE
|
| 206 |
+
|
| 207 |
+
Trademarks: Painted Wolf is a trademark of Painted Wolf LLC, along with the
|
| 208 |
+
product name Painted Wolf Code and the project's logos, mascots, and brand
|
| 209 |
+
artwork. This license grants copyright permissions for source code; it
|
| 210 |
+
does not grant trademark rights. What redistributors may do with the marks
|
| 211 |
+
is in docs/trademarks.md. That policy gives themes limited permission to alter
|
| 212 |
+
the app's visual identity, including the fixed Painted Wolf Code identity's
|
| 213 |
+
theme-controlled colors and logo-glyph visibility, through the built-in theme
|
| 214 |
+
system.
|
| 215 |
+
|
| 216 |
+
Other licenses: Bundled third-party engines and vendored gate rules retain
|
| 217 |
+
their respective upstream licenses. Lycaon-authored rule YAML is licensed
|
| 218 |
+
separately under CC-BY-4.0. See docs/licensing.md.
|
NOTICE
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Bialy decision heads
|
| 2 |
+
Copyright 2026 Painted Wolf LLC
|
| 3 |
+
|
| 4 |
+
Licensed under the Apache License, Version 2.0. See LICENSE.
|
| 5 |
+
|
| 6 |
+
These heads run over a backbone they do not include:
|
| 7 |
+
|
| 8 |
+
- Laya multilingual (convaiinnovations/laya-multilingual, revision
|
| 9 |
+
e4e9ddf21a7b1903b7acffd8814ad4307bf63a67), by Convai Innovations, under
|
| 10 |
+
the Apache License, Version 2.0: https://huggingface.co/convaiinnovations/laya-multilingual
|
| 11 |
+
- mmBERT-base (jhu-clsp/mmBERT-base), the encoder Laya multilingual builds on,
|
| 12 |
+
by the Johns Hopkins University Center for Language and Speech Processing,
|
| 13 |
+
under the MIT License: https://huggingface.co/jhu-clsp/mmBERT-base
|
| 14 |
+
|
| 15 |
+
"Painted Wolf" and "Painted Wolf Code" are trademarks of Painted Wolf LLC.
|
| 16 |
+
This license grants no trademark rights.
|
PROVENANCE.json
ADDED
|
@@ -0,0 +1,114 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "pw-decide-heads-evidence/1",
|
| 3 |
+
"version": "open1-b5-e4",
|
| 4 |
+
"model_repo": "paintedwolfcode/bialy",
|
| 5 |
+
"code_repo": "https://github.com/paintedwolf-ai/bialy",
|
| 6 |
+
"backbone": {
|
| 7 |
+
"model": "convaiinnovations/laya-multilingual",
|
| 8 |
+
"revision": "e4e9ddf21a7b1903b7acffd8814ad4307bf63a67"
|
| 9 |
+
},
|
| 10 |
+
"heads": [
|
| 11 |
+
{
|
| 12 |
+
"file": "turn-load.safetensors",
|
| 13 |
+
"sha256": "550f30b94d5c6a8680d2cb2a607ebc9ee33e709c7679f72606d1e9fda17f2e90",
|
| 14 |
+
"bytes": 59881652,
|
| 15 |
+
"metadata": {
|
| 16 |
+
"by_type": "{\"loss\": {\"tools\": 0.0732}, \"tools\": {\"precision\": 0.417, \"recall\": 0.691, \"positives\": 285, \"options\": 30083}, \"tools@coordinator\": {\"precision\": 0.422, \"recall\": 0.691, \"positives\": 285, \"options\": 27810}, \"tools@worker\": {\"precision\": 0.0, \"recall\": 0.0, \"positives\": 0, \"options\": 2273}}",
|
| 17 |
+
"head_max_len": "512",
|
| 18 |
+
"format": "pw-decide-head/1",
|
| 19 |
+
"label": "open1-turn-load-B5-release-independent",
|
| 20 |
+
"max_len": "1024",
|
| 21 |
+
"tool_option_words": "60",
|
| 22 |
+
"corpus": "dc3bf9215aac8877",
|
| 23 |
+
"val_acc": "0.9879333843034271",
|
| 24 |
+
"turn_rows": "engine-off",
|
| 25 |
+
"train_rows": "2746",
|
| 26 |
+
"rank_levels": "blended",
|
| 27 |
+
"val_loss": "0.07324849329615148",
|
| 28 |
+
"pos_weight": "6.0",
|
| 29 |
+
"tool_weight": "none",
|
| 30 |
+
"seed": "11",
|
| 31 |
+
"state_tokens": "{\"tools\": {\"median\": 86, \"under_16\": 0.0, \"request_kept\": 1.0}}",
|
| 32 |
+
"tool_negatives": "24",
|
| 33 |
+
"backbone": "jhu-clsp/mmBERT-base",
|
| 34 |
+
"model": "convaiinnovations/laya-multilingual",
|
| 35 |
+
"holdout": "[]",
|
| 36 |
+
"tool_truth": "consensus",
|
| 37 |
+
"tool_encoding": "independent"
|
| 38 |
+
}
|
| 39 |
+
},
|
| 40 |
+
{
|
| 41 |
+
"file": "unit-rank.safetensors",
|
| 42 |
+
"sha256": "c62674993eabba94660eccfafbb6e04d1549adc7335656c0ddefa7f068fa58f7",
|
| 43 |
+
"bytes": 59881268,
|
| 44 |
+
"metadata": {
|
| 45 |
+
"val_loss": "0.58553760010303",
|
| 46 |
+
"format": "pw-decide-head/1",
|
| 47 |
+
"label": "open1-unit-rank-e4-dense1",
|
| 48 |
+
"seed": "11",
|
| 49 |
+
"holdout": "[]",
|
| 50 |
+
"turn_rows": "engine-off",
|
| 51 |
+
"pos_weight": "6.0",
|
| 52 |
+
"head_max_len": "512",
|
| 53 |
+
"val_acc": "0.7880391514072964",
|
| 54 |
+
"train_rows": "3792",
|
| 55 |
+
"backbone": "jhu-clsp/mmBERT-base",
|
| 56 |
+
"by_type": "{\"score\": 0.788, \"score_within_one\": 0.921, \"loss\": {\"requests\": 0.5928, \"skills\": 0.5827}}",
|
| 57 |
+
"max_len": "1024",
|
| 58 |
+
"model": "convaiinnovations/laya-multilingual",
|
| 59 |
+
"corpus": "dc3bf9215aac8877",
|
| 60 |
+
"tool_weight": "none",
|
| 61 |
+
"tool_truth": "consensus",
|
| 62 |
+
"rank_levels": "skills-blended",
|
| 63 |
+
"state_tokens": "{}"
|
| 64 |
+
}
|
| 65 |
+
},
|
| 66 |
+
{
|
| 67 |
+
"file": "code-rank.safetensors",
|
| 68 |
+
"sha256": "abe235c347a223e957a0f8b7b979e3a584aebab76ee239e5f62117e839b9c40a",
|
| 69 |
+
"bytes": 59880964,
|
| 70 |
+
"metadata": {
|
| 71 |
+
"label": "open1-code-rank",
|
| 72 |
+
"model": "convaiinnovations/laya-multilingual",
|
| 73 |
+
"val_loss": "0.734294557281827",
|
| 74 |
+
"train_examples": "251360",
|
| 75 |
+
"backbone": "jhu-clsp/mmBERT-base",
|
| 76 |
+
"max_len": "512",
|
| 77 |
+
"format": "pw-decide-head/1",
|
| 78 |
+
"val_acc": "0.684346701164295",
|
| 79 |
+
"val_mae": "0.7976714370407317"
|
| 80 |
+
}
|
| 81 |
+
}
|
| 82 |
+
],
|
| 83 |
+
"release_manifest_sha256": "404f04b03fcf6b757330eda9c0841835248b1f8dc105fb3b61f01dfa9657c872",
|
| 84 |
+
"dataset_version": "open1-b5-e4-bialy",
|
| 85 |
+
"dataset_status": "A companion dataset bundle is prepared separately. This statement does not assert that either bundle has been published.",
|
| 86 |
+
"training_sources": {
|
| 87 |
+
"turn-load": {
|
| 88 |
+
"trainer_commit": "3d79a91c92",
|
| 89 |
+
"factory_commit": "bf0ee33",
|
| 90 |
+
"training_rows_sha256": "2ec8ce650fd19b0e22bbc6b1673be699165fa258e98bbf5f963e6c9354ba80bd",
|
| 91 |
+
"validation_rows_sha256": "67c4c86e3385f0241048f2633d97a4aad78a53a3b7a540592937a20eba669fd3",
|
| 92 |
+
"corpus_sha256": "c77abd8b40965c05cc43a7545e8c2db2405bdcfc18a73bad254632d523b67bcd"
|
| 93 |
+
},
|
| 94 |
+
"unit-rank": {
|
| 95 |
+
"trainer_commit": "946aea8506c9440a662f87a8d5479963f27735d4",
|
| 96 |
+
"training_factory_commit": "b60c15122b4dcddaee75bc45b2e9413a0fe3fa8e",
|
| 97 |
+
"evaluation_factory_commit": "5f2b0b0751030dba9fe45d08f3c2e40f27cab4f9"
|
| 98 |
+
},
|
| 99 |
+
"code-rank": {
|
| 100 |
+
"status": "Selected open1 weights retained; training metadata is embedded in the head. No standalone code-rank benchmark report is asserted by this bundle."
|
| 101 |
+
}
|
| 102 |
+
},
|
| 103 |
+
"source_availability": "The separate companion dataset contains training inputs, sanitized conversation exports, trainer snapshots and provenance. Private originals and additional operational artifacts remain archived separately. Historical commits are not promised to be reachable in a flattened repository.",
|
| 104 |
+
"evidence": [
|
| 105 |
+
"evidence/tool-validation.json",
|
| 106 |
+
"evidence/unit-rank-historical-acceptance.json",
|
| 107 |
+
"evidence/host-policy-context.json"
|
| 108 |
+
],
|
| 109 |
+
"replay_compatibility": "Evidence retains the original scorer semantics. It is not the legacy replay_eval overall schema consumed by bialy audit heads.",
|
| 110 |
+
"assembly": "Copied selected weights without modification. Public reports are aggregate projections of existing evidence; no training, inference, or tests were run during packaging.",
|
| 111 |
+
"public_name": "Bialy",
|
| 112 |
+
"package_version": "open1-b5-e4-bialy",
|
| 113 |
+
"dataset_repo": "paintedwolfcode/bialy-dataset"
|
| 114 |
+
}
|
README.md
CHANGED
|
@@ -1,3 +1,115 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
base_model: convaiinnovations/laya-multilingual
|
| 4 |
+
tags:
|
| 5 |
+
- safetensors
|
| 6 |
+
- coding-agents
|
| 7 |
+
- tool-selection
|
| 8 |
+
- decision-heads
|
| 9 |
---
|
| 10 |
+
|
| 11 |
+
# Bialy
|
| 12 |
+
|
| 13 |
+
Package `open1-b5-e4-bialy` preserves model release `open1-b5-e4` and contains three trained decision heads used by Painted Wolf Code:
|
| 14 |
+
|
| 15 |
+
- `turn-load`: B5 independently scores candidate tools for preloading.
|
| 16 |
+
- `unit-rank`: E4 dense1 scores skills and requested tools for relevance.
|
| 17 |
+
- `code-rank`: open1 scores candidate code units for retrieval.
|
| 18 |
+
|
| 19 |
+
These are head weights, not a standalone language model. They require the
|
| 20 |
+
Laya multilingual backbone and the compatible `bialy` runtime. This repository
|
| 21 |
+
does not bundle the backbone. Training inputs and sanitized session exports are
|
| 22 |
+
prepared separately in the companion dataset `paintedwolfcode/bialy-dataset`.
|
| 23 |
+
Preparation does not assert that either bundle has been published.
|
| 24 |
+
|
| 25 |
+
## Files and identity
|
| 26 |
+
|
| 27 |
+
| File | Label | SHA-256 |
|
| 28 |
+
|---|---|---|
|
| 29 |
+
| `turn-load.safetensors` | `open1-turn-load-B5-release-independent` | `550f30b94d5c6a8680d2cb2a607ebc9ee33e709c7679f72606d1e9fda17f2e90` |
|
| 30 |
+
| `unit-rank.safetensors` | `open1-unit-rank-e4-dense1` | `c62674993eabba94660eccfafbb6e04d1549adc7335656c0ddefa7f068fa58f7` |
|
| 31 |
+
| `code-rank.safetensors` | `open1-code-rank` | `abe235c347a223e957a0f8b7b979e3a584aebab76ee239e5f62117e839b9c40a` |
|
| 32 |
+
|
| 33 |
+
`release.json` pins the weights, backbone revision, and tool option vocabulary.
|
| 34 |
+
`PROVENANCE.json` records the available training identities and evidence scope.
|
| 35 |
+
`SHA256SUMS` covers every public file; the source repository's
|
| 36 |
+
`releases/heads-open1-b5-e4-bialy.json` anchors that checksum list.
|
| 37 |
+
|
| 38 |
+
## Loading the heads
|
| 39 |
+
|
| 40 |
+
Use the Painted Wolf Code runtime and the backbone
|
| 41 |
+
`convaiinnovations/laya-multilingual` at revision
|
| 42 |
+
`e4e9ddf21a7b1903b7acffd8814ad4307bf63a67`. Its encoder is `jhu-clsp/mmBERT-base`.
|
| 43 |
+
|
| 44 |
+
When building Painted Wolf Code, set `BIALY_HEADS_DIR` to the directory containing
|
| 45 |
+
these three `.safetensors` files. The app's committed decision release manifest
|
| 46 |
+
must match their hashes; the staging process verifies them before packaging.
|
| 47 |
+
The B5 head uses independent tool encoding, a 1024-token maximum length, and a
|
| 48 |
+
512-token head budget. A generic Transformers pipeline is not provided.
|
| 49 |
+
|
| 50 |
+
## Tool validation
|
| 51 |
+
|
| 52 |
+
The existing validation set contains 445 complete turns. Consensus labels and
|
| 53 |
+
observed tool calls are separate evaluation targets. B2 is the previous trained
|
| 54 |
+
tool head, not an untuned backbone baseline. B5 selection used this validation
|
| 55 |
+
distribution; these results are not fresh final acceptance or end-to-end task success.
|
| 56 |
+
|
| 57 |
+
This compares two complete configurations: B2 uses joint encoding with six-word
|
| 58 |
+
tool options; B5 uses independent scoring with up to 60-word options. Their corpus
|
| 59 |
+
hashes intentionally differ. Each candidate was bound to its own corpus, prediction
|
| 60 |
+
metadata, head hash and predictions; their validation-row hash is shared.
|
| 61 |
+
|
| 62 |
+
| Head | Cutoff | Consensus precision | Consensus recall | Observed-call precision | Observed-call recall | Loads per turn |
|
| 63 |
+
|---|---|---|---|---|---|---|
|
| 64 |
+
| B2 | 0.89 | 0.757 | 0.295 | 0.517 | 0.149 | 0.339 |
|
| 65 |
+
| B5 | 0.95 | 0.822 | 0.291 | 0.547 | 0.143 | 0.308 |
|
| 66 |
+
| B5 | 0.94 | 0.798 | 0.319 | 0.559 | 0.172 | 0.362 |
|
| 67 |
+
|
| 68 |
+
The 0.94 and 0.95 rows describe two evaluated cutoffs for the same B5 weights.
|
| 69 |
+
The archived host closeout uses tool preload 0.94, at most one automatic skill at
|
| 70 |
+
3.4, and free-text skill lookup 3.2. Those settings do not change the weights.
|
| 71 |
+
Later host tests and startup checks establish implementation correctness within
|
| 72 |
+
their recorded scope, not model task quality or fresh end-to-end acceptance.
|
| 73 |
+
App policy controls cutoffs independently of head files. The evidence file
|
| 74 |
+
retains all archived aggregate threshold rows, input hashes, and report hashes.
|
| 75 |
+
No benchmark was rerun for this publication bundle.
|
| 76 |
+
|
| 77 |
+
## Historical unit-rank evidence
|
| 78 |
+
|
| 79 |
+
E4's archived acceptance report covers 789 requests, of which 660 have a relevant
|
| 80 |
+
skill. Under its historical listing and preload policy, a relevant skill appeared
|
| 81 |
+
in the first six for 92.1% of relevant requests; 122 of 123 automatic preloads were
|
| 82 |
+
relevant. The report contains no observed request-tool needs.
|
| 83 |
+
|
| 84 |
+
The strict skills verdict was **failed**: the common-skill criterion did not pass.
|
| 85 |
+
The other listed skill criteria passed. This evidence does not establish acceptance
|
| 86 |
+
for current free-text lookup or current
|
| 87 |
+
automatic skill preloading. The original policy values and verdict booleans are
|
| 88 |
+
included in `evidence/unit-rank-historical-acceptance.json`.
|
| 89 |
+
|
| 90 |
+
## Limits and evidence access
|
| 91 |
+
|
| 92 |
+
Only aggregate evaluation evidence is published here. Per-example requests,
|
| 93 |
+
training rows, session stores, raw prediction text, machine paths, and operational
|
| 94 |
+
logs are excluded. Original report and input hashes preserve evidence identities;
|
| 95 |
+
they do not make omitted inputs publicly reproducible. Inputs, source snapshots,
|
| 96 |
+
predictions and training artifacts are archived privately. That private archive
|
| 97 |
+
is distinct from what a reader can reproduce from this weights-only public bundle.
|
| 98 |
+
The companion dataset package `open1-b5-e4-bialy` provides training inputs,
|
| 99 |
+
sanitized conversation exports and historical trainer snapshots separately.
|
| 100 |
+
Historical source commits are recorded without claiming they are reachable in
|
| 101 |
+
a flattened public repository.
|
| 102 |
+
No standalone code-rank benchmark or full-current-policy acceptance claim is made.
|
| 103 |
+
|
| 104 |
+
`bialy verify heads` can check this bundle against its committed checksum anchor.
|
| 105 |
+
The included reports use the tool-validation and ranking scorer formats, so the
|
| 106 |
+
legacy `bialy audit heads` replay interface does not support this evidence bundle.
|
| 107 |
+
It should not be presented as a successful replay audit.
|
| 108 |
+
|
| 109 |
+
## Attribution
|
| 110 |
+
|
| 111 |
+
The heads use [Laya](https://github.com/NandhaKishorM/laya), by Convai Innovations,
|
| 112 |
+
and its [multilingual checkpoint](https://huggingface.co/convaiinnovations/laya-multilingual),
|
| 113 |
+
over [mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base).
|
| 114 |
+
Head weights are provided under Apache-2.0. See `LICENSE` and `NOTICE` for
|
| 115 |
+
attribution and the separately licensed backbone dependencies.
|
SHA256SUMS
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
3c2307bbbbb88f79560a165b098c600da8f20fb1ec024586d4be192ebc9c6f8e LICENSE
|
| 2 |
+
b4cdc8ad8e0ec0cc6def7b0c1ad077e5bbd8f473f0ba8bc6a66dd5a46a71abee NOTICE
|
| 3 |
+
ce81c8201ed2afb855c95d99d70ccc107ec8577937ea98ef3506e29eb1d520a6 PROVENANCE.json
|
| 4 |
+
d6001f48ac599ef2361a0b1026bf2780b62c6565dab2eb1f7c24b8c05e3988fd README.md
|
| 5 |
+
abe235c347a223e957a0f8b7b979e3a584aebab76ee239e5f62117e839b9c40a code-rank.safetensors
|
| 6 |
+
0f338ab310d2746541ca08799fcbb76ab25dbdc724a8e41731ea874091ff0549 evidence/host-policy-context.json
|
| 7 |
+
45ea0d69658a114dc73f3984766e529c66a87f6f35fc2deb536d26b84313cd9d evidence/tool-validation.json
|
| 8 |
+
5b97c4f57274870cdcd11a2912baa83c69dc8e40f74df9409f628c076d89a20e evidence/unit-rank-historical-acceptance.json
|
| 9 |
+
404f04b03fcf6b757330eda9c0841835248b1f8dc105fb3b61f01dfa9657c872 release.json
|
| 10 |
+
550f30b94d5c6a8680d2cb2a607ebc9ee33e709c7679f72606d1e9fda17f2e90 turn-load.safetensors
|
| 11 |
+
c62674993eabba94660eccfafbb6e04d1549adc7335656c0ddefa7f068fa58f7 unit-rank.safetensors
|
code-rank.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:abe235c347a223e957a0f8b7b979e3a584aebab76ee239e5f62117e839b9c40a
|
| 3 |
+
size 59880964
|
evidence/host-policy-context.json
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"scope": "Archived host policy closeout; historical implementation and startup checks do not establish current model task quality.",
|
| 3 |
+
"source_report_sha256": "bc2c913cf0db2d60a7808bf0554a32a2fab91a120c8277d447402d6752e9a880",
|
| 4 |
+
"source_deployment_sha256": "65e2d0f6b6e0c3e15c68642837f9d64a962d8189af47d617ffcf5dd46f24ed6b",
|
| 5 |
+
"host_commit": "28d781197b38568d543d459a0702eb38e57348b6",
|
| 6 |
+
"factory_commit": "5b757e846c6108f834fdb14a4c045497b2139df6",
|
| 7 |
+
"policy": {
|
| 8 |
+
"tool_preload_threshold": 0.94,
|
| 9 |
+
"skill_preload_threshold": 3.4,
|
| 10 |
+
"maximum_automatic_skills": 1,
|
| 11 |
+
"skill_lookup_threshold": 3.2,
|
| 12 |
+
"full_skill_catalog_in_prompt": false
|
| 13 |
+
},
|
| 14 |
+
"model_heads_changed": false
|
| 15 |
+
}
|
evidence/tool-validation.json
ADDED
|
@@ -0,0 +1,1141 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"scope": "Existing validation distribution; threshold selection, not fresh final acceptance.",
|
| 3 |
+
"comparison": "Two complete configurations on the same validation rows: B2 joint encoding with six-word tool options; B5 independent encoding with up to 60-word options. Corpus hashes intentionally differ.",
|
| 4 |
+
"transformation": "Per-tool breakdowns omitted; all aggregate threshold rows retained without recalculation.",
|
| 5 |
+
"prediction_metadata_sha256": "de8a6974d1ca4a615f319089b2a6b023e5e57ede9bc03e52de5aa9524bde11c9",
|
| 6 |
+
"reports": {
|
| 7 |
+
"B2": {
|
| 8 |
+
"original_report_sha256": "4b705b7e02a4307e3d23898691c6e82fc72fcfbe9d4e94c0c0341838af21d206",
|
| 9 |
+
"input_sha256": {
|
| 10 |
+
"predictions": "b20f3ddf176f4a5905ac1a4a3073e41a1c4833a930292c942abbba974c10c9b1",
|
| 11 |
+
"rows": "67c4c86e3385f0241048f2633d97a4aad78a53a3b7a540592937a20eba669fd3",
|
| 12 |
+
"corpus": "51e3a65258fc34581bc8cfaae0f7c97f444f0a50f3dda906c77b6554fd5c66d3"
|
| 13 |
+
},
|
| 14 |
+
"prediction_metadata_sha256": "e8d91777999dded4eafe02cf46d7e99a3a89b0bd376c7f6922519e33f7e360dd",
|
| 15 |
+
"encoding": "joint",
|
| 16 |
+
"tool_option_words": 6,
|
| 17 |
+
"head_sha256": "bc432ea7497e9300e39230ee9850de8f42e3f9a3cbbf372259263c49c77011e4",
|
| 18 |
+
"truth": {
|
| 19 |
+
"consensus": [
|
| 20 |
+
{
|
| 21 |
+
"threshold": 0.5,
|
| 22 |
+
"counts": {
|
| 23 |
+
"out_of_catalog_labels": 31,
|
| 24 |
+
"out_of_catalog_positives": 0,
|
| 25 |
+
"turns": 445,
|
| 26 |
+
"loads": 935,
|
| 27 |
+
"empty": 128,
|
| 28 |
+
"needed_turns": 216,
|
| 29 |
+
"any_needed_loaded": 173,
|
| 30 |
+
"all_needed_loaded": 149,
|
| 31 |
+
"tn": 29289,
|
| 32 |
+
"fp": 509,
|
| 33 |
+
"tp": 203,
|
| 34 |
+
"unknown_loaded": 223,
|
| 35 |
+
"fn": 82
|
| 36 |
+
},
|
| 37 |
+
"precision": 0.2851123595505618,
|
| 38 |
+
"recall": 0.712280701754386,
|
| 39 |
+
"f0.5": 0.3239706351739547,
|
| 40 |
+
"loads_per_turn": 2.101123595505618,
|
| 41 |
+
"empty_turn_rate": 0.2876404494382023
|
| 42 |
+
},
|
| 43 |
+
{
|
| 44 |
+
"threshold": 0.6,
|
| 45 |
+
"counts": {
|
| 46 |
+
"out_of_catalog_labels": 31,
|
| 47 |
+
"out_of_catalog_positives": 0,
|
| 48 |
+
"turns": 445,
|
| 49 |
+
"loads": 734,
|
| 50 |
+
"empty": 158,
|
| 51 |
+
"needed_turns": 216,
|
| 52 |
+
"any_needed_loaded": 159,
|
| 53 |
+
"all_needed_loaded": 137,
|
| 54 |
+
"tn": 29438,
|
| 55 |
+
"fp": 360,
|
| 56 |
+
"tp": 186,
|
| 57 |
+
"unknown_loaded": 188,
|
| 58 |
+
"fn": 99
|
| 59 |
+
},
|
| 60 |
+
"precision": 0.34065934065934067,
|
| 61 |
+
"recall": 0.6526315789473685,
|
| 62 |
+
"f0.5": 0.37667071688942894,
|
| 63 |
+
"loads_per_turn": 1.649438202247191,
|
| 64 |
+
"empty_turn_rate": 0.3550561797752809
|
| 65 |
+
},
|
| 66 |
+
{
|
| 67 |
+
"threshold": 0.7,
|
| 68 |
+
"counts": {
|
| 69 |
+
"out_of_catalog_labels": 31,
|
| 70 |
+
"out_of_catalog_positives": 0,
|
| 71 |
+
"turns": 445,
|
| 72 |
+
"loads": 533,
|
| 73 |
+
"empty": 189,
|
| 74 |
+
"needed_turns": 216,
|
| 75 |
+
"any_needed_loaded": 147,
|
| 76 |
+
"all_needed_loaded": 122,
|
| 77 |
+
"tn": 29576,
|
| 78 |
+
"fp": 222,
|
| 79 |
+
"tp": 166,
|
| 80 |
+
"unknown_loaded": 145,
|
| 81 |
+
"fn": 119
|
| 82 |
+
},
|
| 83 |
+
"precision": 0.42783505154639173,
|
| 84 |
+
"recall": 0.5824561403508772,
|
| 85 |
+
"f0.5": 0.4518236254763201,
|
| 86 |
+
"loads_per_turn": 1.1977528089887641,
|
| 87 |
+
"empty_turn_rate": 0.4247191011235955
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"threshold": 0.75,
|
| 91 |
+
"counts": {
|
| 92 |
+
"out_of_catalog_labels": 31,
|
| 93 |
+
"out_of_catalog_positives": 0,
|
| 94 |
+
"turns": 445,
|
| 95 |
+
"loads": 419,
|
| 96 |
+
"empty": 220,
|
| 97 |
+
"needed_turns": 216,
|
| 98 |
+
"any_needed_loaded": 131,
|
| 99 |
+
"all_needed_loaded": 105,
|
| 100 |
+
"tn": 29640,
|
| 101 |
+
"fp": 158,
|
| 102 |
+
"tp": 147,
|
| 103 |
+
"unknown_loaded": 114,
|
| 104 |
+
"fn": 138
|
| 105 |
+
},
|
| 106 |
+
"precision": 0.4819672131147541,
|
| 107 |
+
"recall": 0.5157894736842106,
|
| 108 |
+
"f0.5": 0.4883720930232558,
|
| 109 |
+
"loads_per_turn": 0.9415730337078652,
|
| 110 |
+
"empty_turn_rate": 0.4943820224719101
|
| 111 |
+
},
|
| 112 |
+
{
|
| 113 |
+
"threshold": 0.8,
|
| 114 |
+
"counts": {
|
| 115 |
+
"out_of_catalog_labels": 31,
|
| 116 |
+
"out_of_catalog_positives": 0,
|
| 117 |
+
"turns": 445,
|
| 118 |
+
"loads": 301,
|
| 119 |
+
"empty": 262,
|
| 120 |
+
"needed_turns": 216,
|
| 121 |
+
"any_needed_loaded": 108,
|
| 122 |
+
"all_needed_loaded": 83,
|
| 123 |
+
"tn": 29703,
|
| 124 |
+
"fp": 95,
|
| 125 |
+
"tp": 117,
|
| 126 |
+
"unknown_loaded": 89,
|
| 127 |
+
"fn": 168
|
| 128 |
+
},
|
| 129 |
+
"precision": 0.5518867924528302,
|
| 130 |
+
"recall": 0.4105263157894737,
|
| 131 |
+
"f0.5": 0.5163283318623124,
|
| 132 |
+
"loads_per_turn": 0.6764044943820224,
|
| 133 |
+
"empty_turn_rate": 0.5887640449438202
|
| 134 |
+
},
|
| 135 |
+
{
|
| 136 |
+
"threshold": 0.85,
|
| 137 |
+
"counts": {
|
| 138 |
+
"out_of_catalog_labels": 31,
|
| 139 |
+
"out_of_catalog_positives": 0,
|
| 140 |
+
"turns": 445,
|
| 141 |
+
"loads": 211,
|
| 142 |
+
"empty": 299,
|
| 143 |
+
"needed_turns": 216,
|
| 144 |
+
"any_needed_loaded": 93,
|
| 145 |
+
"all_needed_loaded": 72,
|
| 146 |
+
"tn": 29745,
|
| 147 |
+
"fp": 53,
|
| 148 |
+
"tp": 100,
|
| 149 |
+
"unknown_loaded": 58,
|
| 150 |
+
"fn": 185
|
| 151 |
+
},
|
| 152 |
+
"precision": 0.6535947712418301,
|
| 153 |
+
"recall": 0.3508771929824561,
|
| 154 |
+
"f0.5": 0.5574136008918618,
|
| 155 |
+
"loads_per_turn": 0.47415730337078654,
|
| 156 |
+
"empty_turn_rate": 0.6719101123595506
|
| 157 |
+
},
|
| 158 |
+
{
|
| 159 |
+
"threshold": 0.89,
|
| 160 |
+
"counts": {
|
| 161 |
+
"out_of_catalog_labels": 31,
|
| 162 |
+
"out_of_catalog_positives": 0,
|
| 163 |
+
"turns": 445,
|
| 164 |
+
"loads": 151,
|
| 165 |
+
"empty": 326,
|
| 166 |
+
"needed_turns": 216,
|
| 167 |
+
"any_needed_loaded": 80,
|
| 168 |
+
"all_needed_loaded": 60,
|
| 169 |
+
"tn": 29771,
|
| 170 |
+
"fp": 27,
|
| 171 |
+
"fn": 201,
|
| 172 |
+
"unknown_loaded": 40,
|
| 173 |
+
"tp": 84
|
| 174 |
+
},
|
| 175 |
+
"precision": 0.7567567567567568,
|
| 176 |
+
"recall": 0.29473684210526313,
|
| 177 |
+
"f0.5": 0.5761316872427984,
|
| 178 |
+
"loads_per_turn": 0.3393258426966292,
|
| 179 |
+
"empty_turn_rate": 0.7325842696629213
|
| 180 |
+
},
|
| 181 |
+
{
|
| 182 |
+
"threshold": 0.9,
|
| 183 |
+
"counts": {
|
| 184 |
+
"out_of_catalog_labels": 31,
|
| 185 |
+
"out_of_catalog_positives": 0,
|
| 186 |
+
"turns": 445,
|
| 187 |
+
"loads": 138,
|
| 188 |
+
"empty": 336,
|
| 189 |
+
"needed_turns": 216,
|
| 190 |
+
"any_needed_loaded": 74,
|
| 191 |
+
"all_needed_loaded": 56,
|
| 192 |
+
"tn": 29773,
|
| 193 |
+
"fp": 25,
|
| 194 |
+
"fn": 207,
|
| 195 |
+
"unknown_loaded": 35,
|
| 196 |
+
"tp": 78
|
| 197 |
+
},
|
| 198 |
+
"precision": 0.7572815533980582,
|
| 199 |
+
"recall": 0.2736842105263158,
|
| 200 |
+
"f0.5": 0.5595408895265424,
|
| 201 |
+
"loads_per_turn": 0.3101123595505618,
|
| 202 |
+
"empty_turn_rate": 0.755056179775281
|
| 203 |
+
},
|
| 204 |
+
{
|
| 205 |
+
"threshold": 0.95,
|
| 206 |
+
"counts": {
|
| 207 |
+
"out_of_catalog_labels": 31,
|
| 208 |
+
"out_of_catalog_positives": 0,
|
| 209 |
+
"turns": 445,
|
| 210 |
+
"loads": 84,
|
| 211 |
+
"empty": 373,
|
| 212 |
+
"needed_turns": 216,
|
| 213 |
+
"any_needed_loaded": 51,
|
| 214 |
+
"all_needed_loaded": 43,
|
| 215 |
+
"tn": 29789,
|
| 216 |
+
"fn": 230,
|
| 217 |
+
"unknown_loaded": 20,
|
| 218 |
+
"tp": 55,
|
| 219 |
+
"fp": 9
|
| 220 |
+
},
|
| 221 |
+
"precision": 0.859375,
|
| 222 |
+
"recall": 0.19298245614035087,
|
| 223 |
+
"f0.5": 0.5083179297597042,
|
| 224 |
+
"loads_per_turn": 0.18876404494382024,
|
| 225 |
+
"empty_turn_rate": 0.8382022471910112
|
| 226 |
+
},
|
| 227 |
+
{
|
| 228 |
+
"threshold": 0.97,
|
| 229 |
+
"counts": {
|
| 230 |
+
"out_of_catalog_labels": 31,
|
| 231 |
+
"out_of_catalog_positives": 0,
|
| 232 |
+
"turns": 445,
|
| 233 |
+
"loads": 62,
|
| 234 |
+
"empty": 391,
|
| 235 |
+
"needed_turns": 216,
|
| 236 |
+
"any_needed_loaded": 41,
|
| 237 |
+
"all_needed_loaded": 34,
|
| 238 |
+
"tn": 29793,
|
| 239 |
+
"fn": 241,
|
| 240 |
+
"unknown_loaded": 13,
|
| 241 |
+
"tp": 44,
|
| 242 |
+
"fp": 5
|
| 243 |
+
},
|
| 244 |
+
"precision": 0.8979591836734694,
|
| 245 |
+
"recall": 0.1543859649122807,
|
| 246 |
+
"f0.5": 0.4573804573804574,
|
| 247 |
+
"loads_per_turn": 0.1393258426966292,
|
| 248 |
+
"empty_turn_rate": 0.8786516853932584
|
| 249 |
+
},
|
| 250 |
+
{
|
| 251 |
+
"threshold": 0.99,
|
| 252 |
+
"counts": {
|
| 253 |
+
"out_of_catalog_labels": 31,
|
| 254 |
+
"out_of_catalog_positives": 0,
|
| 255 |
+
"turns": 445,
|
| 256 |
+
"loads": 29,
|
| 257 |
+
"empty": 417,
|
| 258 |
+
"needed_turns": 216,
|
| 259 |
+
"any_needed_loaded": 21,
|
| 260 |
+
"all_needed_loaded": 15,
|
| 261 |
+
"tn": 29795,
|
| 262 |
+
"fn": 264,
|
| 263 |
+
"unknown_loaded": 5,
|
| 264 |
+
"tp": 21,
|
| 265 |
+
"fp": 3
|
| 266 |
+
},
|
| 267 |
+
"precision": 0.875,
|
| 268 |
+
"recall": 0.07368421052631578,
|
| 269 |
+
"f0.5": 0.2755905511811024,
|
| 270 |
+
"loads_per_turn": 0.0651685393258427,
|
| 271 |
+
"empty_turn_rate": 0.9370786516853933
|
| 272 |
+
}
|
| 273 |
+
],
|
| 274 |
+
"called": [
|
| 275 |
+
{
|
| 276 |
+
"threshold": 0.5,
|
| 277 |
+
"counts": {
|
| 278 |
+
"out_of_catalog_labels": 31,
|
| 279 |
+
"out_of_catalog_positives": 0,
|
| 280 |
+
"turns": 445,
|
| 281 |
+
"loads": 935,
|
| 282 |
+
"empty": 128,
|
| 283 |
+
"needed_turns": 228,
|
| 284 |
+
"any_needed_loaded": 182,
|
| 285 |
+
"all_needed_loaded": 91,
|
| 286 |
+
"tn": 29888,
|
| 287 |
+
"fp": 659,
|
| 288 |
+
"tp": 276,
|
| 289 |
+
"fn": 248
|
| 290 |
+
},
|
| 291 |
+
"precision": 0.29518716577540105,
|
| 292 |
+
"recall": 0.5267175572519084,
|
| 293 |
+
"f0.5": 0.3236397748592871,
|
| 294 |
+
"loads_per_turn": 2.101123595505618,
|
| 295 |
+
"empty_turn_rate": 0.2876404494382023
|
| 296 |
+
},
|
| 297 |
+
{
|
| 298 |
+
"threshold": 0.6,
|
| 299 |
+
"counts": {
|
| 300 |
+
"out_of_catalog_labels": 31,
|
| 301 |
+
"out_of_catalog_positives": 0,
|
| 302 |
+
"turns": 445,
|
| 303 |
+
"loads": 734,
|
| 304 |
+
"empty": 158,
|
| 305 |
+
"needed_turns": 228,
|
| 306 |
+
"any_needed_loaded": 166,
|
| 307 |
+
"all_needed_loaded": 76,
|
| 308 |
+
"tn": 30053,
|
| 309 |
+
"fp": 494,
|
| 310 |
+
"tp": 240,
|
| 311 |
+
"fn": 284
|
| 312 |
+
},
|
| 313 |
+
"precision": 0.32697547683923706,
|
| 314 |
+
"recall": 0.4580152671755725,
|
| 315 |
+
"f0.5": 0.3468208092485549,
|
| 316 |
+
"loads_per_turn": 1.649438202247191,
|
| 317 |
+
"empty_turn_rate": 0.3550561797752809
|
| 318 |
+
},
|
| 319 |
+
{
|
| 320 |
+
"threshold": 0.7,
|
| 321 |
+
"counts": {
|
| 322 |
+
"out_of_catalog_labels": 31,
|
| 323 |
+
"out_of_catalog_positives": 0,
|
| 324 |
+
"turns": 445,
|
| 325 |
+
"loads": 533,
|
| 326 |
+
"empty": 189,
|
| 327 |
+
"needed_turns": 228,
|
| 328 |
+
"any_needed_loaded": 149,
|
| 329 |
+
"all_needed_loaded": 59,
|
| 330 |
+
"tn": 30208,
|
| 331 |
+
"tp": 194,
|
| 332 |
+
"fp": 339,
|
| 333 |
+
"fn": 330
|
| 334 |
+
},
|
| 335 |
+
"precision": 0.36397748592870544,
|
| 336 |
+
"recall": 0.3702290076335878,
|
| 337 |
+
"f0.5": 0.36521084337349397,
|
| 338 |
+
"loads_per_turn": 1.1977528089887641,
|
| 339 |
+
"empty_turn_rate": 0.4247191011235955
|
| 340 |
+
},
|
| 341 |
+
{
|
| 342 |
+
"threshold": 0.75,
|
| 343 |
+
"counts": {
|
| 344 |
+
"out_of_catalog_labels": 31,
|
| 345 |
+
"out_of_catalog_positives": 0,
|
| 346 |
+
"turns": 445,
|
| 347 |
+
"loads": 419,
|
| 348 |
+
"empty": 220,
|
| 349 |
+
"needed_turns": 228,
|
| 350 |
+
"any_needed_loaded": 133,
|
| 351 |
+
"all_needed_loaded": 46,
|
| 352 |
+
"tn": 30286,
|
| 353 |
+
"fn": 366,
|
| 354 |
+
"fp": 261,
|
| 355 |
+
"tp": 158
|
| 356 |
+
},
|
| 357 |
+
"precision": 0.37708830548926014,
|
| 358 |
+
"recall": 0.3015267175572519,
|
| 359 |
+
"f0.5": 0.35909090909090907,
|
| 360 |
+
"loads_per_turn": 0.9415730337078652,
|
| 361 |
+
"empty_turn_rate": 0.4943820224719101
|
| 362 |
+
},
|
| 363 |
+
{
|
| 364 |
+
"threshold": 0.8,
|
| 365 |
+
"counts": {
|
| 366 |
+
"out_of_catalog_labels": 31,
|
| 367 |
+
"out_of_catalog_positives": 0,
|
| 368 |
+
"turns": 445,
|
| 369 |
+
"loads": 301,
|
| 370 |
+
"empty": 262,
|
| 371 |
+
"needed_turns": 228,
|
| 372 |
+
"any_needed_loaded": 106,
|
| 373 |
+
"all_needed_loaded": 38,
|
| 374 |
+
"tn": 30368,
|
| 375 |
+
"fn": 402,
|
| 376 |
+
"fp": 179,
|
| 377 |
+
"tp": 122
|
| 378 |
+
},
|
| 379 |
+
"precision": 0.4053156146179402,
|
| 380 |
+
"recall": 0.23282442748091603,
|
| 381 |
+
"f0.5": 0.35300925925925924,
|
| 382 |
+
"loads_per_turn": 0.6764044943820224,
|
| 383 |
+
"empty_turn_rate": 0.5887640449438202
|
| 384 |
+
},
|
| 385 |
+
{
|
| 386 |
+
"threshold": 0.85,
|
| 387 |
+
"counts": {
|
| 388 |
+
"out_of_catalog_labels": 31,
|
| 389 |
+
"out_of_catalog_positives": 0,
|
| 390 |
+
"turns": 445,
|
| 391 |
+
"loads": 211,
|
| 392 |
+
"empty": 299,
|
| 393 |
+
"needed_turns": 228,
|
| 394 |
+
"any_needed_loaded": 83,
|
| 395 |
+
"all_needed_loaded": 32,
|
| 396 |
+
"tn": 30430,
|
| 397 |
+
"fn": 430,
|
| 398 |
+
"fp": 117,
|
| 399 |
+
"tp": 94
|
| 400 |
+
},
|
| 401 |
+
"precision": 0.44549763033175355,
|
| 402 |
+
"recall": 0.17938931297709923,
|
| 403 |
+
"f0.5": 0.3435672514619883,
|
| 404 |
+
"loads_per_turn": 0.47415730337078654,
|
| 405 |
+
"empty_turn_rate": 0.6719101123595506
|
| 406 |
+
},
|
| 407 |
+
{
|
| 408 |
+
"threshold": 0.89,
|
| 409 |
+
"counts": {
|
| 410 |
+
"out_of_catalog_labels": 31,
|
| 411 |
+
"out_of_catalog_positives": 0,
|
| 412 |
+
"turns": 445,
|
| 413 |
+
"loads": 151,
|
| 414 |
+
"empty": 326,
|
| 415 |
+
"needed_turns": 228,
|
| 416 |
+
"any_needed_loaded": 72,
|
| 417 |
+
"all_needed_loaded": 24,
|
| 418 |
+
"tn": 30474,
|
| 419 |
+
"fn": 446,
|
| 420 |
+
"fp": 73,
|
| 421 |
+
"tp": 78
|
| 422 |
+
},
|
| 423 |
+
"precision": 0.5165562913907285,
|
| 424 |
+
"recall": 0.14885496183206107,
|
| 425 |
+
"f0.5": 0.34574468085106386,
|
| 426 |
+
"loads_per_turn": 0.3393258426966292,
|
| 427 |
+
"empty_turn_rate": 0.7325842696629213
|
| 428 |
+
},
|
| 429 |
+
{
|
| 430 |
+
"threshold": 0.9,
|
| 431 |
+
"counts": {
|
| 432 |
+
"out_of_catalog_labels": 31,
|
| 433 |
+
"out_of_catalog_positives": 0,
|
| 434 |
+
"turns": 445,
|
| 435 |
+
"loads": 138,
|
| 436 |
+
"empty": 336,
|
| 437 |
+
"needed_turns": 228,
|
| 438 |
+
"any_needed_loaded": 65,
|
| 439 |
+
"all_needed_loaded": 22,
|
| 440 |
+
"tn": 30479,
|
| 441 |
+
"fn": 454,
|
| 442 |
+
"fp": 68,
|
| 443 |
+
"tp": 70
|
| 444 |
+
},
|
| 445 |
+
"precision": 0.5072463768115942,
|
| 446 |
+
"recall": 0.13358778625954199,
|
| 447 |
+
"f0.5": 0.3252788104089219,
|
| 448 |
+
"loads_per_turn": 0.3101123595505618,
|
| 449 |
+
"empty_turn_rate": 0.755056179775281
|
| 450 |
+
},
|
| 451 |
+
{
|
| 452 |
+
"threshold": 0.95,
|
| 453 |
+
"counts": {
|
| 454 |
+
"out_of_catalog_labels": 31,
|
| 455 |
+
"out_of_catalog_positives": 0,
|
| 456 |
+
"turns": 445,
|
| 457 |
+
"loads": 84,
|
| 458 |
+
"empty": 373,
|
| 459 |
+
"needed_turns": 228,
|
| 460 |
+
"any_needed_loaded": 42,
|
| 461 |
+
"all_needed_loaded": 15,
|
| 462 |
+
"tn": 30508,
|
| 463 |
+
"fn": 479,
|
| 464 |
+
"tp": 45,
|
| 465 |
+
"fp": 39
|
| 466 |
+
},
|
| 467 |
+
"precision": 0.5357142857142857,
|
| 468 |
+
"recall": 0.08587786259541985,
|
| 469 |
+
"f0.5": 0.2616279069767442,
|
| 470 |
+
"loads_per_turn": 0.18876404494382024,
|
| 471 |
+
"empty_turn_rate": 0.8382022471910112
|
| 472 |
+
},
|
| 473 |
+
{
|
| 474 |
+
"threshold": 0.97,
|
| 475 |
+
"counts": {
|
| 476 |
+
"out_of_catalog_labels": 31,
|
| 477 |
+
"out_of_catalog_positives": 0,
|
| 478 |
+
"turns": 445,
|
| 479 |
+
"loads": 62,
|
| 480 |
+
"empty": 391,
|
| 481 |
+
"needed_turns": 228,
|
| 482 |
+
"any_needed_loaded": 32,
|
| 483 |
+
"all_needed_loaded": 9,
|
| 484 |
+
"tn": 30520,
|
| 485 |
+
"fn": 489,
|
| 486 |
+
"tp": 35,
|
| 487 |
+
"fp": 27
|
| 488 |
+
},
|
| 489 |
+
"precision": 0.5645161290322581,
|
| 490 |
+
"recall": 0.06679389312977099,
|
| 491 |
+
"f0.5": 0.2266839378238342,
|
| 492 |
+
"loads_per_turn": 0.1393258426966292,
|
| 493 |
+
"empty_turn_rate": 0.8786516853932584
|
| 494 |
+
},
|
| 495 |
+
{
|
| 496 |
+
"threshold": 0.99,
|
| 497 |
+
"counts": {
|
| 498 |
+
"out_of_catalog_labels": 31,
|
| 499 |
+
"out_of_catalog_positives": 0,
|
| 500 |
+
"turns": 445,
|
| 501 |
+
"loads": 29,
|
| 502 |
+
"empty": 417,
|
| 503 |
+
"needed_turns": 228,
|
| 504 |
+
"any_needed_loaded": 15,
|
| 505 |
+
"all_needed_loaded": 5,
|
| 506 |
+
"tn": 30533,
|
| 507 |
+
"fn": 509,
|
| 508 |
+
"tp": 15,
|
| 509 |
+
"fp": 14
|
| 510 |
+
},
|
| 511 |
+
"precision": 0.5172413793103449,
|
| 512 |
+
"recall": 0.02862595419847328,
|
| 513 |
+
"f0.5": 0.1171875,
|
| 514 |
+
"loads_per_turn": 0.0651685393258427,
|
| 515 |
+
"empty_turn_rate": 0.9370786516853933
|
| 516 |
+
}
|
| 517 |
+
]
|
| 518 |
+
}
|
| 519 |
+
},
|
| 520 |
+
"B5": {
|
| 521 |
+
"original_report_sha256": "18a25333e07dfb5b79c6709cf7c99855517ce07e51e3cd80ccc8e15e8a656a51",
|
| 522 |
+
"input_sha256": {
|
| 523 |
+
"predictions": "e70357d58524a0714882aef5c5de98a6c4cf74a3ad82ae23ee1ed9f97789d1c0",
|
| 524 |
+
"rows": "67c4c86e3385f0241048f2633d97a4aad78a53a3b7a540592937a20eba669fd3",
|
| 525 |
+
"corpus": "c77abd8b40965c05cc43a7545e8c2db2405bdcfc18a73bad254632d523b67bcd"
|
| 526 |
+
},
|
| 527 |
+
"prediction_metadata_sha256": "de8a6974d1ca4a615f319089b2a6b023e5e57ede9bc03e52de5aa9524bde11c9",
|
| 528 |
+
"encoding": "independent",
|
| 529 |
+
"tool_option_words": 60,
|
| 530 |
+
"head_sha256": "550f30b94d5c6a8680d2cb2a607ebc9ee33e709c7679f72606d1e9fda17f2e90",
|
| 531 |
+
"truth": {
|
| 532 |
+
"consensus": [
|
| 533 |
+
{
|
| 534 |
+
"threshold": 0.5,
|
| 535 |
+
"counts": {
|
| 536 |
+
"out_of_catalog_labels": 31,
|
| 537 |
+
"out_of_catalog_positives": 0,
|
| 538 |
+
"turns": 445,
|
| 539 |
+
"loads": 639,
|
| 540 |
+
"empty": 176,
|
| 541 |
+
"needed_turns": 216,
|
| 542 |
+
"any_needed_loaded": 166,
|
| 543 |
+
"all_needed_loaded": 141,
|
| 544 |
+
"tn": 29523,
|
| 545 |
+
"fp": 275,
|
| 546 |
+
"tp": 197,
|
| 547 |
+
"unknown_loaded": 167,
|
| 548 |
+
"fn": 88
|
| 549 |
+
},
|
| 550 |
+
"precision": 0.4173728813559322,
|
| 551 |
+
"recall": 0.6912280701754386,
|
| 552 |
+
"f0.5": 0.45329038196042337,
|
| 553 |
+
"loads_per_turn": 1.4359550561797754,
|
| 554 |
+
"empty_turn_rate": 0.3955056179775281
|
| 555 |
+
},
|
| 556 |
+
{
|
| 557 |
+
"threshold": 0.6,
|
| 558 |
+
"counts": {
|
| 559 |
+
"out_of_catalog_labels": 31,
|
| 560 |
+
"out_of_catalog_positives": 0,
|
| 561 |
+
"turns": 445,
|
| 562 |
+
"loads": 539,
|
| 563 |
+
"empty": 191,
|
| 564 |
+
"needed_turns": 216,
|
| 565 |
+
"any_needed_loaded": 164,
|
| 566 |
+
"all_needed_loaded": 138,
|
| 567 |
+
"tn": 29594,
|
| 568 |
+
"fp": 204,
|
| 569 |
+
"tp": 192,
|
| 570 |
+
"unknown_loaded": 143,
|
| 571 |
+
"fn": 93
|
| 572 |
+
},
|
| 573 |
+
"precision": 0.48484848484848486,
|
| 574 |
+
"recall": 0.6736842105263158,
|
| 575 |
+
"f0.5": 0.5136436597110754,
|
| 576 |
+
"loads_per_turn": 1.2112359550561798,
|
| 577 |
+
"empty_turn_rate": 0.42921348314606744
|
| 578 |
+
},
|
| 579 |
+
{
|
| 580 |
+
"threshold": 0.7,
|
| 581 |
+
"counts": {
|
| 582 |
+
"out_of_catalog_labels": 31,
|
| 583 |
+
"out_of_catalog_positives": 0,
|
| 584 |
+
"turns": 445,
|
| 585 |
+
"loads": 429,
|
| 586 |
+
"empty": 222,
|
| 587 |
+
"needed_turns": 216,
|
| 588 |
+
"any_needed_loaded": 145,
|
| 589 |
+
"all_needed_loaded": 118,
|
| 590 |
+
"tn": 29653,
|
| 591 |
+
"fp": 145,
|
| 592 |
+
"tp": 167,
|
| 593 |
+
"unknown_loaded": 117,
|
| 594 |
+
"fn": 118
|
| 595 |
+
},
|
| 596 |
+
"precision": 0.5352564102564102,
|
| 597 |
+
"recall": 0.5859649122807018,
|
| 598 |
+
"f0.5": 0.5446836268754077,
|
| 599 |
+
"loads_per_turn": 0.9640449438202248,
|
| 600 |
+
"empty_turn_rate": 0.49887640449438203
|
| 601 |
+
},
|
| 602 |
+
{
|
| 603 |
+
"threshold": 0.75,
|
| 604 |
+
"counts": {
|
| 605 |
+
"out_of_catalog_labels": 31,
|
| 606 |
+
"out_of_catalog_positives": 0,
|
| 607 |
+
"turns": 445,
|
| 608 |
+
"loads": 371,
|
| 609 |
+
"empty": 238,
|
| 610 |
+
"needed_turns": 216,
|
| 611 |
+
"any_needed_loaded": 136,
|
| 612 |
+
"all_needed_loaded": 108,
|
| 613 |
+
"tn": 29690,
|
| 614 |
+
"fp": 108,
|
| 615 |
+
"tp": 156,
|
| 616 |
+
"unknown_loaded": 107,
|
| 617 |
+
"fn": 129
|
| 618 |
+
},
|
| 619 |
+
"precision": 0.5909090909090909,
|
| 620 |
+
"recall": 0.5473684210526316,
|
| 621 |
+
"f0.5": 0.5816554809843401,
|
| 622 |
+
"loads_per_turn": 0.8337078651685393,
|
| 623 |
+
"empty_turn_rate": 0.5348314606741573
|
| 624 |
+
},
|
| 625 |
+
{
|
| 626 |
+
"threshold": 0.8,
|
| 627 |
+
"counts": {
|
| 628 |
+
"out_of_catalog_labels": 31,
|
| 629 |
+
"out_of_catalog_positives": 0,
|
| 630 |
+
"turns": 445,
|
| 631 |
+
"loads": 307,
|
| 632 |
+
"empty": 261,
|
| 633 |
+
"needed_turns": 216,
|
| 634 |
+
"any_needed_loaded": 126,
|
| 635 |
+
"all_needed_loaded": 100,
|
| 636 |
+
"tn": 29720,
|
| 637 |
+
"fp": 78,
|
| 638 |
+
"tp": 142,
|
| 639 |
+
"unknown_loaded": 87,
|
| 640 |
+
"fn": 143
|
| 641 |
+
},
|
| 642 |
+
"precision": 0.6454545454545455,
|
| 643 |
+
"recall": 0.4982456140350877,
|
| 644 |
+
"f0.5": 0.6094420600858369,
|
| 645 |
+
"loads_per_turn": 0.6898876404494382,
|
| 646 |
+
"empty_turn_rate": 0.5865168539325842
|
| 647 |
+
},
|
| 648 |
+
{
|
| 649 |
+
"threshold": 0.85,
|
| 650 |
+
"counts": {
|
| 651 |
+
"out_of_catalog_labels": 31,
|
| 652 |
+
"out_of_catalog_positives": 0,
|
| 653 |
+
"turns": 445,
|
| 654 |
+
"loads": 267,
|
| 655 |
+
"empty": 278,
|
| 656 |
+
"needed_turns": 216,
|
| 657 |
+
"any_needed_loaded": 118,
|
| 658 |
+
"all_needed_loaded": 91,
|
| 659 |
+
"tn": 29734,
|
| 660 |
+
"fp": 64,
|
| 661 |
+
"tp": 131,
|
| 662 |
+
"unknown_loaded": 72,
|
| 663 |
+
"fn": 154
|
| 664 |
+
},
|
| 665 |
+
"precision": 0.6717948717948717,
|
| 666 |
+
"recall": 0.45964912280701753,
|
| 667 |
+
"f0.5": 0.6150234741784038,
|
| 668 |
+
"loads_per_turn": 0.6,
|
| 669 |
+
"empty_turn_rate": 0.6247191011235955
|
| 670 |
+
},
|
| 671 |
+
{
|
| 672 |
+
"threshold": 0.89,
|
| 673 |
+
"counts": {
|
| 674 |
+
"out_of_catalog_labels": 31,
|
| 675 |
+
"out_of_catalog_positives": 0,
|
| 676 |
+
"turns": 445,
|
| 677 |
+
"loads": 222,
|
| 678 |
+
"empty": 295,
|
| 679 |
+
"needed_turns": 216,
|
| 680 |
+
"any_needed_loaded": 104,
|
| 681 |
+
"all_needed_loaded": 78,
|
| 682 |
+
"tn": 29754,
|
| 683 |
+
"fp": 44,
|
| 684 |
+
"tp": 115,
|
| 685 |
+
"unknown_loaded": 63,
|
| 686 |
+
"fn": 170
|
| 687 |
+
},
|
| 688 |
+
"precision": 0.7232704402515723,
|
| 689 |
+
"recall": 0.40350877192982454,
|
| 690 |
+
"f0.5": 0.6243213897937026,
|
| 691 |
+
"loads_per_turn": 0.49887640449438203,
|
| 692 |
+
"empty_turn_rate": 0.6629213483146067
|
| 693 |
+
},
|
| 694 |
+
{
|
| 695 |
+
"threshold": 0.9,
|
| 696 |
+
"counts": {
|
| 697 |
+
"out_of_catalog_labels": 31,
|
| 698 |
+
"out_of_catalog_positives": 0,
|
| 699 |
+
"turns": 445,
|
| 700 |
+
"loads": 216,
|
| 701 |
+
"empty": 297,
|
| 702 |
+
"needed_turns": 216,
|
| 703 |
+
"any_needed_loaded": 101,
|
| 704 |
+
"all_needed_loaded": 75,
|
| 705 |
+
"tn": 29757,
|
| 706 |
+
"fp": 41,
|
| 707 |
+
"tp": 112,
|
| 708 |
+
"unknown_loaded": 63,
|
| 709 |
+
"fn": 173
|
| 710 |
+
},
|
| 711 |
+
"precision": 0.7320261437908496,
|
| 712 |
+
"recall": 0.3929824561403509,
|
| 713 |
+
"f0.5": 0.6243032329988851,
|
| 714 |
+
"loads_per_turn": 0.4853932584269663,
|
| 715 |
+
"empty_turn_rate": 0.6674157303370787
|
| 716 |
+
},
|
| 717 |
+
{
|
| 718 |
+
"threshold": 0.95,
|
| 719 |
+
"counts": {
|
| 720 |
+
"out_of_catalog_labels": 31,
|
| 721 |
+
"out_of_catalog_positives": 0,
|
| 722 |
+
"turns": 445,
|
| 723 |
+
"loads": 137,
|
| 724 |
+
"empty": 338,
|
| 725 |
+
"needed_turns": 216,
|
| 726 |
+
"any_needed_loaded": 77,
|
| 727 |
+
"all_needed_loaded": 53,
|
| 728 |
+
"tn": 29780,
|
| 729 |
+
"fp": 18,
|
| 730 |
+
"fn": 202,
|
| 731 |
+
"unknown_loaded": 36,
|
| 732 |
+
"tp": 83
|
| 733 |
+
},
|
| 734 |
+
"precision": 0.8217821782178217,
|
| 735 |
+
"recall": 0.2912280701754386,
|
| 736 |
+
"f0.5": 0.602322206095791,
|
| 737 |
+
"loads_per_turn": 0.30786516853932583,
|
| 738 |
+
"empty_turn_rate": 0.7595505617977528
|
| 739 |
+
},
|
| 740 |
+
{
|
| 741 |
+
"threshold": 0.97,
|
| 742 |
+
"counts": {
|
| 743 |
+
"out_of_catalog_labels": 31,
|
| 744 |
+
"out_of_catalog_positives": 0,
|
| 745 |
+
"turns": 445,
|
| 746 |
+
"loads": 85,
|
| 747 |
+
"empty": 374,
|
| 748 |
+
"needed_turns": 216,
|
| 749 |
+
"any_needed_loaded": 51,
|
| 750 |
+
"all_needed_loaded": 37,
|
| 751 |
+
"tn": 29788,
|
| 752 |
+
"fp": 10,
|
| 753 |
+
"fn": 231,
|
| 754 |
+
"unknown_loaded": 21,
|
| 755 |
+
"tp": 54
|
| 756 |
+
},
|
| 757 |
+
"precision": 0.84375,
|
| 758 |
+
"recall": 0.18947368421052632,
|
| 759 |
+
"f0.5": 0.49907578558225507,
|
| 760 |
+
"loads_per_turn": 0.19101123595505617,
|
| 761 |
+
"empty_turn_rate": 0.8404494382022472
|
| 762 |
+
},
|
| 763 |
+
{
|
| 764 |
+
"threshold": 0.99,
|
| 765 |
+
"counts": {
|
| 766 |
+
"out_of_catalog_labels": 31,
|
| 767 |
+
"out_of_catalog_positives": 0,
|
| 768 |
+
"turns": 445,
|
| 769 |
+
"loads": 12,
|
| 770 |
+
"empty": 433,
|
| 771 |
+
"needed_turns": 216,
|
| 772 |
+
"any_needed_loaded": 9,
|
| 773 |
+
"all_needed_loaded": 5,
|
| 774 |
+
"tn": 29798,
|
| 775 |
+
"fn": 276,
|
| 776 |
+
"unknown_loaded": 3,
|
| 777 |
+
"tp": 9
|
| 778 |
+
},
|
| 779 |
+
"precision": 1.0,
|
| 780 |
+
"recall": 0.031578947368421054,
|
| 781 |
+
"f0.5": 0.14018691588785046,
|
| 782 |
+
"loads_per_turn": 0.02696629213483146,
|
| 783 |
+
"empty_turn_rate": 0.9730337078651685
|
| 784 |
+
}
|
| 785 |
+
],
|
| 786 |
+
"called": [
|
| 787 |
+
{
|
| 788 |
+
"threshold": 0.5,
|
| 789 |
+
"counts": {
|
| 790 |
+
"out_of_catalog_labels": 31,
|
| 791 |
+
"out_of_catalog_positives": 0,
|
| 792 |
+
"turns": 445,
|
| 793 |
+
"loads": 639,
|
| 794 |
+
"empty": 176,
|
| 795 |
+
"needed_turns": 228,
|
| 796 |
+
"any_needed_loaded": 170,
|
| 797 |
+
"all_needed_loaded": 74,
|
| 798 |
+
"tn": 30152,
|
| 799 |
+
"fn": 280,
|
| 800 |
+
"fp": 395,
|
| 801 |
+
"tp": 244
|
| 802 |
+
},
|
| 803 |
+
"precision": 0.38184663536776214,
|
| 804 |
+
"recall": 0.46564885496183206,
|
| 805 |
+
"f0.5": 0.3961038961038961,
|
| 806 |
+
"loads_per_turn": 1.4359550561797754,
|
| 807 |
+
"empty_turn_rate": 0.3955056179775281
|
| 808 |
+
},
|
| 809 |
+
{
|
| 810 |
+
"threshold": 0.6,
|
| 811 |
+
"counts": {
|
| 812 |
+
"out_of_catalog_labels": 31,
|
| 813 |
+
"out_of_catalog_positives": 0,
|
| 814 |
+
"turns": 445,
|
| 815 |
+
"loads": 539,
|
| 816 |
+
"empty": 191,
|
| 817 |
+
"needed_turns": 228,
|
| 818 |
+
"any_needed_loaded": 165,
|
| 819 |
+
"all_needed_loaded": 68,
|
| 820 |
+
"tn": 30235,
|
| 821 |
+
"fn": 297,
|
| 822 |
+
"fp": 312,
|
| 823 |
+
"tp": 227
|
| 824 |
+
},
|
| 825 |
+
"precision": 0.42115027829313545,
|
| 826 |
+
"recall": 0.43320610687022904,
|
| 827 |
+
"f0.5": 0.42350746268656714,
|
| 828 |
+
"loads_per_turn": 1.2112359550561798,
|
| 829 |
+
"empty_turn_rate": 0.42921348314606744
|
| 830 |
+
},
|
| 831 |
+
{
|
| 832 |
+
"threshold": 0.7,
|
| 833 |
+
"counts": {
|
| 834 |
+
"out_of_catalog_labels": 31,
|
| 835 |
+
"out_of_catalog_positives": 0,
|
| 836 |
+
"turns": 445,
|
| 837 |
+
"loads": 429,
|
| 838 |
+
"empty": 222,
|
| 839 |
+
"needed_turns": 228,
|
| 840 |
+
"any_needed_loaded": 148,
|
| 841 |
+
"all_needed_loaded": 57,
|
| 842 |
+
"tn": 30312,
|
| 843 |
+
"fn": 330,
|
| 844 |
+
"fp": 235,
|
| 845 |
+
"tp": 194
|
| 846 |
+
},
|
| 847 |
+
"precision": 0.4522144522144522,
|
| 848 |
+
"recall": 0.3702290076335878,
|
| 849 |
+
"f0.5": 0.4330357142857143,
|
| 850 |
+
"loads_per_turn": 0.9640449438202248,
|
| 851 |
+
"empty_turn_rate": 0.49887640449438203
|
| 852 |
+
},
|
| 853 |
+
{
|
| 854 |
+
"threshold": 0.75,
|
| 855 |
+
"counts": {
|
| 856 |
+
"out_of_catalog_labels": 31,
|
| 857 |
+
"out_of_catalog_positives": 0,
|
| 858 |
+
"turns": 445,
|
| 859 |
+
"loads": 371,
|
| 860 |
+
"empty": 238,
|
| 861 |
+
"needed_turns": 228,
|
| 862 |
+
"any_needed_loaded": 141,
|
| 863 |
+
"all_needed_loaded": 51,
|
| 864 |
+
"tn": 30354,
|
| 865 |
+
"fn": 346,
|
| 866 |
+
"fp": 193,
|
| 867 |
+
"tp": 178
|
| 868 |
+
},
|
| 869 |
+
"precision": 0.4797843665768194,
|
| 870 |
+
"recall": 0.33969465648854963,
|
| 871 |
+
"f0.5": 0.44322709163346613,
|
| 872 |
+
"loads_per_turn": 0.8337078651685393,
|
| 873 |
+
"empty_turn_rate": 0.5348314606741573
|
| 874 |
+
},
|
| 875 |
+
{
|
| 876 |
+
"threshold": 0.8,
|
| 877 |
+
"counts": {
|
| 878 |
+
"out_of_catalog_labels": 31,
|
| 879 |
+
"out_of_catalog_positives": 0,
|
| 880 |
+
"turns": 445,
|
| 881 |
+
"loads": 307,
|
| 882 |
+
"empty": 261,
|
| 883 |
+
"needed_turns": 228,
|
| 884 |
+
"any_needed_loaded": 129,
|
| 885 |
+
"all_needed_loaded": 45,
|
| 886 |
+
"tn": 30395,
|
| 887 |
+
"fn": 369,
|
| 888 |
+
"fp": 152,
|
| 889 |
+
"tp": 155
|
| 890 |
+
},
|
| 891 |
+
"precision": 0.504885993485342,
|
| 892 |
+
"recall": 0.2958015267175573,
|
| 893 |
+
"f0.5": 0.442351598173516,
|
| 894 |
+
"loads_per_turn": 0.6898876404494382,
|
| 895 |
+
"empty_turn_rate": 0.5865168539325842
|
| 896 |
+
},
|
| 897 |
+
{
|
| 898 |
+
"threshold": 0.85,
|
| 899 |
+
"counts": {
|
| 900 |
+
"out_of_catalog_labels": 31,
|
| 901 |
+
"out_of_catalog_positives": 0,
|
| 902 |
+
"turns": 445,
|
| 903 |
+
"loads": 267,
|
| 904 |
+
"empty": 278,
|
| 905 |
+
"needed_turns": 228,
|
| 906 |
+
"any_needed_loaded": 116,
|
| 907 |
+
"all_needed_loaded": 36,
|
| 908 |
+
"tn": 30417,
|
| 909 |
+
"fn": 387,
|
| 910 |
+
"fp": 130,
|
| 911 |
+
"tp": 137
|
| 912 |
+
},
|
| 913 |
+
"precision": 0.5131086142322098,
|
| 914 |
+
"recall": 0.26145038167938933,
|
| 915 |
+
"f0.5": 0.43027638190954776,
|
| 916 |
+
"loads_per_turn": 0.6,
|
| 917 |
+
"empty_turn_rate": 0.6247191011235955
|
| 918 |
+
},
|
| 919 |
+
{
|
| 920 |
+
"threshold": 0.89,
|
| 921 |
+
"counts": {
|
| 922 |
+
"out_of_catalog_labels": 31,
|
| 923 |
+
"out_of_catalog_positives": 0,
|
| 924 |
+
"turns": 445,
|
| 925 |
+
"loads": 222,
|
| 926 |
+
"empty": 295,
|
| 927 |
+
"needed_turns": 228,
|
| 928 |
+
"any_needed_loaded": 105,
|
| 929 |
+
"all_needed_loaded": 31,
|
| 930 |
+
"tn": 30444,
|
| 931 |
+
"fn": 405,
|
| 932 |
+
"fp": 103,
|
| 933 |
+
"tp": 119
|
| 934 |
+
},
|
| 935 |
+
"precision": 0.536036036036036,
|
| 936 |
+
"recall": 0.22709923664122136,
|
| 937 |
+
"f0.5": 0.4213881019830028,
|
| 938 |
+
"loads_per_turn": 0.49887640449438203,
|
| 939 |
+
"empty_turn_rate": 0.6629213483146067
|
| 940 |
+
},
|
| 941 |
+
{
|
| 942 |
+
"threshold": 0.9,
|
| 943 |
+
"counts": {
|
| 944 |
+
"out_of_catalog_labels": 31,
|
| 945 |
+
"out_of_catalog_positives": 0,
|
| 946 |
+
"turns": 445,
|
| 947 |
+
"loads": 216,
|
| 948 |
+
"empty": 297,
|
| 949 |
+
"needed_turns": 228,
|
| 950 |
+
"any_needed_loaded": 103,
|
| 951 |
+
"all_needed_loaded": 31,
|
| 952 |
+
"tn": 30448,
|
| 953 |
+
"fn": 407,
|
| 954 |
+
"fp": 99,
|
| 955 |
+
"tp": 117
|
| 956 |
+
},
|
| 957 |
+
"precision": 0.5416666666666666,
|
| 958 |
+
"recall": 0.22328244274809161,
|
| 959 |
+
"f0.5": 0.4214697406340058,
|
| 960 |
+
"loads_per_turn": 0.4853932584269663,
|
| 961 |
+
"empty_turn_rate": 0.6674157303370787
|
| 962 |
+
},
|
| 963 |
+
{
|
| 964 |
+
"threshold": 0.95,
|
| 965 |
+
"counts": {
|
| 966 |
+
"out_of_catalog_labels": 31,
|
| 967 |
+
"out_of_catalog_positives": 0,
|
| 968 |
+
"turns": 445,
|
| 969 |
+
"loads": 137,
|
| 970 |
+
"empty": 338,
|
| 971 |
+
"needed_turns": 228,
|
| 972 |
+
"any_needed_loaded": 71,
|
| 973 |
+
"all_needed_loaded": 19,
|
| 974 |
+
"tn": 30485,
|
| 975 |
+
"fn": 449,
|
| 976 |
+
"fp": 62,
|
| 977 |
+
"tp": 75
|
| 978 |
+
},
|
| 979 |
+
"precision": 0.5474452554744526,
|
| 980 |
+
"recall": 0.1431297709923664,
|
| 981 |
+
"f0.5": 0.3498134328358209,
|
| 982 |
+
"loads_per_turn": 0.30786516853932583,
|
| 983 |
+
"empty_turn_rate": 0.7595505617977528
|
| 984 |
+
},
|
| 985 |
+
{
|
| 986 |
+
"threshold": 0.97,
|
| 987 |
+
"counts": {
|
| 988 |
+
"out_of_catalog_labels": 31,
|
| 989 |
+
"out_of_catalog_positives": 0,
|
| 990 |
+
"turns": 445,
|
| 991 |
+
"loads": 85,
|
| 992 |
+
"empty": 374,
|
| 993 |
+
"needed_turns": 228,
|
| 994 |
+
"any_needed_loaded": 45,
|
| 995 |
+
"all_needed_loaded": 11,
|
| 996 |
+
"tn": 30511,
|
| 997 |
+
"fn": 475,
|
| 998 |
+
"fp": 36,
|
| 999 |
+
"tp": 49
|
| 1000 |
+
},
|
| 1001 |
+
"precision": 0.5764705882352941,
|
| 1002 |
+
"recall": 0.09351145038167939,
|
| 1003 |
+
"f0.5": 0.2835648148148148,
|
| 1004 |
+
"loads_per_turn": 0.19101123595505617,
|
| 1005 |
+
"empty_turn_rate": 0.8404494382022472
|
| 1006 |
+
},
|
| 1007 |
+
{
|
| 1008 |
+
"threshold": 0.99,
|
| 1009 |
+
"counts": {
|
| 1010 |
+
"out_of_catalog_labels": 31,
|
| 1011 |
+
"out_of_catalog_positives": 0,
|
| 1012 |
+
"turns": 445,
|
| 1013 |
+
"loads": 12,
|
| 1014 |
+
"empty": 433,
|
| 1015 |
+
"needed_turns": 228,
|
| 1016 |
+
"any_needed_loaded": 7,
|
| 1017 |
+
"all_needed_loaded": 1,
|
| 1018 |
+
"tn": 30542,
|
| 1019 |
+
"fn": 517,
|
| 1020 |
+
"fp": 5,
|
| 1021 |
+
"tp": 7
|
| 1022 |
+
},
|
| 1023 |
+
"precision": 0.5833333333333334,
|
| 1024 |
+
"recall": 0.013358778625954198,
|
| 1025 |
+
"f0.5": 0.06118881118881119,
|
| 1026 |
+
"loads_per_turn": 0.02696629213483146,
|
| 1027 |
+
"empty_turn_rate": 0.9730337078651685
|
| 1028 |
+
}
|
| 1029 |
+
]
|
| 1030 |
+
}
|
| 1031 |
+
},
|
| 1032 |
+
"B5-094": {
|
| 1033 |
+
"original_report_sha256": "35c09c36938053b2eb285f4ca8700e2b61b0bb2ba96f1a63d1bbe2004e929bcf",
|
| 1034 |
+
"input_sha256": {
|
| 1035 |
+
"predictions": "e70357d58524a0714882aef5c5de98a6c4cf74a3ad82ae23ee1ed9f97789d1c0",
|
| 1036 |
+
"rows": "67c4c86e3385f0241048f2633d97a4aad78a53a3b7a540592937a20eba669fd3",
|
| 1037 |
+
"corpus": "c77abd8b40965c05cc43a7545e8c2db2405bdcfc18a73bad254632d523b67bcd"
|
| 1038 |
+
},
|
| 1039 |
+
"prediction_metadata_sha256": "de8a6974d1ca4a615f319089b2a6b023e5e57ede9bc03e52de5aa9524bde11c9",
|
| 1040 |
+
"encoding": "independent",
|
| 1041 |
+
"tool_option_words": 60,
|
| 1042 |
+
"head_sha256": "550f30b94d5c6a8680d2cb2a607ebc9ee33e709c7679f72606d1e9fda17f2e90",
|
| 1043 |
+
"truth": {
|
| 1044 |
+
"consensus": [
|
| 1045 |
+
{
|
| 1046 |
+
"threshold": 0.95,
|
| 1047 |
+
"counts": {
|
| 1048 |
+
"out_of_catalog_labels": 31,
|
| 1049 |
+
"out_of_catalog_positives": 0,
|
| 1050 |
+
"turns": 445,
|
| 1051 |
+
"loads": 137,
|
| 1052 |
+
"empty": 338,
|
| 1053 |
+
"needed_turns": 216,
|
| 1054 |
+
"any_needed_loaded": 77,
|
| 1055 |
+
"all_needed_loaded": 53,
|
| 1056 |
+
"tn": 29780,
|
| 1057 |
+
"fp": 18,
|
| 1058 |
+
"fn": 202,
|
| 1059 |
+
"unknown_loaded": 36,
|
| 1060 |
+
"tp": 83
|
| 1061 |
+
},
|
| 1062 |
+
"precision": 0.8217821782178217,
|
| 1063 |
+
"recall": 0.2912280701754386,
|
| 1064 |
+
"f0.5": 0.602322206095791,
|
| 1065 |
+
"loads_per_turn": 0.30786516853932583,
|
| 1066 |
+
"empty_turn_rate": 0.7595505617977528
|
| 1067 |
+
},
|
| 1068 |
+
{
|
| 1069 |
+
"threshold": 0.94,
|
| 1070 |
+
"counts": {
|
| 1071 |
+
"out_of_catalog_labels": 31,
|
| 1072 |
+
"out_of_catalog_positives": 0,
|
| 1073 |
+
"turns": 445,
|
| 1074 |
+
"loads": 161,
|
| 1075 |
+
"empty": 326,
|
| 1076 |
+
"needed_turns": 216,
|
| 1077 |
+
"any_needed_loaded": 83,
|
| 1078 |
+
"all_needed_loaded": 58,
|
| 1079 |
+
"tn": 29775,
|
| 1080 |
+
"fp": 23,
|
| 1081 |
+
"fn": 194,
|
| 1082 |
+
"unknown_loaded": 47,
|
| 1083 |
+
"tp": 91
|
| 1084 |
+
},
|
| 1085 |
+
"precision": 0.7982456140350878,
|
| 1086 |
+
"recall": 0.3192982456140351,
|
| 1087 |
+
"f0.5": 0.6140350877192983,
|
| 1088 |
+
"loads_per_turn": 0.36179775280898874,
|
| 1089 |
+
"empty_turn_rate": 0.7325842696629213
|
| 1090 |
+
}
|
| 1091 |
+
],
|
| 1092 |
+
"called": [
|
| 1093 |
+
{
|
| 1094 |
+
"threshold": 0.95,
|
| 1095 |
+
"counts": {
|
| 1096 |
+
"out_of_catalog_labels": 31,
|
| 1097 |
+
"out_of_catalog_positives": 0,
|
| 1098 |
+
"turns": 445,
|
| 1099 |
+
"loads": 137,
|
| 1100 |
+
"empty": 338,
|
| 1101 |
+
"needed_turns": 228,
|
| 1102 |
+
"any_needed_loaded": 71,
|
| 1103 |
+
"all_needed_loaded": 19,
|
| 1104 |
+
"tn": 30485,
|
| 1105 |
+
"fn": 449,
|
| 1106 |
+
"fp": 62,
|
| 1107 |
+
"tp": 75
|
| 1108 |
+
},
|
| 1109 |
+
"precision": 0.5474452554744526,
|
| 1110 |
+
"recall": 0.1431297709923664,
|
| 1111 |
+
"f0.5": 0.3498134328358209,
|
| 1112 |
+
"loads_per_turn": 0.30786516853932583,
|
| 1113 |
+
"empty_turn_rate": 0.7595505617977528
|
| 1114 |
+
},
|
| 1115 |
+
{
|
| 1116 |
+
"threshold": 0.94,
|
| 1117 |
+
"counts": {
|
| 1118 |
+
"out_of_catalog_labels": 31,
|
| 1119 |
+
"out_of_catalog_positives": 0,
|
| 1120 |
+
"turns": 445,
|
| 1121 |
+
"loads": 161,
|
| 1122 |
+
"empty": 326,
|
| 1123 |
+
"needed_turns": 228,
|
| 1124 |
+
"any_needed_loaded": 83,
|
| 1125 |
+
"all_needed_loaded": 21,
|
| 1126 |
+
"tn": 30476,
|
| 1127 |
+
"fn": 434,
|
| 1128 |
+
"fp": 71,
|
| 1129 |
+
"tp": 90
|
| 1130 |
+
},
|
| 1131 |
+
"precision": 0.5590062111801242,
|
| 1132 |
+
"recall": 0.1717557251908397,
|
| 1133 |
+
"f0.5": 0.3852739726027397,
|
| 1134 |
+
"loads_per_turn": 0.36179775280898874,
|
| 1135 |
+
"empty_turn_rate": 0.7325842696629213
|
| 1136 |
+
}
|
| 1137 |
+
]
|
| 1138 |
+
}
|
| 1139 |
+
}
|
| 1140 |
+
}
|
| 1141 |
+
}
|
evidence/unit-rank-historical-acceptance.json
ADDED
|
@@ -0,0 +1,78 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"scope": "Historical E4 acceptance under the earlier listing and preload policy; not current free-text lookup or current preload acceptance.",
|
| 3 |
+
"head_sha256": "c62674993eabba94660eccfafbb6e04d1549adc7335656c0ddefa7f068fa58f7",
|
| 4 |
+
"original_report_sha256": "aa6910d6a8c4769fea3aeb987fe351c735772780a189b06c837d678e6af334df",
|
| 5 |
+
"original_verdict_sha256": "17d97a6e41623c55e5f2d678cf4f442097ee3ea2095e419522fcefbc2cf3815b",
|
| 6 |
+
"head_binding_record_sha256": "769aaafde400eb9b77238ff6c1fbf1136761b58131f8a2554ba86234f52d811f",
|
| 7 |
+
"policy": {
|
| 8 |
+
"preload_at": 3.99,
|
| 9 |
+
"list_at": 0.5,
|
| 10 |
+
"request": {
|
| 11 |
+
"load_at": 1.95,
|
| 12 |
+
"max_loads": 4,
|
| 13 |
+
"nearest_loads": 1
|
| 14 |
+
}
|
| 15 |
+
},
|
| 16 |
+
"skills": {
|
| 17 |
+
"rows": 789,
|
| 18 |
+
"no_skill_rows": 50,
|
| 19 |
+
"generated_none_rows": 57,
|
| 20 |
+
"generated_none_agreed_negative": 23,
|
| 21 |
+
"relevant_rows": 660,
|
| 22 |
+
"relevant_preloaded": 122,
|
| 23 |
+
"any_raw@6": 0.9212121212121213,
|
| 24 |
+
"any_visible@6": 0.9212121212121213,
|
| 25 |
+
"all_visible@6": 0.7666666666666667,
|
| 26 |
+
"any_raw@8": 0.943939393939394,
|
| 27 |
+
"any_visible@8": 0.943939393939394,
|
| 28 |
+
"all_visible@8": 0.8090909090909091,
|
| 29 |
+
"preloads": 123,
|
| 30 |
+
"preloads_unscored": 0,
|
| 31 |
+
"preloads_correct": 122,
|
| 32 |
+
"no_skill_preloads": 0,
|
| 33 |
+
"generated_none_preloads": 0,
|
| 34 |
+
"preloads_possible": 1,
|
| 35 |
+
"preload_precision": 0.991869918699187,
|
| 36 |
+
"no_skill_preload_rate": 0.0,
|
| 37 |
+
"bands": {
|
| 38 |
+
"rare": {
|
| 39 |
+
"support": 572,
|
| 40 |
+
"visible@6": 0.8041958041958042,
|
| 41 |
+
"macro_support3": 0.8244539343687479
|
| 42 |
+
},
|
| 43 |
+
"mid": {
|
| 44 |
+
"support": 254,
|
| 45 |
+
"visible@6": 0.84251968503937,
|
| 46 |
+
"macro_support3": 0.8774649925511994
|
| 47 |
+
},
|
| 48 |
+
"common": {
|
| 49 |
+
"support": 257,
|
| 50 |
+
"visible@6": 0.8793774319066148,
|
| 51 |
+
"macro_support3": 0.8631465825713925
|
| 52 |
+
}
|
| 53 |
+
}
|
| 54 |
+
},
|
| 55 |
+
"requests": {
|
| 56 |
+
"needs": 0,
|
| 57 |
+
"top1": null,
|
| 58 |
+
"served": null,
|
| 59 |
+
"precision": null,
|
| 60 |
+
"f1": 0,
|
| 61 |
+
"loads_per_need": null
|
| 62 |
+
},
|
| 63 |
+
"verdict": {
|
| 64 |
+
"checks": {
|
| 65 |
+
"rare_pooled": true,
|
| 66 |
+
"rare_macro": true,
|
| 67 |
+
"mid_pooled": true,
|
| 68 |
+
"mid_macro": true,
|
| 69 |
+
"common_pooled": false,
|
| 70 |
+
"no_category_collapse": true,
|
| 71 |
+
"preload_precision": true,
|
| 72 |
+
"negative_false_preloads": true,
|
| 73 |
+
"preload_coverage": true
|
| 74 |
+
},
|
| 75 |
+
"skills_passed": false
|
| 76 |
+
},
|
| 77 |
+
"transformation": "Aggregate policy, skills and request metrics retained; per-skill and redundant strata breakdowns omitted. Verdict booleans retained."
|
| 78 |
+
}
|
release.json
ADDED
|
@@ -0,0 +1,149 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"version": 1,
|
| 3 |
+
"release": "open1-b5-e4",
|
| 4 |
+
"backbone": {
|
| 5 |
+
"model": "convaiinnovations/laya-multilingual",
|
| 6 |
+
"revision": "e4e9ddf21a7b1903b7acffd8814ad4307bf63a67"
|
| 7 |
+
},
|
| 8 |
+
"heads": {
|
| 9 |
+
"turn-load": {
|
| 10 |
+
"sha256": "550f30b94d5c6a8680d2cb2a607ebc9ee33e709c7679f72606d1e9fda17f2e90",
|
| 11 |
+
"label": "open1-turn-load-B5-release-independent"
|
| 12 |
+
},
|
| 13 |
+
"unit-rank": {
|
| 14 |
+
"sha256": "c62674993eabba94660eccfafbb6e04d1549adc7335656c0ddefa7f068fa58f7",
|
| 15 |
+
"label": "open1-unit-rank-e4-dense1"
|
| 16 |
+
},
|
| 17 |
+
"code-rank": {
|
| 18 |
+
"sha256": "abe235c347a223e957a0f8b7b979e3a584aebab76ee239e5f62117e839b9c40a",
|
| 19 |
+
"label": "open1-code-rank"
|
| 20 |
+
}
|
| 21 |
+
},
|
| 22 |
+
"preload": {
|
| 23 |
+
"catalog_revision": "dc3bf9215aac8877",
|
| 24 |
+
"encoding": "independent",
|
| 25 |
+
"option_words": 60,
|
| 26 |
+
"max_len": 1024,
|
| 27 |
+
"head_tokens": 512,
|
| 28 |
+
"options": {
|
| 29 |
+
"answer_decision": "Answer a worker's needs_decision: pick one of its offered options; the worker resumes on its same session with your choice.",
|
| 30 |
+
"ask_user": "Structured question for the user \u2014 this tool is the only way to ask one, and assistant prose is not an answer. Host parks until the user resolves; next wake returns this tool result. Prefer choices[] when present. Parallel while pending \u2192 ASK_USER_ALREADY_PENDING. Fill args per field docs below.",
|
| 31 |
+
"capture_page": "One-shot open\u2192act\u2192snapshot\u2192close; prefer page_* for live/authenticated state. Use project_dir for a static tree or url plus process_handle for a loopback app. Targets are exclusive. Returns structure, requests, errors, console, and a screenshot; use filmstrip for settled sequence claims and timeline for anything that happens while the page is painting.",
|
| 32 |
+
"chmod": "Change approval bits on repo-relative paths.",
|
| 33 |
+
"chown": "Change file ownership within write scope (never use command chown).",
|
| 34 |
+
"code_rewrite": "Rewrite a code shape across path or paths[]. $VAR matches one node; $$$VAR a sequence. Narrow bare `$A | $B` patterns (CODE_REWRITE_PATTERN_TOO_BROAD). Parse-breaking rewrites are refused (MUTATION_BROKE_PARSE). For search, use grep structural: true. Track changed, blocked (code/details), failed, unobserved_files and truncated separately: blocked is not no-match, and successful siblings may already be saved. Address each cause without repeating saved mutations",
|
| 35 |
+
"command": "Needs `command` or `pipeline` (not both). Runs as argv without a shell: `&&`/`||`/`;` sequence commands, `pipeline` carries `|`, `>`/`>>`/`2>` redirect, globs and `@scratch/<path>` expand, `$()`/backticks reject. Leading `NAME=value` assignments scope to their command; shared variables belong in `env`. Pass cwd; `cd` is rejected. Omit output paths to return output tails.",
|
| 36 |
+
"command_output": "Read command job output since cursor and current host-observed network facts. Inspect the handle of the process making a failed connection before attributing an application error to the sandbox. `network` records observed destinations and allow/deny decisions; `boundary_refusal` reports host attribution. Allowed access does not imply a successful upstream response.",
|
| 37 |
+
"command_stop": "Request termination of a command job process tree. stop_requested acknowledges the request; running stays true until exit is observed. Request once, then wait on the handle or inspect command_output rather than repeating stop.",
|
| 38 |
+
"complete_leg": "Finish this worker leg. Call once when the assignment is done or blocked. The host builds the parent report from these fields plus disk proof.",
|
| 39 |
+
"copy": "Copy files within write scope (files only; never use command cp).",
|
| 40 |
+
"delegate_decompose": "Decompose an approved plan into delegation legs",
|
| 41 |
+
"delegate_dispatch": "Dispatch a delegation leg to the worker queue",
|
| 42 |
+
"delegate_init": "Create a delegation departure for the current coordinator session",
|
| 43 |
+
"delegate_status": "Return delegation departure status and legs",
|
| 44 |
+
"delete": "Remove files and empty directories within write scope; the person can undo each removal.",
|
| 45 |
+
"diff": "Unified diff between two repo-relative file paths (read-scope only; never use command diff).",
|
| 46 |
+
"edit": "Replace an exact span in a write-scope file. Batch independent edits to distinct files. Read in an earlier response (same-batch read+edit rejects: EDITOR_DOCUMENT_READ_REQUIRED). Copy old_string after line-number tab; unique unless replace_all. Parse-breaking splices: MUTATION_BROKE_PARSE. New files: write.",
|
| 47 |
+
"extend_worker_budget": "Grant a running worker a higher tool-round ceiling, usually in answer to its request_budget; the worker is told on its next round and its request closes.",
|
| 48 |
+
"extract_archive": "Unpack .zip or .tar.gz into a write-scoped directory (replaces unzip/tar xzf; HITL ask by default).",
|
| 49 |
+
"fanout_plan": "Stamp a planned parallel fan-out (1\u2013N read-only legs). Call on fanout_planned phases, then workflow_advance. Stamping records the plan only; it does not enqueue workers. In the worker phase, dispatch each leg with task and its host-issued workflow_work_id. Recover incomplete legs within the plan's attempt allowance.",
|
| 50 |
+
"fetch_url": "Read a web page, documentation site, or other online resource. mode=text (default) extracts readable page text (HTML\u2192markdown); large bodies return a head + symbol map \u2014 page with offset+limit (cached). mode=raw returns the verbatim resource: textual MIME (HTML/CSS/JS/JSON/SVG/\u2026) as text with the same paging; image MIME types without dest automatically ingest into the session visual store and emit visual captures",
|
| 51 |
+
"find": "Bounded recursive directory walk \u2014 canonical for ** name_glob. Default type=any returns files and directories together. Wide unscoped walk at path=\".\" (2000+ paths) returns view digest (selected=0) with distribution + FIND_OVERFLOW_NARROW \u2014 pass name_glob when you can name a filename pattern. A concrete path (not \".\") or name_glob returns literal results[] (same as list_dir targeted). Explicit max_results or offset also",
|
| 52 |
+
"git_blame": "Per-line blame JSON (`available`, `lines` with commit/author/date/content).",
|
| 53 |
+
"git_branches": "List local branches JSON (`available`, `branches` with name/current).",
|
| 54 |
+
"git_checkout": "Switch to an existing local branch, or explicitly create and switch to a new one. Rehearses and reviews file changes before applying. Preserves unrelated local edits; refuses conflicting edits. Reports the actual branch and HEAD even after failure.",
|
| 55 |
+
"git_commit": "Stage and commit paths with a message; amend=true replaces HEAD with the selected paths, including new files, leaving unrelated staged changes alone. Success JSON: available, hash, paths, and uncommitted_paths/uncommitted_count for dirty files this commit left alone. Failure JSON: available=false, error, optional hook_failed, output_tail (last hook/CLI lines), and hint. Empty paths stages only files this session wrote; name paths explicitly to",
|
| 56 |
+
"git_compare": "Compare frozen commit IDs: ahead/behind counts for head_ref relative to base_ref, plus all merge bases.",
|
| 57 |
+
"git_diff": "Git diff JSON, one page of files in git's order (`available`, `files` with per-file insertions/deletions and, unless `stat`, that file's hunks in `diff`; `available:false` on non-git projects). Pages hold up to 80 files and 5 KiB of hunks by default: read `files_total`, use `next_offset` for more files when needed; exhaust the selection for tasks requiring complete coverage. `wire_spill_path` retains the",
|
| 58 |
+
"git_log": "Read paged commit history from a branch, tag, commit, or revision range. Use ref for revisions and path only for a file. main..feature lists commits unique to feature; main...feature lists both sides. Follow next_offset to continue.",
|
| 59 |
+
"git_merge": "Merge a revision into the current branch. Rehearses and reviews file effects. Conflicts are explicit and retain the merge state: edit resolutions, then continue with explicit paths; abort only when asked to discard merge work. Check status and after state, not diagnostics, for success.",
|
| 60 |
+
"git_ref": "Resolve refs to SHAs JSON (`available`, `refs` with ref/sha/peeled).",
|
| 61 |
+
"git_restore": "Restore explicit paths in the Git worktree, index, or both. Defaults to Git's ordinary worktree-from-index behavior. This discards selected state \u2014 use only when the user explicitly asked to revert named paths.",
|
| 62 |
+
"git_show": "Show commit or file at ref JSON (`available`, `show` with kind diff/blob).",
|
| 63 |
+
"git_stash": "Save explicit repository paths temporarily, apply a saved stash, or explicitly drop one. Save preserves unrelated edits and index entries. Apply retains the stash even on success; use drop separately after verifying restoration. Conflicts and partial effects are reported in structured state.",
|
| 64 |
+
"git_stash_list": "List stashes with immutable IDs and current selectors for apply/drop.",
|
| 65 |
+
"git_status": "Git working tree status JSON (`available`, `branch`, `head_short`, `upstream`, `ahead`/`behind`, `dirty`, staged/unstaged/untracked counts, `files` with porcelain XY status one entry per file \u2014 untracked directories are expanded; `available:false` on non-git projects). Pages of 80 files: read `files_total`, use `next_offset` to continue when more coverage is needed; exhaust pages when the task requires the complete selection. `paths` narrows the selection to",
|
| 66 |
+
"grep": "Search files with RE2 text patterns or tree-sitter structural patterns. Recurses under path with optional path_glob, searches gitignored files, and skips engine/VCS metadata. Wide results may return a view digest.",
|
| 67 |
+
"handoff_health": "Report reservation counts for a handoff session plane.",
|
| 68 |
+
"handoff_init": "Bind the parent session handoff plane for sibling worker coordination.",
|
| 69 |
+
"handoff_release": "Release one or more path reservations held by the caller agent.",
|
| 70 |
+
"handoff_release_all": "Release every path reservation held by the caller agent on the session plane.",
|
| 71 |
+
"handoff_reserve": "Record repo-relative path reservations on the parent session plane (informational; merge resolves overlaps).",
|
| 72 |
+
"held_result": "Read a held call by its handle, running or settled with its original outcome, codes, and result. wait_ms waits up to 30 s for a running call.",
|
| 73 |
+
"held_stop": "Cancel a running held call. Request once, then wait on the handle or read call_result.",
|
| 74 |
+
"http_request": "Send one bounded request to an API, webhook, or service endpoint and return its status, headers, and raw body.",
|
| 75 |
+
"jq": "Query JSON, YAML, or TOML via sandboxed jq. Pass limit/offset for literal values[].",
|
| 76 |
+
"jq_edit": "Update a JSON, YAML, or TOML document with a jq program and write the result in the same format, in place or to dest. The program returns the whole document (`.a.b = 1`, `del(.x)`, `.items |= map(...)`). Unchanged values keep their literals, key order, and YAML comments; syntax a rewrite would lose is refused (JQ_EDIT_LOSSY).",
|
| 77 |
+
"list_dir": "Locate repository areas with {\"path\":\".\"}. A named directory or explicit listing controls return entries instead. Use summarize to explain the area.",
|
| 78 |
+
"measure_page": "Measure selector geometry in CSS pixels, and whether the pointer can reach each element or what covers or clips it. Use id for an existing live page, or one exclusive project_dir/loopback url target. annotate adds boxes; page_geometry remains the numeric evidence.",
|
| 79 |
+
"mkdir": "Create directories within write scope (never use command mkdir).",
|
| 80 |
+
"move": "Rename files within write scope (never use command mv); a replaced destination can be undone.",
|
| 81 |
+
"pack_board": "When the board is stale, workers changed, or you need a scan/layout pulse \u2014 return the board snapshot. Not a substitute for summarize or a known-file read.",
|
| 82 |
+
"page_act": "Drive a held page with trusted input. Each step aims where its target receives the pointer; the result reports that point plus the requests, errors, and console output the drive caused. With record, the drive is also a timeline seen as a labeled contact sheet.",
|
| 83 |
+
"page_close": "Tear down a held page; optional final snapshot.",
|
| 84 |
+
"page_open": "Open a live page; returns an id held across turns. The page has its own cookies and storage, which persist until page_close. Up to max_pages stay open per session; re-opening a held target reloads it with the given viewport and routes. Prefer capture_page for a single check. Bindings match capture_page (project_dir or url).",
|
| 85 |
+
"page_snapshot": "Universal {state, snapshot, log, artifact} for a held page (does not close it).",
|
| 86 |
+
"parse_decomposition": "Validate decomposition.",
|
| 87 |
+
"parse_delegation_plan": "Validate delegation plan.",
|
| 88 |
+
"parse_evaluation": "Validate evaluation.",
|
| 89 |
+
"parse_extract_json": "Extract JSON.",
|
| 90 |
+
"parse_validate": "Validate JSON.",
|
| 91 |
+
"plan_append_review_evidence": "Append advisory plan review evidence to critic.jsonl",
|
| 92 |
+
"preview_overlay": "Merge preview \u2014 same plan as promote_overlay without landing. overlay_id == job_id. Returns path_status and ready_resolutions; spill_path is host-data-relative (promote-spills/<id>.json). Example: {\"overlay_id\":\"<job_id>\"}.",
|
| 93 |
+
"process_list": "Inspect host processes after approval. Returns names, executable paths, UID, parent PID, and opaque instance references; no argv or environment. Use pid for a known target. Use next_after_pid for another page of the changing snapshot. Unavailable metadata does not mean exited. Use command_output and command_stop for jobs you launched with command.",
|
| 94 |
+
"process_signal": "Send a signal to exact host process instances after approval. Pass opaque references returned by process_list in this task. PID reuse or changed identity rejects; names, bare PIDs, groups, and descendants are not targets. TERM requests graceful termination; KILL can discard unsaved work. Delivered means the signal was accepted, not that exit was observed. Inspect the result for each target.",
|
| 95 |
+
"promote_overlay": "Atomically promote a worker overlay (overlay_id=job_id), considering all observed changes. Conflicts return a menu without writes; resolve all via resolutions/drop.",
|
| 96 |
+
"read": "Read a file and return its content or structure in JSON; unscoped large files return a structure digest. For literal content, set mode / offset / limit / ranges / symbol.",
|
| 97 |
+
"recall": "Look up what was already observed instead of observing it again. Reach for it when a finished worker leg's envelope omits a detail the leg saw, when a continuation record or a `[compacted \u2026]` banner names a handle whose body is gone from context, or \u2014 from a root session \u2014 when an earlier session on this project may already",
|
| 98 |
+
"record_finding": "Post peer intent for parallel siblings \u2014 shared paths, registry slots, merge-relevant facts. Cite a real path/handle.",
|
| 99 |
+
"reject_overlay": "Abort overlay without landing. overlay_id == job_id. Wrong/superseded only \u2014 not landable conflicts. Example: {\"overlay_id\":\"<job_id>\",\"reason\":\"Superseded by corrected dispatch\"}.",
|
| 100 |
+
"render_view": "Rasterize SVG or HTML into an authored visual: UI/product mockups (especially before building), diagrams, comparisons, timelines, or charts. Use when spatial layout improves clarity; never as runtime evidence \u2014 use capture_page for observed UI. Host injects an offline design kit (OFL fonts, Lucide-adapted icons, theme tokens, spacing/type scales, media placeholders). No CDN links, scripts, or external https assets \u2014 use",
|
| 101 |
+
"replace_lines": "Atomically replace or reindent line ranges. Use direct fields once, or operations for coupled non-overlapping seams in one snapshot. Read existing editable files in an earlier response before applying changes; concurrent edits rebase against that read. EDITOR_DOCUMENT_READ_REQUIRED or EDITOR_DOCUMENT_CHANGING requires read and recomputation.",
|
| 102 |
+
"request_budget": "Ask your coordinator for more tool rounds when this leg's goal needs more than remain. Name the rounds and the concrete work they cover. You keep working while the coordinator decides; a granted ceiling arrives as a host notice on your next round. If none arrives, finish with an honest partial report at your current ceiling. One request can be",
|
| 103 |
+
"request_decision": "Escalate an ambiguity you can't resolve in scope \u2014 question + 2+ options, the accepted call suspends your run. The coordinator answers and you resume with the same workspace. Do not also call complete_leg.",
|
| 104 |
+
"request_tools": "Load more tool schemas for the next model call by describing what the work needs",
|
| 105 |
+
"restore_version": "Restore a file to a previously retained source version as a forward mutation in the source ledger. Use source_history to inspect available versions or diffs.",
|
| 106 |
+
"scan_compare": "Diff two complete scans \u2014 returns the findings introduced, resolved, and unchanged since the baseline. After a fix, pass the post-fix scan as new_scan_id and omit old_scan_id: the host resolves the previous complete scan in the same category. Use it to prove a fix landed; a board Scan: tail is not proof.",
|
| 107 |
+
"scan_list": "List recent code scans for the active project (read-only). Returns a slim id roster with no guidance bodies \u2014 use it to resolve a scan_id for scan_summary, scan_query, or scan_compare when the board does not already show one.",
|
| 108 |
+
"scan_pack": "Run the host's configured code scanners over the active project and return one scan_pack. Use it when the ask is security or code-quality work and the board shows no complete scan at the current head. Returns per_scan[].scan_id \u2014 pass that exact array to scan_summary or scan_query for a cross-engine aggregate, or set completion=summary to receive the aggregate when this call",
|
| 109 |
+
"scan_query": "Page findings for one or more scans. One id queries one scan; the exact id array returned by scan_pack queries that pack as a cross-engine aggregate. Narrow with rule_id, path, code, or level rather than reading every finding. When a result is truncated, page it with offset \u2014 do not re-run the scan.",
|
| 110 |
+
"scan_summary": "Read status and guidance for one or more scans. One id reads one scan; the exact id array returned by scan_pack reads that pack as a single cross-engine aggregate, and a pass_id reads a full pass, including scanners that have not started yet. This is the first call after a scan completes \u2014 start here rather than grepping the repo",
|
| 111 |
+
"secret_generate": "Generate random secret material and return an opaque {{paintedwolf-secret:\u2026}} reference for tool execution.",
|
| 112 |
+
"secret_list": "List visible managed secret references and metadata, never values.",
|
| 113 |
+
"secret_revoke": "Revoke a managed secret reference while retaining provider-screening evidence.",
|
| 114 |
+
"skills_read": "Read a skill or resource by name, find skills by describing the work, or omit arguments to list every loaded skill.",
|
| 115 |
+
"source_history": "Query the source ledger for file effects (mode=effects), current line authorship (mode=lines), this session's paths (mode=mine), historical version content (mode=version), or version diffs (mode=diff). Includes uncommitted state. Actors: you / agent / user / external / mixed / unknown. Missing history is reported explicitly.",
|
| 116 |
+
"stat": "File metadata (mode, size, mtime, entry_count for directories)",
|
| 117 |
+
"state_close": "Exit the active workflow run for the session",
|
| 118 |
+
"state_query": "Query active workflow run and scaffold vars",
|
| 119 |
+
"state_start": "Propose a workflow run on the current session. The host records the proposal; an explicit human action starts it.",
|
| 120 |
+
"state_update": "Update model-authored scaffold data on the active workflow run",
|
| 121 |
+
"submit_verdict": "Record the phase's review_loop verdict per verdict_schema. Terminal verdicts satisfy its evidence_passed gate; other gates and advancement rules still apply. Non-terminal verdicts re-loop within the cap. Terminal verdicts must cite observed evidence paths/handles in cited_evidence or fetched URLs in cited_urls. Invented references reject; non-terminal verdicts need no citations.",
|
| 122 |
+
"summarize": "Explain a named file or area with a bounded, source-cited briefing. Path sets scope; task ranks within it. Use next_actions only when more evidence is needed.",
|
| 123 |
+
"surface_note": "Post a concise, grounded mid-turn note for the user, then keep working \u2014 this does not end the turn. Use it for meaningful progress or expectation changes. Say what happened and what comes next. Cite at least one cited_evidence or cited_urls entry. Carries artifact_ids, which is how a still reaches the user mid-run. For a question use ask_user; the final",
|
| 124 |
+
"survey_repo": "One catalog map (Tier 1 digest, selected=0), not a briefing. Call when you still cannot name a destination and the ask is unknown layout (layout_overview), duplicated env/config/stores (ssot_drift), or where HTTP routes/handlers live (api_routes). Then summarize the destination. For how/what on a named file/package/dir, use summarize(path=\u2026) \u2014 never survey_repo for depth on a path you can already say.",
|
| 125 |
+
"task": "Start background workers for parallel execution across concurrent legs and return control immediately. Dispatch ready legs together in the same turn or launch companion legs while work runs. For single-thread, localized changes, execute inline directly. Match the worker profile and scope to the assignment.",
|
| 126 |
+
"terminal_close": "Tear down a held pty (graceful then kill). It returns lifecycle acknowledgement only; use terminal_read or terminal_snapshot before closing when output matters.",
|
| 127 |
+
"terminal_open": "Live host terminal (pty): run a command-class argv command. Prefer command when you only need exit code/tail for yourself. Use this tool when the session must be held or the user should see the screen \u2014 then terminal_snapshot. Returns an id held across turns plus the requested observation (screen by default). Never supply known plaintext via later terminal_send \u2014 use",
|
| 128 |
+
"terminal_read": "Read incremental pty output since the last read (idle/timeout bounded).",
|
| 129 |
+
"terminal_send": "Write literal text and/or named control tokens ({Enter}, {Ctrl-C}, \u2026) to a held pty, then return the settled vt grid. Never send known plaintext passwords, tokens, or passphrases. An opaque {{paintedwolf-secret:\u2026}} reference is allowed when the interactive prompt is the consuming boundary; the host resolves and screens it.",
|
| 130 |
+
"terminal_snapshot": "User-visible photograph of the held terminal session (CLI output or TUI), parallel to capture_page for pages. Returns one bounded VT screen and a receipt, never a duplicate main-stream log. Use for any held session \u2014 not only full-screen TUIs. Settles on output quiescence; does not advance terminal_read's cursor.",
|
| 131 |
+
"update_progress": "Set session progress checklist (markdown). Replaces the whole checklist \u2014 see content for markers and caps.",
|
| 132 |
+
"verify": "Run a verification check and report its terminal outcome. Bare verify runs the selected project command. With no selected command, supply the documented check appropriate to the task. A different command or pipeline is rejected when a project command is selected; use command(verification: true) for targeted checks. Selecting a command does not require it after every edit. outcome is passed",
|
| 133 |
+
"view_image": "Inspect an image file (.svg, .png, .jpg, .jpeg, .webp, .gif) or a session visual and perceive its rendered pixels and extracted text. Returns format metadata and extracted text, and emits a visual capture for multimodal models.",
|
| 134 |
+
"view_video": "See moments of a screen recording or other video (.mp4, .mov, .webm), including an attached one. Returns one sheet of frames, each labeled with its timestamp, plus the video's duration and size. Without times_ms it spreads count frames across start_ms to end_ms (default: the whole video); name exact times_ms to look closer at a moment, and crop to read a",
|
| 135 |
+
"wait": "Park without a worker slot until a condition is satisfied or timeout expires. Use until_complete for exact process handles without periodic timer wakes; omit conditions for a timer.",
|
| 136 |
+
"wc": "Count lines, bytes, and words in files or recursively in directories. bytes is the true file size; truncated marks counts taken from a capped scan.",
|
| 137 |
+
"web_search": "Search the web by time window. The host selects Direct search or custom providers from Settings \u2192 Web research.",
|
| 138 |
+
"worker_cancel": "Cancel a pending or running worker job; running workers get a prose-only cancellation report before the leg ends",
|
| 139 |
+
"workflow_advance": "Leave the current coordinator-controlled phase when gates pass. Returns phase_gate_unmet with failed_leaves when blocked; already_advanced if host auto-advanced first. Unavailable on host-auto phases.",
|
| 140 |
+
"workflow_catalog_summaries": "List bundled, session-composed, and template workflow ids.",
|
| 141 |
+
"workflow_compose": "Compose a session workflow from YAML. Returns effective_summary; host replays coordinator_brief each Prompt.",
|
| 142 |
+
"workflow_compose_from_template": "Compose from a bundled template. Returns effective_summary; read prepended coordinator_brief before proposing with state_start.",
|
| 143 |
+
"workflow_persist": "Save a session-composed workflow to .paintedwolf/workflows/<workflow_id>/workflow.yaml (requires user confirmation).",
|
| 144 |
+
"workflow_transition": "Fire choice leave by transition_id (Phase exit). Pending input latches block; human_approval_awaiting does not.",
|
| 145 |
+
"workflow_user_feedback": "Pending user feedback for the active run. Wait for the host card \u2014 do not infer answers.",
|
| 146 |
+
"write": "Create/replace files in write scope (append: true appends). Batch independent writes to distinct files; stage same-file or dependent writes. New or previously clean supported source must parse (MUTATION_BROKE_PARSE); broken files must improve and finish clean. Scope violation: WRITE_SCOPE_DENIED. Read existing editable files in an earlier response; concurrent edits rebase against it. EDITOR_DOCUMENT_READ_REQUIRED or EDITOR_DOCUMENT_CHANGING requires rereading and recomputatio"
|
| 147 |
+
}
|
| 148 |
+
}
|
| 149 |
+
}
|
turn-load.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:550f30b94d5c6a8680d2cb2a607ebc9ee33e709c7679f72606d1e9fda17f2e90
|
| 3 |
+
size 59881652
|
unit-rank.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c62674993eabba94660eccfafbb6e04d1549adc7335656c0ddefa7f068fa58f7
|
| 3 |
+
size 59881268
|