bradnow Claude Opus 4.8 commited on
Commit
5ff6441
Β·
1 Parent(s): 94f9b26

Add leaderboard auto-sync implementation plan

Browse files

12 TDD tasks: pure transform lib (parse/derive/validate/diff) with unit
tests, sync.mjs orchestrator (check/apply/push), page render + fallback,
SKILL.md, daily CronCreate. Push gated behind explicit --push flag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs/superpowers/plans/2026-06-02-leaderboard-sync.md ADDED
@@ -0,0 +1,1123 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Leaderboard Auto-Sync Implementation Plan
2
+
3
+ > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
4
+
5
+ **Goal:** Make the three landing-page leaderboards (EOG, EVA-Accuracy, EVA-Experience) render from a synced `data/leaderboard.json`, and add a Claude Code skill that daily pulls the two upstream sources, re-derives that JSON, and pushes to staging only when our displayed data changes and passes sanity checks.
6
+
7
+ **Architecture:** Pure transform functions (no I/O) parse upstream β†’ derive our schema; they're unit-tested with small synthetic fixtures. A `sync.mjs` orchestrator does fetch/diff/git/notify around them. The page keeps its current static markup as a fallback and an inline script overlays the fetched JSON on load. Push to staging is gated behind an explicit `--push` flag so dev runs can never deploy by accident.
8
+
9
+ **Tech Stack:** Node 20 (global `fetch`, `node:test`, `node:assert`, `node:crypto`, `node:child_process`), zero npm dependencies, vanilla inline JS on the page, Tailwind-CDN markup (unchanged classes).
10
+
11
+ ---
12
+
13
+ ## File Structure
14
+
15
+ **Create:**
16
+ - `.claude/skills/sync-nowai-leaderboard/package.json` β€” marks ESM + `test` script.
17
+ - `.claude/skills/sync-nowai-leaderboard/lib/transform.mjs` β€” pure parse/derive/validate/diff functions. No I/O.
18
+ - `.claude/skills/sync-nowai-leaderboard/sync.mjs` β€” orchestrator: fetch, hash, diff, git, notify, arg parsing.
19
+ - `.claude/skills/sync-nowai-leaderboard/SKILL.md` β€” when/how to run, guardrails.
20
+ - `.claude/skills/sync-nowai-leaderboard/test/transform.test.mjs` β€” unit tests for transform.mjs.
21
+ - `data/leaderboard.json` β€” the rendered data (generated from live upstream in Task 8).
22
+
23
+ **Modify:**
24
+ - `index.html` β€” add id hooks to the three cards' dynamic regions + an inline render script with fetch + fallback.
25
+
26
+ **Responsibilities:** `transform.mjs` is the only place that knows upstream shapes and our schema (testable in isolation). `sync.mjs` owns side effects (network, fs, git). The page's inline script owns presentation. Split by responsibility so each unit fits in context.
27
+
28
+ ---
29
+
30
+ ### Task 1: Skill scaffolding
31
+
32
+ **Files:**
33
+ - Create: `.claude/skills/sync-nowai-leaderboard/package.json`
34
+
35
+ - [ ] **Step 1: Create package.json**
36
+
37
+ ```json
38
+ {
39
+ "name": "sync-nowai-leaderboard",
40
+ "private": true,
41
+ "type": "module",
42
+ "scripts": {
43
+ "test": "node --test"
44
+ }
45
+ }
46
+ ```
47
+
48
+ - [ ] **Step 2: Verify test runner wiring (no tests yet β†’ exits 0 with "no test files")**
49
+
50
+ Run: `cd .claude/skills/sync-nowai-leaderboard && node --test`
51
+ Expected: exit 0, message like `tests 0`. (If it errors on no files, that's fine β€” proceed; Task 2 adds the first test.)
52
+
53
+ - [ ] **Step 3: Commit**
54
+
55
+ ```bash
56
+ git add .claude/skills/sync-nowai-leaderboard/package.json
57
+ git commit -m "chore: scaffold sync-nowai-leaderboard skill"
58
+ ```
59
+
60
+ ---
61
+
62
+ ### Task 2: EOG parser (`parseEogData`)
63
+
64
+ **Files:**
65
+ - Create: `.claude/skills/sync-nowai-leaderboard/test/transform.test.mjs`
66
+ - Create: `.claude/skills/sync-nowai-leaderboard/lib/transform.mjs`
67
+
68
+ - [ ] **Step 1: Write the failing test**
69
+
70
+ Create `test/transform.test.mjs`:
71
+
72
+ ```js
73
+ import { test } from 'node:test';
74
+ import assert from 'node:assert/strict';
75
+ import { parseEogData } from '../lib/transform.mjs';
76
+
77
+ const EOG_SAMPLE = `
78
+ // ── Leaderboard Data ──
79
+ const LEADERBOARD_DATA = [
80
+ // Closed Source
81
+ { model: "Gemini 3.5 Flash (High)", type: "closed", teams: 48.0, csm: 47.9, email: 55.8, itsm: 42.5, calendar: 42.3, hr: 40.8, drive: 55.2, hybrid: 35.3, avg: 46.0 },
82
+ { model: "Claude Opus 4.5", type: "closed", teams: 50.0, csm: 34.2, email: 51.9, itsm: 23.8, calendar: 43.2, hr: 32.1, drive: 49.5, hybrid: 30.7, avg: 37.0 },
83
+ ];
84
+ function renderLeaderboard() {}
85
+ `;
86
+
87
+ test('parseEogData extracts the array of model objects', () => {
88
+ const arr = parseEogData(EOG_SAMPLE);
89
+ assert.equal(arr.length, 2);
90
+ assert.equal(arr[0].model, 'Gemini 3.5 Flash (High)');
91
+ assert.equal(arr[0].type, 'closed');
92
+ assert.equal(arr[0].avg, 46.0);
93
+ assert.equal(arr[1].model, 'Claude Opus 4.5');
94
+ });
95
+
96
+ test('parseEogData throws when marker missing', () => {
97
+ assert.throws(() => parseEogData('const X = 1;'), /LEADERBOARD_DATA not found/);
98
+ });
99
+ ```
100
+
101
+ - [ ] **Step 2: Run test to verify it fails**
102
+
103
+ Run: `cd .claude/skills/sync-nowai-leaderboard && node --test`
104
+ Expected: FAIL β€” `Cannot find module '../lib/transform.mjs'`.
105
+
106
+ - [ ] **Step 3: Write minimal implementation**
107
+
108
+ Create `lib/transform.mjs`:
109
+
110
+ ```js
111
+ // Pure transforms: parse upstream sources and derive our leaderboard schema.
112
+ // No network or filesystem access lives here.
113
+
114
+ export function parseEogData(scriptText) {
115
+ const start = scriptText.indexOf('LEADERBOARD_DATA');
116
+ if (start === -1) throw new Error('LEADERBOARD_DATA not found in EOG script');
117
+ const open = scriptText.indexOf('[', start);
118
+ if (open === -1) throw new Error('EOG array open bracket not found');
119
+
120
+ // Match brackets to find the array literal end. Model names contain no [ or ].
121
+ let depth = 0;
122
+ let end = -1;
123
+ for (let i = open; i < scriptText.length; i++) {
124
+ const c = scriptText[i];
125
+ if (c === '[') depth++;
126
+ else if (c === ']') {
127
+ depth--;
128
+ if (depth === 0) { end = i; break; }
129
+ }
130
+ }
131
+ if (end === -1) throw new Error('EOG array close bracket not found');
132
+
133
+ let body = scriptText.slice(open, end + 1);
134
+ body = body.replace(/\/\/[^\n]*/g, ''); // strip line comments
135
+ body = body.replace(/([{,]\s*)([A-Za-z_]\w*)\s*:/g, '$1"$2":'); // quote keys
136
+ body = body.replace(/,(\s*[}\]])/g, '$1'); // strip trailing commas
137
+
138
+ try {
139
+ return JSON.parse(body);
140
+ } catch (e) {
141
+ throw new Error('EOG array parse failed: ' + e.message);
142
+ }
143
+ }
144
+ ```
145
+
146
+ - [ ] **Step 4: Run test to verify it passes**
147
+
148
+ Run: `cd .claude/skills/sync-nowai-leaderboard && node --test`
149
+ Expected: PASS (2/2).
150
+
151
+ - [ ] **Step 5: Commit**
152
+
153
+ ```bash
154
+ git add .claude/skills/sync-nowai-leaderboard/lib/transform.mjs .claude/skills/sync-nowai-leaderboard/test/transform.test.mjs
155
+ git commit -m "feat: parse EOG LEADERBOARD_DATA array from script.js"
156
+ ```
157
+
158
+ ---
159
+
160
+ ### Task 3: EOG derivation (`deriveEog`) + vendor map (`vendorFromName`)
161
+
162
+ **Files:**
163
+ - Modify: `.claude/skills/sync-nowai-leaderboard/test/transform.test.mjs`
164
+ - Modify: `.claude/skills/sync-nowai-leaderboard/lib/transform.mjs`
165
+
166
+ - [ ] **Step 1: Add failing tests**
167
+
168
+ Append to `test/transform.test.mjs`:
169
+
170
+ ```js
171
+ import { deriveEog, vendorFromName } from '../lib/transform.mjs';
172
+
173
+ test('vendorFromName maps known model families', () => {
174
+ assert.equal(vendorFromName('Gemini 3.5 Flash'), 'Google');
175
+ assert.equal(vendorFromName('GPT-5.4'), 'OpenAI');
176
+ assert.equal(vendorFromName('Claude Opus 4.5'), 'Anthropic');
177
+ assert.equal(vendorFromName('Mystery-7B'), 'Mystery-7B'); // fallback: first token
178
+ });
179
+
180
+ test('deriveEog sorts by avg desc, takes top 5, computes bars', () => {
181
+ const arr = [
182
+ { model: 'A', type: 'closed', avg: 20 },
183
+ { model: 'B', type: 'open', avg: 40 },
184
+ { model: 'C', type: 'closed', avg: 30 },
185
+ ];
186
+ const out = deriveEog(arr);
187
+ assert.equal(out.metricLabel, 'Task Success Rate Β· Oracle mode');
188
+ assert.deepEqual(out.rows.map(r => r.model), ['B', 'C', 'A']);
189
+ assert.equal(out.rows[0].rank, 1);
190
+ assert.equal(out.rows[0].score, 40);
191
+ assert.equal(out.rows[0].bar, 100);
192
+ assert.equal(out.rows[1].bar, 75); // 30/40
193
+ assert.equal(out.rows[2].bar, 50); // 20/40
194
+ assert.equal(out.rows[0].type, 'open');
195
+ });
196
+ ```
197
+
198
+ - [ ] **Step 2: Run test to verify it fails**
199
+
200
+ Run: `cd .claude/skills/sync-nowai-leaderboard && node --test`
201
+ Expected: FAIL β€” `deriveEog`/`vendorFromName` not exported.
202
+
203
+ - [ ] **Step 3: Write minimal implementation**
204
+
205
+ Append to `lib/transform.mjs`:
206
+
207
+ ```js
208
+ const round1 = (n) => Math.round(n * 10) / 10;
209
+
210
+ const VENDOR_MAP = [
211
+ [/gemini/i, 'Google'],
212
+ [/gpt|openai|\brealtime\b/i, 'OpenAI'],
213
+ [/whisper/i, 'OpenAI'],
214
+ [/claude/i, 'Anthropic'],
215
+ [/nova/i, 'Amazon'],
216
+ [/qwen/i, 'Alibaba'],
217
+ [/deepseek/i, 'DeepSeek'],
218
+ [/llama/i, 'Meta'],
219
+ ];
220
+
221
+ export function vendorFromName(name) {
222
+ for (const [re, vendor] of VENDOR_MAP) if (re.test(name)) return vendor;
223
+ return name.split(/[\s+]/)[0];
224
+ }
225
+
226
+ export function deriveEog(eogArray) {
227
+ const sorted = [...eogArray].sort((a, b) => b.avg - a.avg).slice(0, 5);
228
+ const top = sorted[0]?.avg ?? 0;
229
+ return {
230
+ metricLabel: 'Task Success Rate Β· Oracle mode',
231
+ rows: sorted.map((m, i) => ({
232
+ rank: i + 1,
233
+ model: m.model,
234
+ org: vendorFromName(m.model),
235
+ type: m.type,
236
+ score: round1(m.avg),
237
+ bar: top ? Math.round((m.avg / top) * 100) : 0,
238
+ })),
239
+ };
240
+ }
241
+ ```
242
+
243
+ - [ ] **Step 4: Run test to verify it passes**
244
+
245
+ Run: `cd .claude/skills/sync-nowai-leaderboard && node --test`
246
+ Expected: PASS (all).
247
+
248
+ - [ ] **Step 5: Commit**
249
+
250
+ ```bash
251
+ git add .claude/skills/sync-nowai-leaderboard/lib/transform.mjs .claude/skills/sync-nowai-leaderboard/test/transform.test.mjs
252
+ git commit -m "feat: derive EOG top-5 rows with vendor + relative bars"
253
+ ```
254
+
255
+ ---
256
+
257
+ ### Task 4: EVA board derivation (`parseEvaData`, `deriveEvaBoard`)
258
+
259
+ **Files:**
260
+ - Modify: `.claude/skills/sync-nowai-leaderboard/test/transform.test.mjs`
261
+ - Modify: `.claude/skills/sync-nowai-leaderboard/lib/transform.mjs`
262
+
263
+ - [ ] **Step 1: Add failing tests**
264
+
265
+ Append to `test/transform.test.mjs`:
266
+
267
+ ```js
268
+ import { parseEvaData, deriveEvaBoard } from '../lib/transform.mjs';
269
+
270
+ const EVA_SAMPLE = JSON.stringify({
271
+ systems: [
272
+ {
273
+ id: 'a', name: 'Gemini Live', type: 's2s', stt: '-', llm: 'gemini', tts: '-',
274
+ clean: {
275
+ 'EVA-A_mean': { pooled: { point: 0.30, ci_lower: 0.28, ci_upper: 0.32 } },
276
+ 'EVA-X_mean': { pooled: { point: 0.49, ci_lower: 0.47, ci_upper: 0.51 } },
277
+ },
278
+ },
279
+ {
280
+ id: 'b', name: 'Nova + GPT + Sonic', type: 'cascade', stt: 'Nova', llm: 'GPT', tts: 'Sonic',
281
+ clean: {
282
+ 'EVA-A_mean': { pooled: { point: 0.41, ci_lower: 0.39, ci_upper: 0.43 } },
283
+ 'EVA-X_mean': { pooled: { point: 0.20, ci_lower: 0.18, ci_upper: 0.22 } },
284
+ },
285
+ },
286
+ ],
287
+ });
288
+
289
+ test('parseEvaData returns parsed object with systems array', () => {
290
+ const data = parseEvaData(EVA_SAMPLE);
291
+ assert.equal(data.systems.length, 2);
292
+ });
293
+
294
+ test('deriveEvaBoard ranks by accuracy point, builds subtitle + ci + bars', () => {
295
+ const data = parseEvaData(EVA_SAMPLE);
296
+ const out = deriveEvaBoard(data.systems, 'EVA-A_mean');
297
+ assert.equal(out.metricLabel, 'Pass@1');
298
+ assert.deepEqual(out.rows.map(r => r.name), ['Nova + GPT + Sonic', 'Gemini Live']);
299
+ assert.equal(out.rows[0].score, 0.41);
300
+ assert.equal(out.rows[0].subtitle, 'Mixed Models Β· Cascade');
301
+ assert.equal(out.rows[0].bar, 100);
302
+ assert.equal(out.rows[0].ciLower, 0.39);
303
+ assert.equal(out.rows[0].ciUpper, 0.43);
304
+ assert.equal(out.rows[1].subtitle, 'Google Β· Speech-to-Speech');
305
+ assert.equal(out.rows[1].bar, 73); // 0.30/0.41 = 0.7317 β†’ 73
306
+ });
307
+
308
+ test('deriveEvaBoard ranks experience board independently', () => {
309
+ const data = parseEvaData(EVA_SAMPLE);
310
+ const out = deriveEvaBoard(data.systems, 'EVA-X_mean');
311
+ assert.deepEqual(out.rows.map(r => r.name), ['Gemini Live', 'Nova + GPT + Sonic']);
312
+ assert.equal(out.rows[0].score, 0.49);
313
+ });
314
+ ```
315
+
316
+ - [ ] **Step 2: Run test to verify it fails**
317
+
318
+ Run: `cd .claude/skills/sync-nowai-leaderboard && node --test`
319
+ Expected: FAIL β€” `parseEvaData`/`deriveEvaBoard` not exported.
320
+
321
+ - [ ] **Step 3: Write minimal implementation**
322
+
323
+ Append to `lib/transform.mjs`:
324
+
325
+ ```js
326
+ const round2 = (n) => Math.round(n * 100) / 100;
327
+
328
+ export function parseEvaData(jsonText) {
329
+ const data = JSON.parse(jsonText);
330
+ if (!Array.isArray(data.systems)) throw new Error('EVA data missing systems[] array');
331
+ return data;
332
+ }
333
+
334
+ export function deriveEvaBoard(systems, metricKey) {
335
+ const scored = systems
336
+ .map((s) => ({ s, pooled: s.clean?.[metricKey]?.pooled }))
337
+ .filter((x) => x.pooled && typeof x.pooled.point === 'number')
338
+ .sort((a, b) => b.pooled.point - a.pooled.point)
339
+ .slice(0, 5);
340
+ const top = scored[0]?.pooled.point ?? 0;
341
+ return {
342
+ metricLabel: 'Pass@1',
343
+ rows: scored.map(({ s, pooled }, i) => ({
344
+ rank: i + 1,
345
+ name: s.name,
346
+ subtitle:
347
+ s.type === 'cascade'
348
+ ? 'Mixed Models Β· Cascade'
349
+ : `${vendorFromName(s.name)} Β· Speech-to-Speech`,
350
+ score: round2(pooled.point),
351
+ ciLower: round2(pooled.ci_lower),
352
+ ciUpper: round2(pooled.ci_upper),
353
+ bar: top ? Math.round((pooled.point / top) * 100) : 0,
354
+ })),
355
+ };
356
+ }
357
+ ```
358
+
359
+ - [ ] **Step 4: Run test to verify it passes**
360
+
361
+ Run: `cd .claude/skills/sync-nowai-leaderboard && node --test`
362
+ Expected: PASS (all).
363
+
364
+ - [ ] **Step 5: Commit**
365
+
366
+ ```bash
367
+ git add .claude/skills/sync-nowai-leaderboard/lib/transform.mjs .claude/skills/sync-nowai-leaderboard/test/transform.test.mjs
368
+ git commit -m "feat: derive EVA accuracy/experience boards from leaderboardStats.json"
369
+ ```
370
+
371
+ ---
372
+
373
+ ### Task 5: Assemble full object (`buildLeaderboard`) + change diff (`diffBoards`, `hasChanges`)
374
+
375
+ **Files:**
376
+ - Modify: `.claude/skills/sync-nowai-leaderboard/test/transform.test.mjs`
377
+ - Modify: `.claude/skills/sync-nowai-leaderboard/lib/transform.mjs`
378
+
379
+ - [ ] **Step 1: Add failing tests**
380
+
381
+ Append to `test/transform.test.mjs`:
382
+
383
+ ```js
384
+ import { buildLeaderboard, hasChanges, diffBoards } from '../lib/transform.mjs';
385
+
386
+ test('buildLeaderboard assembles all three boards plus meta', () => {
387
+ const out = buildLeaderboard({
388
+ eogScript: EOG_SAMPLE,
389
+ evaJson: EVA_SAMPLE,
390
+ sources: { eog: { url: 'u1', hash: 'h1' }, eva: { url: 'u2', hash: 'h2' } },
391
+ now: '2026-06-02T00:00:00Z',
392
+ });
393
+ assert.equal(out.updated, '2026-06-02T00:00:00Z');
394
+ assert.equal(out.sources.eog.hash, 'h1');
395
+ assert.equal(out.eog.rows[0].model, 'Gemini 3.5 Flash (High)');
396
+ assert.equal(out.evaAccuracy.rows[0].name, 'Nova + GPT + Sonic');
397
+ assert.equal(out.evaExperience.rows[0].name, 'Gemini Live');
398
+ });
399
+
400
+ test('hasChanges ignores meta, detects board changes', () => {
401
+ const base = buildLeaderboard({ eogScript: EOG_SAMPLE, evaJson: EVA_SAMPLE, sources: {}, now: 'A' });
402
+ const sameDataNewMeta = buildLeaderboard({ eogScript: EOG_SAMPLE, evaJson: EVA_SAMPLE, sources: { eog: { hash: 'x' } }, now: 'B' });
403
+ assert.equal(hasChanges(base, sameDataNewMeta), false);
404
+
405
+ const changedEog = EOG_SAMPLE.replace('avg: 46.0', 'avg: 99.0');
406
+ const changed = buildLeaderboard({ eogScript: changedEog, evaJson: EVA_SAMPLE, sources: {}, now: 'A' });
407
+ assert.equal(hasChanges(base, changed), true);
408
+ });
409
+
410
+ test('diffBoards produces human-readable change lines', () => {
411
+ const base = buildLeaderboard({ eogScript: EOG_SAMPLE, evaJson: EVA_SAMPLE, sources: {}, now: 'A' });
412
+ const changedEog = EOG_SAMPLE.replace('avg: 46.0', 'avg: 41.0');
413
+ const changed = buildLeaderboard({ eogScript: changedEog, evaJson: EVA_SAMPLE, sources: {}, now: 'A' });
414
+ const lines = diffBoards(base, changed);
415
+ assert.ok(lines.some((l) => l.startsWith('eog:')));
416
+ });
417
+ ```
418
+
419
+ - [ ] **Step 2: Run test to verify it fails**
420
+
421
+ Run: `cd .claude/skills/sync-nowai-leaderboard && node --test`
422
+ Expected: FAIL β€” `buildLeaderboard`/`hasChanges`/`diffBoards` not exported.
423
+
424
+ - [ ] **Step 3: Write minimal implementation**
425
+
426
+ Append to `lib/transform.mjs`:
427
+
428
+ ```js
429
+ export function buildLeaderboard({ eogScript, evaJson, sources, now }) {
430
+ const eogArray = parseEogData(eogScript);
431
+ const evaData = parseEvaData(evaJson);
432
+ return {
433
+ updated: now,
434
+ sources,
435
+ eog: deriveEog(eogArray),
436
+ evaAccuracy: deriveEvaBoard(evaData.systems, 'EVA-A_mean'),
437
+ evaExperience: deriveEvaBoard(evaData.systems, 'EVA-X_mean'),
438
+ };
439
+ }
440
+
441
+ const BOARD_KEYS = ['eog', 'evaAccuracy', 'evaExperience'];
442
+
443
+ // Compare only the parts we render (ignore `updated`/`sources` meta).
444
+ export function hasChanges(oldData, newData) {
445
+ if (!oldData) return true;
446
+ const pick = (d) => JSON.stringify(BOARD_KEYS.map((k) => d[k]));
447
+ return pick(oldData) !== pick(newData);
448
+ }
449
+
450
+ export function diffBoards(oldData, newData) {
451
+ const label = (r) => r.model ?? r.name;
452
+ const lines = [];
453
+ for (const key of BOARD_KEYS) {
454
+ const oldRows = oldData?.[key]?.rows ?? [];
455
+ const newRows = newData[key].rows;
456
+ const oldStr = oldRows.map((r) => `${label(r)}=${r.score}`).join(', ');
457
+ const newStr = newRows.map((r) => `${label(r)}=${r.score}`).join(', ');
458
+ if (oldStr !== newStr) {
459
+ lines.push(`${key}:\n old: ${oldStr || '(none)'}\n new: ${newStr}`);
460
+ }
461
+ }
462
+ return lines;
463
+ }
464
+ ```
465
+
466
+ - [ ] **Step 4: Run test to verify it passes**
467
+
468
+ Run: `cd .claude/skills/sync-nowai-leaderboard && node --test`
469
+ Expected: PASS (all).
470
+
471
+ - [ ] **Step 5: Commit**
472
+
473
+ ```bash
474
+ git add .claude/skills/sync-nowai-leaderboard/lib/transform.mjs .claude/skills/sync-nowai-leaderboard/test/transform.test.mjs
475
+ git commit -m "feat: assemble full leaderboard object + change detection/diff"
476
+ ```
477
+
478
+ ---
479
+
480
+ ### Task 6: Sanity gates (`validate`)
481
+
482
+ **Files:**
483
+ - Modify: `.claude/skills/sync-nowai-leaderboard/test/transform.test.mjs`
484
+ - Modify: `.claude/skills/sync-nowai-leaderboard/lib/transform.mjs`
485
+
486
+ - [ ] **Step 1: Add failing tests**
487
+
488
+ Append to `test/transform.test.mjs`:
489
+
490
+ ```js
491
+ import { validate } from '../lib/transform.mjs';
492
+
493
+ function boardWith(rows) { return { metricLabel: 'x', rows }; }
494
+ function dataWith(eogRows, accRows, expRows) {
495
+ return { eog: boardWith(eogRows), evaAccuracy: boardWith(accRows), evaExperience: boardWith(expRows) };
496
+ }
497
+
498
+ test('validate passes clean data', () => {
499
+ const good = dataWith(
500
+ [{ rank: 1, model: 'A', score: 46 }, { rank: 2, model: 'B', score: 40 }, { rank: 3, model: 'C', score: 30 }],
501
+ [{ rank: 1, name: 'X', score: 0.4 }, { rank: 2, name: 'Y', score: 0.3 }, { rank: 3, name: 'Z', score: 0.2 }],
502
+ [{ rank: 1, name: 'X', score: 0.5 }, { rank: 2, name: 'Y', score: 0.4 }, { rank: 3, name: 'Z', score: 0.3 }],
503
+ );
504
+ assert.deepEqual(validate(good, null), []);
505
+ });
506
+
507
+ test('validate flags too-few rows', () => {
508
+ const bad = dataWith([{ rank: 1, model: 'A', score: 46 }], [], []);
509
+ const issues = validate(bad, null);
510
+ assert.ok(issues.some((i) => i.includes('eog') && i.includes('<3')));
511
+ });
512
+
513
+ test('validate flags out-of-range scores', () => {
514
+ const bad = dataWith(
515
+ [{ rank: 1, model: 'A', score: 150 }, { rank: 2, model: 'B', score: 40 }, { rank: 3, model: 'C', score: 30 }],
516
+ [{ rank: 1, name: 'X', score: 0.4 }, { rank: 2, name: 'Y', score: 0.3 }, { rank: 3, name: 'Z', score: 0.2 }],
517
+ [{ rank: 1, name: 'X', score: 0.5 }, { rank: 2, name: 'Y', score: 0.4 }, { rank: 3, name: 'Z', score: 0.3 }],
518
+ );
519
+ assert.ok(validate(bad, null).some((i) => i.includes('out of')));
520
+ });
521
+
522
+ test('validate flags suspicious top-score swing vs previous', () => {
523
+ const prev = dataWith(
524
+ [{ rank: 1, model: 'A', score: 46 }, { rank: 2, model: 'B', score: 40 }, { rank: 3, model: 'C', score: 30 }],
525
+ [{ rank: 1, name: 'X', score: 0.4 }, { rank: 2, name: 'Y', score: 0.3 }, { rank: 3, name: 'Z', score: 0.2 }],
526
+ [{ rank: 1, name: 'X', score: 0.5 }, { rank: 2, name: 'Y', score: 0.4 }, { rank: 3, name: 'Z', score: 0.3 }],
527
+ );
528
+ const now = dataWith(
529
+ [{ rank: 1, model: 'A', score: 10 }, { rank: 2, model: 'B', score: 8 }, { rank: 3, model: 'C', score: 6 }],
530
+ [{ rank: 1, name: 'X', score: 0.4 }, { rank: 2, name: 'Y', score: 0.3 }, { rank: 3, name: 'Z', score: 0.2 }],
531
+ [{ rank: 1, name: 'X', score: 0.5 }, { rank: 2, name: 'Y', score: 0.4 }, { rank: 3, name: 'Z', score: 0.3 }],
532
+ );
533
+ assert.ok(validate(now, prev).some((i) => i.includes('swing')));
534
+ });
535
+ ```
536
+
537
+ - [ ] **Step 2: Run test to verify it fails**
538
+
539
+ Run: `cd .claude/skills/sync-nowai-leaderboard && node --test`
540
+ Expected: FAIL β€” `validate` not exported.
541
+
542
+ - [ ] **Step 3: Write minimal implementation**
543
+
544
+ Append to `lib/transform.mjs`:
545
+
546
+ ```js
547
+ // Returns an array of human-readable issue strings; empty array = clean.
548
+ export function validate(newData, oldData) {
549
+ const issues = [];
550
+ const specs = [
551
+ ['eog', newData.eog, 0, 100, 25, oldData?.eog],
552
+ ['evaAccuracy', newData.evaAccuracy, 0, 1, 0.25, oldData?.evaAccuracy],
553
+ ['evaExperience', newData.evaExperience, 0, 1, 0.25, oldData?.evaExperience],
554
+ ];
555
+ for (const [name, board, min, max, guard, oldBoard] of specs) {
556
+ if (!board || !Array.isArray(board.rows)) {
557
+ issues.push(`${name}: missing rows`);
558
+ continue;
559
+ }
560
+ if (board.rows.length < 3) issues.push(`${name}: only ${board.rows.length} rows (<3)`);
561
+ for (const r of board.rows) {
562
+ if (r.score < min || r.score > max) {
563
+ issues.push(`${name}: score ${r.score} out of [${min},${max}]`);
564
+ }
565
+ }
566
+ const oldTop = oldBoard?.rows?.[0]?.score;
567
+ const newTop = board.rows[0]?.score;
568
+ if (typeof oldTop === 'number' && typeof newTop === 'number') {
569
+ const delta = Math.abs(newTop - oldTop);
570
+ if (delta > guard) issues.push(`${name}: top score swing ${delta.toFixed(2)} > ${guard}`);
571
+ }
572
+ }
573
+ return issues;
574
+ }
575
+ ```
576
+
577
+ - [ ] **Step 4: Run test to verify it passes**
578
+
579
+ Run: `cd .claude/skills/sync-nowai-leaderboard && node --test`
580
+ Expected: PASS (all).
581
+
582
+ - [ ] **Step 5: Commit**
583
+
584
+ ```bash
585
+ git add .claude/skills/sync-nowai-leaderboard/lib/transform.mjs .claude/skills/sync-nowai-leaderboard/test/transform.test.mjs
586
+ git commit -m "feat: add sanity-gate validation for derived leaderboard"
587
+ ```
588
+
589
+ ---
590
+
591
+ ### Task 7: Orchestrator `sync.mjs` (fetch, hash, git, notify, arg parsing)
592
+
593
+ **Files:**
594
+ - Create: `.claude/skills/sync-nowai-leaderboard/sync.mjs`
595
+
596
+ Modes: `--check` (diff only, no writes), `--apply` (write `data/leaderboard.json` + git commit, NO push), `--push` (implies apply + push to `origin`/staging). Default is `--check`. Push is opt-in so dev runs never deploy.
597
+
598
+ - [ ] **Step 1: Create sync.mjs**
599
+
600
+ ```js
601
+ #!/usr/bin/env node
602
+ // Orchestrator for the leaderboard sync. Side effects only; pure logic in lib/transform.mjs.
603
+ import { readFile, writeFile } from 'node:fs/promises';
604
+ import { existsSync } from 'node:fs';
605
+ import { createHash } from 'node:crypto';
606
+ import { execFileSync } from 'node:child_process';
607
+ import { join } from 'node:path';
608
+ import { buildLeaderboard, hasChanges, diffBoards, validate } from './lib/transform.mjs';
609
+
610
+ const EOG_URL = 'https://raw.githubusercontent.com/EnterpriseOps-Gym/EnterpriseOps-Gym.github.io/main/script.js';
611
+ const EVA_URL = 'https://raw.githubusercontent.com/ServiceNow/eva/main/website/src/data/leaderboardStats.json';
612
+
613
+ function repoRoot() {
614
+ return execFileSync('git', ['rev-parse', '--show-toplevel'], { encoding: 'utf8' }).trim();
615
+ }
616
+
617
+ function sha256(text) {
618
+ return 'sha256:' + createHash('sha256').update(text).digest('hex');
619
+ }
620
+
621
+ async function fetchText(url) {
622
+ const res = await fetch(url);
623
+ if (!res.ok) throw new Error(`fetch ${url} β†’ HTTP ${res.status}`);
624
+ return res.text();
625
+ }
626
+
627
+ async function notify(message) {
628
+ const url = process.env.TEAMS_WEBHOOK_URL;
629
+ if (!url) {
630
+ console.log('[notify] (no TEAMS_WEBHOOK_URL set) ' + message);
631
+ return;
632
+ }
633
+ try {
634
+ const res = await fetch(url, {
635
+ method: 'POST',
636
+ headers: { 'Content-Type': 'application/json' },
637
+ body: JSON.stringify({ text: message }),
638
+ });
639
+ if (!res.ok) console.error(`[notify] webhook HTTP ${res.status}`);
640
+ } catch (e) {
641
+ console.error('[notify] webhook failed: ' + e.message);
642
+ }
643
+ }
644
+
645
+ function git(root, args) {
646
+ return execFileSync('git', args, { cwd: root, encoding: 'utf8' });
647
+ }
648
+
649
+ async function main() {
650
+ const args = new Set(process.argv.slice(2));
651
+ const mode = args.has('--push') ? 'push' : args.has('--apply') ? 'apply' : 'check';
652
+
653
+ const root = repoRoot();
654
+ const dataPath = join(root, 'data', 'leaderboard.json');
655
+
656
+ // 1. Fetch upstream.
657
+ let eogScript, evaJson;
658
+ try {
659
+ [eogScript, evaJson] = await Promise.all([fetchText(EOG_URL), fetchText(EVA_URL)]);
660
+ } catch (e) {
661
+ console.error('[sync] fetch failed: ' + e.message);
662
+ await notify('Leaderboard sync: upstream fetch failed β€” ' + e.message);
663
+ process.exit(1);
664
+ }
665
+
666
+ // 2. Build new data.
667
+ let newData;
668
+ try {
669
+ newData = buildLeaderboard({
670
+ eogScript,
671
+ evaJson,
672
+ sources: {
673
+ eog: { url: EOG_URL, hash: sha256(eogScript), fetchedAt: new Date().toISOString() },
674
+ eva: { url: EVA_URL, hash: sha256(evaJson), fetchedAt: new Date().toISOString() },
675
+ },
676
+ now: new Date().toISOString(),
677
+ });
678
+ } catch (e) {
679
+ console.error('[sync] parse/derive failed: ' + e.message);
680
+ await notify('Leaderboard sync: parse failed (schema drift?) β€” ' + e.message);
681
+ process.exit(1);
682
+ }
683
+
684
+ // 3. Load existing.
685
+ const oldData = existsSync(dataPath)
686
+ ? JSON.parse(await readFile(dataPath, 'utf8'))
687
+ : null;
688
+
689
+ // 4. Change detection.
690
+ if (!hasChanges(oldData, newData)) {
691
+ console.log('[sync] no display changes. Done.');
692
+ return;
693
+ }
694
+ const changeLines = diffBoards(oldData, newData);
695
+ console.log('[sync] changes detected:\n' + changeLines.join('\n'));
696
+
697
+ // 5. Sanity gates.
698
+ const issues = validate(newData, oldData);
699
+ if (issues.length) {
700
+ console.error('[sync] sanity gates failed:\n - ' + issues.join('\n - '));
701
+ await notify('Leaderboard sync ABORTED β€” sanity gates failed:\n' + issues.join('\n'));
702
+ process.exit(2);
703
+ }
704
+
705
+ if (mode === 'check') {
706
+ console.log('[sync] --check: no writes. Re-run with --apply (commit) or --push (commit + staging).');
707
+ return;
708
+ }
709
+
710
+ // 6. Write + commit.
711
+ await writeFile(dataPath, JSON.stringify(newData, null, 2) + '\n');
712
+ git(root, ['add', dataPath]);
713
+ const today = new Date().toISOString().slice(0, 10);
714
+ const msg = `Sync leaderboard: EOG + EVA ${today}\n\n${changeLines.join('\n')}`;
715
+ git(root, ['commit', '-m', msg]);
716
+ console.log('[sync] committed.');
717
+
718
+ // 7. Push (staging only) if requested.
719
+ if (mode === 'push') {
720
+ try {
721
+ git(root, ['push', 'origin', 'HEAD:main']);
722
+ console.log('[sync] pushed to origin (staging).');
723
+ await notify(`Leaderboard synced to staging (${changeLines.length} board change(s)). Review + promote to prod.`);
724
+ } catch (e) {
725
+ console.error('[sync] push failed: ' + e.message);
726
+ await notify('Leaderboard sync: commit OK but push failed β€” ' + e.message);
727
+ process.exit(3);
728
+ }
729
+ } else {
730
+ console.log('[sync] --apply: committed locally, not pushed. Use --push to deploy to staging.');
731
+ }
732
+ }
733
+
734
+ main().catch((e) => {
735
+ console.error('[sync] unexpected error: ' + e.message);
736
+ process.exit(1);
737
+ });
738
+ ```
739
+
740
+ - [ ] **Step 2: Run check mode against live upstream (smoke test, no writes)**
741
+
742
+ Run: `cd .claude/skills/sync-nowai-leaderboard && node sync.mjs --check`
743
+ Expected: prints `[sync] changes detected:` with EOG/EVA lines (since no `data/leaderboard.json` exists yet), then `--check: no writes`. Exit 0. No files changed (`git status` clean except untracked plan/docs).
744
+
745
+ - [ ] **Step 3: Commit**
746
+
747
+ ```bash
748
+ git add .claude/skills/sync-nowai-leaderboard/sync.mjs
749
+ git commit -m "feat: sync orchestrator with check/apply/push modes + notify"
750
+ ```
751
+
752
+ ---
753
+
754
+ ### Task 8: Generate initial `data/leaderboard.json` from live upstream
755
+
756
+ **Files:**
757
+ - Create: `data/leaderboard.json` (generated)
758
+
759
+ - [ ] **Step 1: Generate + commit locally (NO push)**
760
+
761
+ Run: `cd .claude/skills/sync-nowai-leaderboard && node sync.mjs --apply`
762
+ Expected: `changes detected` β†’ sanity gates pass β†’ `committed.` β†’ `--apply: committed locally, not pushed.` Exit 0.
763
+
764
+ - [ ] **Step 2: Inspect the generated file**
765
+
766
+ Run: `cat data/leaderboard.json` (from repo root)
767
+ Expected: valid JSON with `eog.rows` (5 entries, top = Gemini 3.5 Flash ~46), `evaAccuracy.rows`, `evaExperience.rows` (5 each), `sources` with sha256 hashes, `updated` timestamp.
768
+
769
+ Verify ranges: EOG scores 0–100, EVA scores 0–1. If anything looks malformed, STOP β€” the parser needs fixing before proceeding (revisit Task 2/4).
770
+
771
+ - [ ] **Step 3: (commit already made by --apply in Step 1 β€” verify)**
772
+
773
+ Run: `git log --oneline -1`
774
+ Expected: top commit is the `Sync leaderboard:` commit created by the script.
775
+
776
+ ---
777
+
778
+ ### Task 9: Page render β€” id hooks + inline fetch/fallback script
779
+
780
+ **Files:**
781
+ - Modify: `index.html`
782
+
783
+ The three cards keep their current static rows as the never-empty fallback. We add `id` hooks to the dynamic regions and an inline script that overlays the fetched JSON on load.
784
+
785
+ - [ ] **Step 1: Add id hooks to the EOG card**
786
+
787
+ In `index.html`, the EOG top-model block currently reads:
788
+
789
+ ```html
790
+ <div class="text-infinite-blue font-bold text-xl leading-tight">Claude Opus 4.5</div>
791
+ <div class="text-on-surface-variant text-sm font-medium">Anthropic</div>
792
+ ```
793
+
794
+ Replace with:
795
+
796
+ ```html
797
+ <div id="eog-top-name" class="text-infinite-blue font-bold text-xl leading-tight">Claude Opus 4.5</div>
798
+ <div id="eog-top-org" class="text-on-surface-variant text-sm font-medium">Anthropic</div>
799
+ ```
800
+
801
+ The EOG score span currently reads:
802
+
803
+ ```html
804
+ <span class="text-infinite-blue font-bold text-3xl tabular-nums leading-none">37.4<span class="text-on-surface-variant text-lg font-medium ml-0.5">%</span></span>
805
+ ```
806
+
807
+ Replace with (add id on the outer span):
808
+
809
+ ```html
810
+ <span id="eog-top-score" class="text-infinite-blue font-bold text-3xl tabular-nums leading-none">37.4<span class="text-on-surface-variant text-lg font-medium ml-0.5">%</span></span>
811
+ ```
812
+
813
+ The EOG runners wrapper currently reads `<div class="space-y-4">` (the one immediately containing `<!-- Runner 2 -->`). Replace its opening tag with:
814
+
815
+ ```html
816
+ <div id="eog-runners" class="space-y-4">
817
+ ```
818
+
819
+ - [ ] **Step 2: Add id hooks to the EVA-Accuracy split**
820
+
821
+ The accuracy top block currently reads:
822
+
823
+ ```html
824
+ <div class="text-infinite-blue font-bold text-base leading-tight">Nova + GPT-5.4 + Sonic</div>
825
+ <div class="text-on-surface-variant text-[11px] font-medium">Mixed Models Β· Cascade</div>
826
+ ```
827
+
828
+ Replace with:
829
+
830
+ ```html
831
+ <div id="eva-acc-name" class="text-infinite-blue font-bold text-base leading-tight">Nova + GPT-5.4 + Sonic</div>
832
+ <div id="eva-acc-sub" class="text-on-surface-variant text-[11px] font-medium">Mixed Models Β· Cascade</div>
833
+ ```
834
+
835
+ The accuracy top score currently reads:
836
+
837
+ ```html
838
+ <span class="text-infinite-blue font-bold text-2xl tabular-nums">0.41</span>
839
+ ```
840
+
841
+ Replace with:
842
+
843
+ ```html
844
+ <span id="eva-acc-score" class="text-infinite-blue font-bold text-2xl tabular-nums">0.41</span>
845
+ ```
846
+
847
+ The accuracy runners wrapper currently reads `<div class="space-y-2.5">` (the one whose first child row contains `Claude Opus 4.5` at `0.32`). Replace its opening tag with:
848
+
849
+ ```html
850
+ <div id="eva-acc-runners" class="space-y-2.5">
851
+ ```
852
+
853
+ - [ ] **Step 3: Add id hooks to the EVA-Experience split**
854
+
855
+ The experience top block currently reads:
856
+
857
+ ```html
858
+ <div class="text-infinite-blue font-bold text-base leading-tight">Gemini Live</div>
859
+ <div class="text-on-surface-variant text-[11px] font-medium">Google Β· Speech-to-Speech</div>
860
+ ```
861
+
862
+ Replace with:
863
+
864
+ ```html
865
+ <div id="eva-exp-name" class="text-infinite-blue font-bold text-base leading-tight">Gemini Live</div>
866
+ <div id="eva-exp-sub" class="text-on-surface-variant text-[11px] font-medium">Google Β· Speech-to-Speech</div>
867
+ ```
868
+
869
+ The experience top score currently reads:
870
+
871
+ ```html
872
+ <span class="text-infinite-blue font-bold text-2xl tabular-nums">0.49</span>
873
+ ```
874
+
875
+ Replace with:
876
+
877
+ ```html
878
+ <span id="eva-exp-score" class="text-infinite-blue font-bold text-2xl tabular-nums">0.49</span>
879
+ ```
880
+
881
+ The experience runners wrapper currently reads `<div class="space-y-2.5">` (the one whose first child row contains `GPT-Realtime` at `0.47`). Replace its opening tag with:
882
+
883
+ ```html
884
+ <div id="eva-exp-runners" class="space-y-2.5">
885
+ ```
886
+
887
+ - [ ] **Step 4: Add the inline render script before `</body>`**
888
+
889
+ Find the closing `</body>` tag in `index.html` and insert this block immediately before it:
890
+
891
+ ```html
892
+ <script>
893
+ // Overlay synced leaderboard data on the static fallback markup.
894
+ // On any failure, the static markup already in the DOM remains.
895
+ (function () {
896
+ function esc(s) {
897
+ return String(s).replace(/[&<>"']/g, (c) => ({ '&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;' }[c]));
898
+ }
899
+
900
+ function eogRunnerRow(r) {
901
+ return `<div class="grid grid-cols-[20px_1fr_100px_45px] items-center gap-3">
902
+ <span class="text-on-surface-variant text-xs tabular-nums font-medium">${r.rank}</span>
903
+ <span class="text-on-surface text-sm font-medium">${esc(r.model)}</span>
904
+ <div class="h-1.5 w-full bg-infinite-soft rounded-full overflow-hidden">
905
+ <div class="h-full fill-infinite rounded-full" style="width: ${r.bar}%;"></div>
906
+ </div>
907
+ <span class="text-infinite-blue text-sm font-bold text-right tabular-nums">${r.score.toFixed(1)}</span>
908
+ </div>`;
909
+ }
910
+
911
+ function evaRunnerRow(r, fill) {
912
+ return `<div class="grid grid-cols-[18px_1fr_80px_35px] items-center gap-2 text-xs">
913
+ <span class="text-on-surface-variant tabular-nums font-medium">${r.rank}</span>
914
+ <span class="text-on-surface font-medium truncate">${esc(r.name)}</span>
915
+ <div class="h-1 w-full bg-${fill}-soft rounded-full overflow-hidden">
916
+ <div class="h-full fill-${fill} rounded-full" style="width: ${r.bar}%;"></div>
917
+ </div>
918
+ <span class="text-infinite-blue font-bold text-right tabular-nums">${r.score.toFixed(2)}</span>
919
+ </div>`;
920
+ }
921
+
922
+ function setText(id, value) {
923
+ const el = document.getElementById(id);
924
+ if (el) el.textContent = value;
925
+ }
926
+
927
+ function renderEog(board) {
928
+ const top = board.rows[0];
929
+ if (!top) return;
930
+ setText('eog-top-name', top.model);
931
+ setText('eog-top-org', top.org);
932
+ const score = document.getElementById('eog-top-score');
933
+ if (score) score.innerHTML = `${top.score.toFixed(1)}<span class="text-on-surface-variant text-lg font-medium ml-0.5">%</span>`;
934
+ const runners = document.getElementById('eog-runners');
935
+ if (runners) runners.innerHTML = board.rows.slice(1).map(eogRunnerRow).join('');
936
+ }
937
+
938
+ function renderEvaSplit(board, prefix, fill) {
939
+ const top = board.rows[0];
940
+ if (!top) return;
941
+ setText(`${prefix}-name`, top.name);
942
+ setText(`${prefix}-sub`, top.subtitle);
943
+ setText(`${prefix}-score`, top.score.toFixed(2));
944
+ const runners = document.getElementById(`${prefix}-runners`);
945
+ if (runners) runners.innerHTML = board.rows.slice(1).map((r) => evaRunnerRow(r, fill)).join('');
946
+ }
947
+
948
+ fetch('./data/leaderboard.json', { cache: 'no-cache' })
949
+ .then((res) => { if (!res.ok) throw new Error('HTTP ' + res.status); return res.json(); })
950
+ .then((data) => {
951
+ if (data.eog) renderEog(data.eog);
952
+ if (data.evaAccuracy) renderEvaSplit(data.evaAccuracy, 'eva-acc', 'indigo');
953
+ if (data.evaExperience) renderEvaSplit(data.evaExperience, 'eva-exp', 'blue');
954
+ })
955
+ .catch((err) => {
956
+ console.warn('Leaderboard JSON load failed; showing static fallback.', err);
957
+ });
958
+ })();
959
+ </script>
960
+ ```
961
+
962
+ - [ ] **Step 5: Commit**
963
+
964
+ ```bash
965
+ git add index.html
966
+ git commit -m "feat: render leaderboards from data/leaderboard.json with static fallback"
967
+ ```
968
+
969
+ ---
970
+
971
+ ### Task 10: Verify render in browser
972
+
973
+ **Files:** none (verification only)
974
+
975
+ - [ ] **Step 1: Serve the site locally**
976
+
977
+ The user runs a local web server (IntelliJ) at the project root. Confirm the page is reachable (e.g. `http://localhost:63342/NowAI-Bench/index.html` or the user's configured URL). If not running, start one: `cd <repo root> && python3 -m http.server 8080` and use `http://localhost:8080/`.
978
+
979
+ - [ ] **Step 2: Load the page and assert via Chrome DevTools MCP**
980
+
981
+ Navigate to the page. Using `mcp__plugin_chrome-devtools-mcp_chrome-devtools__evaluate_script`, run:
982
+
983
+ ```js
984
+ () => ({
985
+ eogRunners: document.querySelectorAll('#eog-runners > div').length,
986
+ accRunners: document.querySelectorAll('#eva-acc-runners > div').length,
987
+ expRunners: document.querySelectorAll('#eva-exp-runners > div').length,
988
+ eogTop: document.getElementById('eog-top-name').textContent,
989
+ accTop: document.getElementById('eva-acc-name').textContent,
990
+ expTop: document.getElementById('eva-exp-name').textContent,
991
+ })
992
+ ```
993
+
994
+ Expected: each `*Runners` count is `4` (ranks 2–5; top is rendered separately), totaling 5 per board. `eogTop` reflects the synced #1 (e.g. "Gemini 3.5 Flash (High)"), not the old static "Claude Opus 4.5".
995
+
996
+ - [ ] **Step 3: Check console for errors**
997
+
998
+ Use `mcp__plugin_chrome-devtools-mcp_chrome-devtools__list_console_messages`.
999
+ Expected: no errors. (A "JSON load failed" warning means the server isn't serving `data/leaderboard.json` at the expected relative path β€” fix the path/serving before continuing.)
1000
+
1001
+ - [ ] **Step 4: Screenshot for visual confirmation**
1002
+
1003
+ Use `mcp__plugin_chrome-devtools-mcp_chrome-devtools__take_screenshot` and confirm the three cards show 5 rows each with bars, matching the prior styling.
1004
+
1005
+ ---
1006
+
1007
+ ### Task 11: Write `SKILL.md`
1008
+
1009
+ **Files:**
1010
+ - Create: `.claude/skills/sync-nowai-leaderboard/SKILL.md`
1011
+
1012
+ - [ ] **Step 1: Create SKILL.md**
1013
+
1014
+ ```markdown
1015
+ ---
1016
+ name: sync-nowai-leaderboard
1017
+ description: Use when syncing the NowAI-Bench landing-page leaderboards from upstream EOG + EVA sources, or when a scheduled run fires to check for new results. Pulls both sources, re-derives data/leaderboard.json, and pushes to staging only on sane changes.
1018
+ ---
1019
+
1020
+ # Sync NowAI-Bench Leaderboard
1021
+
1022
+ Pulls the two upstream leaderboard sources, re-derives `data/leaderboard.json`,
1023
+ and (when run with `--push`) deploys the change to the **staging** HF Space only.
1024
+
1025
+ ## Sources
1026
+ - EOG: `https://raw.githubusercontent.com/EnterpriseOps-Gym/EnterpriseOps-Gym.github.io/main/script.js` (`LEADERBOARD_DATA` array)
1027
+ - EVA: `https://raw.githubusercontent.com/ServiceNow/eva/main/website/src/data/leaderboardStats.json` (`systems[]`)
1028
+
1029
+ ## How to run
1030
+
1031
+ From the repo root:
1032
+
1033
+ 1. **Preview (always do this first):**
1034
+ `node .claude/skills/sync-nowai-leaderboard/sync.mjs --check`
1035
+ Prints the diff vs the committed `data/leaderboard.json`. No writes.
1036
+
1037
+ 2. **Inspect the printed diff.** If it looks sane (no garbled names, scores in
1038
+ range, no wild swings), proceed. If sanity gates already flagged issues, the
1039
+ script aborts β€” investigate upstream before forcing anything.
1040
+
1041
+ 3. **Apply + deploy to staging:**
1042
+ `node .claude/skills/sync-nowai-leaderboard/sync.mjs --push`
1043
+ Writes the file, commits, and pushes to `origin` (staging). Notifies via
1044
+ Teams webhook if `TEAMS_WEBHOOK_URL` is set.
1045
+
1046
+ To commit locally without deploying, use `--apply` instead of `--push`.
1047
+
1048
+ ## Guardrails (NON-NEGOTIABLE)
1049
+ - **Push to `origin` (staging) ONLY. NEVER push to `prod`.** Promotion to prod is
1050
+ a human decision made after reviewing staging.
1051
+ - The script exits non-zero and notifies (does not commit) if: a fetch fails, a
1052
+ source fails to parse, any board has <3 rows, any score is out of range
1053
+ (EOG 0–100, EVA 0–1), or the top score swings >25 (EOG) / >0.25 (EVA) vs the
1054
+ committed data. Do not override these by editing the JSON by hand to bypass.
1055
+
1056
+ ## Tests
1057
+ `cd .claude/skills/sync-nowai-leaderboard && node --test`
1058
+
1059
+ ## Open TBD
1060
+ - Teams webhook URL not yet provisioned. Until then, notifications print to the
1061
+ run log. Set `TEAMS_WEBHOOK_URL` (as a secret/env var, never committed) to enable.
1062
+ ```
1063
+
1064
+ - [ ] **Step 2: Verify the skill is discoverable**
1065
+
1066
+ Run: `git check-ignore .claude/skills/sync-nowai-leaderboard/SKILL.md`
1067
+ Expected: **no output** (file is NOT ignored β€” the `!.claude/skills/` rule keeps it tracked). If it prints the path, the `.gitignore` negation is wrong β€” fix before committing.
1068
+
1069
+ - [ ] **Step 3: Commit**
1070
+
1071
+ ```bash
1072
+ git add .claude/skills/sync-nowai-leaderboard/SKILL.md
1073
+ git commit -m "docs: add SKILL.md for sync-nowai-leaderboard"
1074
+ ```
1075
+
1076
+ ---
1077
+
1078
+ ### Task 12: Schedule the daily run
1079
+
1080
+ **Files:** none (creates a user-level Claude Code cron, not a repo file)
1081
+
1082
+ - [ ] **Step 1: Create the daily schedule**
1083
+
1084
+ Use the `CronCreate` tool (load its schema via ToolSearch: `select:CronCreate`). Create a daily job whose prompt instructs Claude to run the skill end-to-end:
1085
+
1086
+ - Schedule: daily (e.g. `0 14 * * *` β€” 14:00 UTC β‰ˆ 7am PT; adjust to preference).
1087
+ - Prompt (verbatim intent): "Invoke the sync-nowai-leaderboard skill in the NowAI-Bench repo. Run `node .claude/skills/sync-nowai-leaderboard/sync.mjs --check` first; if it reports sane changes and no gate failures, run it again with `--push` to deploy to staging. Report the diff summary. Never touch prod."
1088
+
1089
+ - [ ] **Step 2: Confirm it registered**
1090
+
1091
+ Use the `CronList` tool (`select:CronList`). Expected: the new daily job appears.
1092
+
1093
+ Note: the schedule lives in user-level Claude Code config, not the repo. It must be re-created on any machine that should run the sync.
1094
+
1095
+ ---
1096
+
1097
+ ## Self-Review
1098
+
1099
+ **Spec coverage:**
1100
+ - Page refactor to `data.json` + inline render + fallback β†’ Tasks 8, 9, 10. βœ“
1101
+ - Sync skill in `.claude/skills/` β†’ Tasks 1, 7, 11. βœ“
1102
+ - Transform/mapping rules (EOG avg sort, EVA A/X boards, subtitles, vendor map, bars, CI) β†’ Tasks 2–5. βœ“
1103
+ - Compare-derived-output change detection β†’ Task 5 (`hasChanges`). βœ“
1104
+ - Sanity gates (HTTP, β‰₯3 rows, range, Ξ” guard) β†’ Task 6 + wired in Task 7. βœ“
1105
+ - Staging-only push, never prod β†’ Task 7 (`--push` β†’ `origin` only), Task 11 guardrails. βœ“
1106
+ - Provenance hashes in `sources` β†’ Task 7 (`sha256`). βœ“
1107
+ - Teams webhook notify (TBD) + fallback β†’ Task 7 `notify()`, Task 11 Open TBD. βœ“
1108
+ - Daily cadence via CronCreate β†’ Task 12. βœ“
1109
+ - `.claude/skills/` tracked, rest ignored β†’ verified in Task 11 Step 2. βœ“
1110
+ - Tests (transform unit + render check) β†’ Tasks 2–6 (unit), Task 10 (render). βœ“
1111
+
1112
+ **Refinement noted vs spec:** spec described `--apply` as write+commit+push. Plan splits this: `--apply` = write+commit (no push), `--push` = write+commit+push to staging. Push becomes explicit opt-in so dev/init runs (Task 8) cannot accidentally deploy. Safer; same end behavior for the cron (`--push`).
1113
+
1114
+ **Placeholder scan:** no TBD/TODO in code steps; the only "TBD" is the Teams webhook, which is a real external dependency documented as such. βœ“
1115
+
1116
+ **Type consistency:** `parseEogData`/`deriveEog`/`parseEvaData`/`deriveEvaBoard`/`buildLeaderboard`/`hasChanges`/`diffBoards`/`validate`/`vendorFromName` names match across lib, tests, and `sync.mjs` imports. EOG rows expose `{rank,model,org,type,score,bar}`; EVA rows expose `{rank,name,subtitle,score,ciLower,ciUpper,bar}` β€” consistent between derive functions, the render script, and `diffBoards` (`label = r.model ?? r.name`). βœ“
1117
+
1118
+ ---
1119
+
1120
+ ## Out of Scope (per spec)
1121
+ - Auto-promotion to prod.
1122
+ - Per-domain / radar view (v1.1).
1123
+ - Historical trend tracking.