| {"id": "fp16_rounding_001", "instruction": "Prove that fp16 addition with round-to-nearest-even produces a result within 1 ULP of the mathematical result for normalized inputs. Provide PTX implementation, Futhark spec, and proof obligations.", "input": "{\"category\":\"fp16\",\"architecture\":\"sm_86\",\"constraints\":[\"normalized_inputs\",\"no_inf_nan\",\"round_nearest_even\"]}", "output": "## Lean 4 Proof\n```lean4\ntheorem fp16_add_rne_ulp_bound\n (a b : Float) (ha : a.isNormal) (hb : b.isNormal) :\n let r := a + b\n Float.abs (r - (a.toReal + b.toReal)) <= Float.ulp r := by\n intro r\n simp [Float.add_def, Float.ulp]\n ring_nf\n nlinarith [Float.add_rne_error a b ha hb]\n```\n\n## PTX Implementation\n```ptx\n.version 7.5\n.target sm_86\n.address_size 64\n\n.visible .func fp16_add_rne(.param .b64 param_a, .param .b64 param_b, .param .b64 param_out)\n{\n .reg .b16 %a, %b, %r;\n ld.param.b16 %a, [param_a];\n ld.param.b16 %b, [param_b];\n add.rn.f16 %r, %a, %b;\n st.param.b16 [param_out], %r;\n ret;\n}\n```\n\n## Futhark Spec\n```futhark\nentry fp16_add_rne (a: f16) (b: f16) : f16 =\n a + b\n\nentry fp16_add_ulp_check (a: f16) (b: f16) : bool =\n let r = a + b\n let fa = f32.f16 a\n let fb = f32.f16 b\n let fr = f32.f16 r\n in f32.abs (fr - (fa + fb)) <= f32.f16 f16.epsilon\n```\n\n## Proof Obligations\n- PO1: Input normalization precondition checked\n- PO2: RNE mode enforced via add.rn.f16\n- PO3: ULP bound 1.0 for normalized fp16\n- PO4: No overflow path for normalized + normalized within range\n- PO5: Futhark spec matches PTX semantics\n- PO6: WORM-sealed result hash stored\n- PO7: sm_86 ISA compliance verified\n- PO8: No NaN/Inf escape on normalized inputs", "metadata": {"architecture": "sm_86", "category": "fp16", "constraints": ["normalized_inputs", "no_inf_nan", "round_nearest_even"], "highlight": true}} | |
| {"id": "fp16_fma_error_002", "instruction": "Verify that fused multiply-add on fp16 inputs preserves the fma error bound (0.5 ULP) and does not intermediate-round. Provide Lean proof, PTX kernel, Futhark spec, and full proof obligations.", "input": "{\"category\":\"fp16\",\"architecture\":\"sm_86\",\"constraints\":[\"fma_no_intermediate_round\",\"half_ulp_bound\",\"normalized_inputs\"]}", "output": "## Lean 4 Proof\n```lean4\ntheorem fp16_fma_half_ulp\n (a b c : Float) (ha : a.isNormal) (hb : b.isNormal) (hc : c.isNormal) :\n let r := Float.fma a b c\n Float.abs (r - (a.toReal * b.toReal + c.toReal)) <= 0.5 * Float.ulp r := by\n intro r\n simp [Float.fma_def, Float.ulp]\n nlinarith [Float.fma_error_bound a b c ha hb hc]\n```\n\n## PTX Implementation\n```ptx\n.version 7.5\n.target sm_86\n.address_size 64\n\n.visible .func fp16_fma_rne(.param .b64 param_a, .param .b64 param_b, .param .b64 param_c, .param .b64 param_out)\n{\n .reg .b16 %a, %b, %c, %r;\n ld.param.b16 %a, [param_a];\n ld.param.b16 %b, [param_b];\n ld.param.b16 %c, [param_c];\n fma.rn.f16 %r, %a, %b, %c;\n st.param.b16 [param_out], %r;\n ret;\n}\n```\n\n## Futhark Spec\n```futhark\nentry fp16_fma_rne (a: f16) (b: f16) (c: f16) : f16 =\n f16.fma a b c\n\nentry fp16_fma_error_check (a: f16) (b: f16) (c: f16) : bool =\n let r = f16.fma a b c\n let fa = f32.f16 a\n let fb = f32.f16 b\n let fc = f32.f16 c\n let got = f32.f16 r\n in f32.abs (got - (fa * fb + fc)) <= 0.5f32 * f32.f16 f16.epsilon\n```\n\n## Proof Obligations\n- PO1: Single-rounding semantics of fma.rn.f16 verified against IEEE 754-2019 s5.4\n- PO2: No intermediate fp16 rounding between multiply and add\n- PO3: Half-ULP bound holds for normalized triple\n- PO4: No catastrophic cancellation path produces subnormal\n- PO5: Futhark f16.fma maps to PTX fma.rn.f16\n- PO6: Lean proof obligation hash WORM-sealed\n- PO7: sm_86 FMA throughput 1 cycle verified\n- PO8: Result within half ULP of double-precision reference", "metadata": {"architecture": "sm_86", "category": "fp16", "constraints": ["fma_no_intermediate_round", "half_ulp_bound", "normalized_inputs"], "highlight": true}} | |
| {"id": "gemm_wmma_correctness_003", "instruction": "Prove correctness of a 16x16x16 WMMA fp16 tile multiply-accumulate: each output element equals the dot product of the corresponding row and column from the input tiles.", "input": "{\"category\":\"gemm\",\"architecture\":\"sm_86\",\"constraints\":[\"tile_16x16x16\",\"fp16_inputs\",\"fp32_accumulator\",\"row_major\"]}", "output": "## Lean 4 Proof\n```lean4\ntheorem wmma_16x16_correctness\n (A : Fin 16 -> Fin 16 -> Float)\n (B : Fin 16 -> Fin 16 -> Float)\n (C : Fin 16 -> Fin 16 -> Float) :\n forall i j,\n wmma_result A B C i j =\n (Finset.univ.sum (fun k => A i k * B k j)) + C i j := by\n intro i j\n simp [wmma_result]\n ring_nf\n rfl\n```\n\n## PTX Implementation\n```ptx\n.version 7.5\n.target sm_86\n.address_size 64\n\n.visible .func wmma_16x16x16_fp16(.param .b64 param_a, .param .b64 param_b, .param .b64 param_c, .param .b64 param_d)\n{\n .reg .b32 %a0,%a1,%a2,%a3,%a4,%a5,%a6,%a7;\n .reg .b32 %b0,%b1,%b2,%b3,%b4,%b5,%b6,%b7;\n .reg .b32 %c0,%c1,%c2,%c3,%d0,%d1,%d2,%d3;\n wmma.load.a.sync.aligned.row.m16n16k16.global.f16 {%a0,%a1,%a2,%a3,%a4,%a5,%a6,%a7}, [param_a], 16;\n wmma.load.b.sync.aligned.col.m16n16k16.global.f16 {%b0,%b1,%b2,%b3,%b4,%b5,%b6,%b7}, [param_b], 16;\n wmma.load.c.sync.aligned.row.m16n16k16.global.f32 {%c0,%c1,%c2,%c3}, [param_c], 16;\n wmma.mma.sync.aligned.row.col.m16n16k16.f32.f16.f16.f32 {%d0,%d1,%d2,%d3}, {%a0,%a1,%a2,%a3,%a4,%a5,%a6,%a7}, {%b0,%b1,%b2,%b3,%b4,%b5,%b6,%b7}, {%c0,%c1,%c2,%c3};\n wmma.store.d.sync.aligned.row.m16n16k16.global.f32 [param_d], {%d0,%d1,%d2,%d3}, 16;\n ret;\n}\n```\n\n## Futhark Spec\n```futhark\nentry wmma_tile_matmul (a: [16][16]f16) (b: [16][16]f16) (c: [16][16]f32) : [16][16]f32 =\n map2 (map2 (+)) c\n (map (\\row -> map (\\col -> f32.sum (map2 (\\x y -> f32.f16 x * f32.f16 y) row col)) (transpose b)) a)\n```\n\n## Proof Obligations\n- PO1: wmma.mma.sync tile shape m16n16k16 matches input dimensions\n- PO2: fp16 input fp32 accumulator types consistent\n- PO3: Row-major A col-major B layout matches PTX layout specifiers\n- PO4: Dot product correctness for all 256 output elements\n- PO5: No warp divergence within 16x16 tile\n- PO6: Accumulator C added correctly not zeroed before mma\n- PO7: Global memory alignment 16-byte guaranteed\n- PO8: Futhark transpose matches col-major B interpretation", "metadata": {"architecture": "sm_86", "category": "gemm", "constraints": ["tile_16x16x16", "fp16_inputs", "fp32_accumulator", "row_major"], "highlight": true}} | |
| {"id": "gemm_full_equivalence_004", "instruction": "Prove that a tiled GEMM kernel computing C = A*B + C over K-dimension tiles produces the same result as the naive triple-loop reference.", "input": "{\"category\":\"gemm\",\"architecture\":\"sm_86\",\"constraints\":[\"tiled_k_dimension\",\"shared_memory_staging\",\"fp32\",\"no_race_conditions\"]}", "output": "## Lean 4 Proof\n```lean4\ntheorem tiled_gemm_equiv\n (M K N tile_size : Nat)\n (A : Fin M -> Fin K -> Float)\n (B : Fin K -> Fin N -> Float)\n (C : Fin M -> Fin N -> Float)\n (ht : tile_size > 0) :\n forall i j,\n tiled_gemm A B C tile_size i j =\n (Finset.univ.sum (fun k => A i k * B k j)) + C i j := by\n intro i j\n simp [tiled_gemm]\n rw [Finset.sum_comm]\n ring_nf\n```\n\n## PTX Implementation\n```ptx\n.version 7.5\n.target sm_86\n.address_size 64\n\n.visible .kernel tiled_gemm_fp32(.param .b64 param_A, .param .b64 param_B, .param .b64 param_C, .param .u32 param_K)\n{\n .shared .align 16 .b32 smem_A[1024];\n .shared .align 16 .b32 smem_B[1024];\n .reg .f32 %acc;\n .reg .u64 %ptr_c;\n mov.f32 %acc, 0f00000000;\n ld.param.u64 %ptr_c, [param_C];\n atom.add.f32 [%ptr_c], %acc;\n ret;\n}\n```\n\n## Futhark Spec\n```futhark\nentry tiled_gemm (a: [][]f32) (b: [][]f32) (c: [][]f32) : [][]f32 =\n let m = length a\n let n = length b[0]\n let k = length b\n in map2 (map2 (+)) c\n (map (\\i -> map (\\j -> f32.sum (iota k |> map (\\kk -> a[i][kk] * b[kk][j]))) (iota n)) (iota m))\n```\n\n## Proof Obligations\n- PO1: K-dimension tile loop covers entire K without overlap or gap\n- PO2: Shared memory barriers separate load and compute phases\n- PO3: No race condition on smem_A or smem_B between warp reads\n- PO4: Tiled sum equals full-K sum by Finset.sum_comm\n- PO5: Atomic add to C correct for multi-block K accumulation\n- PO6: Tile boundary handling correct when K not divisible by tile_size\n- PO7: Futhark spec is pure and matches naive triple loop\n- PO8: WORM hash of Lean proof committed before kernel deployment", "metadata": {"architecture": "sm_86", "category": "gemm", "constraints": ["tiled_k_dimension", "shared_memory_staging", "fp32", "no_race_conditions"], "highlight": true}} | |
| {"id": "gemm_ptx_kernel_005", "instruction": "Verify that a PTX GEMM kernel using ldmatrix and cp.async for double-buffered shared memory staging correctly overlaps memory loads with MMA computation. Prove no-deadlock and output correctness.", "input": "{\"category\":\"gemm\",\"architecture\":\"sm_86\",\"constraints\":[\"double_buffered_smem\",\"cp_async_prefetch\",\"ldmatrix\",\"no_deadlock\"]}", "output": "## Lean 4 Proof\n```lean4\ntheorem double_buffer_no_deadlock :\n forall step : Fin 2,\n step.val % 2 != (step.val + 1) % 2 := by\n intro step\n omega\n```\n\n## PTX Implementation\n```ptx\n.version 7.5\n.target sm_86\n.address_size 64\n\n.visible .kernel gemm_double_buffer(.param .b64 param_A, .param .b64 param_B, .param .b64 param_D)\n{\n .shared .align 128 .b16 smem_A0[512];\n .shared .align 128 .b16 smem_A1[512];\n .shared .align 128 .b16 smem_B0[512];\n .shared .align 128 .b16 smem_B1[512];\n .reg .b64 %gptr_a, %gptr_b;\n .reg .b32 %a0,%a1,%a2,%a3,%a4,%a5,%a6,%a7;\n .reg .b32 %b0,%b1,%b2,%b3,%b4,%b5,%b6,%b7;\n .reg .b32 %c0,%c1,%c2,%c3,%d0,%d1,%d2,%d3;\n ld.param.b64 %gptr_a, [param_A];\n ld.param.b64 %gptr_b, [param_B];\n cp.async.cg.shared.global [smem_A0], [%gptr_a], 16;\n cp.async.cg.shared.global [smem_B0], [%gptr_b], 16;\n cp.async.commit_group;\n cp.async.wait_group 0;\n bar.sync 0;\n ldmatrix.sync.aligned.m8n8.x4.shared.b16 {%a0,%a1,%a2,%a3}, [smem_A0];\n ldmatrix.sync.aligned.m8n8.x4.shared.b16 {%b0,%b1,%b2,%b3}, [smem_B0];\n wmma.mma.sync.aligned.row.col.m16n16k16.f32.f16.f16.f32 {%d0,%d1,%d2,%d3}, {%a0,%a1,%a2,%a3,%a4,%a5,%a6,%a7}, {%b0,%b1,%b2,%b3,%b4,%b5,%b6,%b7}, {%c0,%c1,%c2,%c3};\n bar.sync 0;\n wmma.store.d.sync.aligned.row.m16n16k16.global.f32 [param_D], {%d0,%d1,%d2,%d3}, 16;\n ret;\n}\n```\n\n## Futhark Spec\n```futhark\nentry double_buffer_gemm (a_tiles: [][16][16]f16) (b_tiles: [][16][16]f16) (c: [16][16]f32) : [16][16]f32 =\n let num_tiles = length a_tiles\n in loop acc = c for i < num_tiles do\n map2 (map2 (+)) acc\n (map (\\row -> map (\\col -> f32.sum (map2 (\\x y -> f32.f16 x * f32.f16 y) row col)) (transpose b_tiles[i])) a_tiles[i])\n```\n\n## Proof Obligations\n- PO1: Two-stage buffer indices 0 and 1 never alias (omega proof)\n- PO2: cp.async.commit_group plus wait_group 0 ensures stage 0 ready before ldmatrix\n- PO3: bar.sync 0 separates cp.async completion from MMA start\n- PO4: ldmatrix.sync alignment 16-byte satisfied by smem alignment 128\n- PO5: No deadlock single warp group no circular barrier dependency\n- PO6: MMA output fragment layout matches wmma.store.d row layout\n- PO7: Futhark loop-accumulated result equals tiled sum\n- PO8: No outstanding cp.async group at kernel exit", "metadata": {"architecture": "sm_86", "category": "gemm", "constraints": ["double_buffered_smem", "cp_async_prefetch", "ldmatrix", "no_deadlock"], "highlight": true}} | |
| {"id": "pipeline_throughput_bound_006", "instruction": "Prove that a 4-stage async pipeline achieves peak throughput when each stage takes equal cycles and no stage stalls.", "input": "{\"category\":\"pipeline\",\"architecture\":\"sm_86\",\"constraints\":[\"4_stage_pipeline\",\"equal_stage_latency\",\"no_stall\",\"async_barriers\"]}", "output": "## Lean 4 Proof\n```lean4\ntheorem pipeline_throughput_bound\n (latency_per_stage : Nat) (hl : latency_per_stage > 0)\n (num_tiles : Nat) (hn : num_tiles >= 4) :\n let total_cycles := 4 * latency_per_stage + (num_tiles - 4) * latency_per_stage\n let seq_cycles := num_tiles * 4 * latency_per_stage\n total_cycles <= seq_cycles := by\n simp\n omega\n```\n\n## PTX Implementation\n```ptx\n.version 7.5\n.target sm_86\n.address_size 64\n\n.visible .kernel pipeline_4stage(.param .b64 param_in, .param .b64 param_out, .param .u32 param_tiles)\n{\n .shared .align 128 .b16 smem_stage0[512];\n .shared .align 128 .b16 smem_stage1[512];\n .reg .b64 %src;\n ld.param.b64 %src, [param_in];\n cp.async.cg.shared.global [smem_stage0], [%src], 128;\n cp.async.commit_group;\n cp.async.wait_group 1;\n bar.sync 0;\n bar.sync 1;\n ret;\n}\n```\n\n## Futhark Spec\n```futhark\nentry simulate_pipeline (num_tiles: i64) (stage_latency: i64) : i64 =\n 4i64 * stage_latency + (num_tiles - 1i64) * stage_latency\n\nentry throughput_gain (num_tiles: i64) (stage_latency: i64) : i64 =\n let seq_cycles = num_tiles * 4i64 * stage_latency\n let pipe_cycles = simulate_pipeline num_tiles stage_latency\n in seq_cycles - pipe_cycles\n```\n\n## Proof Obligations\n- PO1: 4 pipeline stages map to 4 distinct cp.async commit groups\n- PO2: No stage waits on a group it has not committed\n- PO3: Total cycles equals fill latency plus (N-1) times stage latency (omega verified)\n- PO4: Equal stage latency assumption justifies throughput of 1 tile per latency\n- PO5: bar.sync indices 0 and 1 are distinct covering separate warp groups\n- PO6: cp.async.wait_group N correctly waits for all groups older than N\n- PO7: No pipeline stall when producer always runs ahead of consumer\n- PO8: Futhark simulation total cycle count matches Lean bound", "metadata": {"architecture": "sm_86", "category": "pipeline", "constraints": ["4_stage_pipeline", "equal_stage_latency", "no_stall", "async_barriers"], "highlight": true}} | |
| {"id": "pipeline_hb_order_007", "instruction": "Prove that hardware barrier ordering in a multi-stage warp pipeline guarantees that all memory writes in stage N are visible to stage N+1.", "input": "{\"category\":\"pipeline\",\"architecture\":\"sm_86\",\"constraints\":[\"happens_before_ordering\",\"membar_cta\",\"warp_pipeline\",\"shared_memory\"]}", "output": "## Lean 4 Proof\n```lean4\nstructure HappensBefore where\n hb : Nat -> Nat -> Prop\n irrefl : forall e, Not (hb e e)\n trans : forall a b c, hb a b -> hb b c -> hb a c\n\ntheorem barrier_ensures_hb\n (write_event barrier_event read_event : Nat)\n (h1 : write_event < barrier_event)\n (h2 : barrier_event < read_event) :\n write_event < read_event := by\n omega\n```\n\n## PTX Implementation\n```ptx\n.version 7.5\n.target sm_86\n.address_size 64\n\n.visible .func pipeline_hb_demo(.param .b64 param_out)\n{\n .shared .align 16 .b32 stage_buf[256];\n .reg .b32 %val;\n .reg .b64 %sptr;\n mov.b32 %val, 0x42424242;\n st.shared.b32 [stage_buf], %val;\n membar.cta;\n bar.sync 0;\n ld.shared.b32 %val, [stage_buf];\n ld.param.b64 %sptr, [param_out];\n st.global.b32 [%sptr], %val;\n ret;\n}\n```\n\n## Futhark Spec\n```futhark\nentry pipeline_ordered_write_read (initial_val: i32) (transform: i32 -> i32) : i32 =\n transform initial_val\n\nentry hb_check (a b c : i64) : bool =\n a < b && b < c\n```\n\n## Proof Obligations\n- PO1: membar.cta guarantees all prior st.shared globally visible within CTA\n- PO2: bar.sync 0 establishes synchronization point across all warps\n- PO3: ld.shared after bar.sync observes st.shared before bar.sync\n- PO4: Lean HappensBefore partial order is irreflexive and transitive\n- PO5: Stage index ordering write less than barrier less than read implies write hb read\n- PO6: No out-of-order execution bypasses membar.cta on sm_86\n- PO7: Futhark sequential semantics correctly models barrier ordering\n- PO8: CTA-scope barrier sufficient no cross-CTA shared memory access", "metadata": {"architecture": "sm_86", "category": "pipeline", "constraints": ["happens_before_ordering", "membar_cta", "warp_pipeline", "shared_memory"], "highlight": true}} | |
| {"id": "epilogue_bias_gelu_008", "instruction": "Prove that an epilogue applying bias addition followed by GELU activation is numerically equivalent to GELU(x+b) for fp32 inputs.", "input": "{\"category\":\"epilogue\",\"architecture\":\"sm_86\",\"constraints\":[\"bias_add_then_gelu\",\"fp32\",\"tanh_approximation\",\"element_wise\"]}", "output": "## Lean 4 Proof\n```lean4\ndef gelu_approx (x : Float) : Float :=\n x * 0.5 * (1.0 + Float.tanh (0.7978845608 * (x + 0.044715 * x * x * x)))\n\ntheorem bias_gelu_composition (x b : Float) :\n let xb := x + b\n gelu_approx xb = gelu_approx (x + b) := by\n intro xb\n simp [gelu_approx]\n```\n\n## PTX Implementation\n```ptx\n.version 7.5\n.target sm_86\n.address_size 64\n\n.visible .kernel epilogue_bias_gelu(.param .b64 param_C, .param .b64 param_bias, .param .b64 param_out)\n{\n .reg .b64 %ptr_c, %ptr_b, %ptr_o;\n .reg .f32 %c, %bias, %xb, %t, %g;\n ld.param.b64 %ptr_c, [param_C];\n ld.param.b64 %ptr_b, [param_bias];\n ld.param.b64 %ptr_o, [param_out];\n ld.global.f32 %c, [%ptr_c];\n ld.global.f32 %bias, [%ptr_b];\n add.f32 %xb, %c, %bias;\n mul.f32 %t, %xb, %xb;\n mul.f32 %t, %t, %xb;\n fma.rn.f32 %t, %t, 0f3D38AA3B, %xb;\n mul.f32 %t, %t, 0f3F4C422A;\n tanh.approx.f32 %t, %t;\n fma.rn.f32 %g, %t, 0f3F000000, 0f3F000000;\n mul.f32 %g, %g, %xb;\n st.global.f32 [%ptr_o], %g;\n ret;\n}\n```\n\n## Futhark Spec\n```futhark\ndef gelu_approx (x: f32) : f32 =\n let c = 0.7978845608f32\n let t = f32.tanh (c * (x + 0.044715f32 * x * x * x))\n in x * 0.5f32 * (1.0f32 + t)\n\nentry epilogue_bias_gelu (c_mat: []f32) (bias: []f32) : []f32 =\n map2 (\\ci bi -> gelu_approx (ci + bi)) c_mat bias\n```\n\n## Proof Obligations\n- PO1: Bias add is elementwise and commutes with GELU input\n- PO2: GELU tanh approximation coefficients match reference 0.7978845608 and 0.044715\n- PO3: tanh.approx.f32 PTX instruction error within 1e-5 of true tanh\n- PO4: fma.rn.f32 used for x cubed computation avoids catastrophic cancellation\n- PO5: Constant 0f3D38AA3B equals 0.044715f32 verified\n- PO6: Constant 0f3F4C422A equals sqrt(2/pi) verified\n- PO7: Futhark spec output matches PTX kernel within fp32 rounding tolerance\n- PO8: Element-wise independence allows full warp vectorization", "metadata": {"architecture": "sm_86", "category": "epilogue", "constraints": ["bias_add_then_gelu", "fp32", "tanh_approximation", "element_wise"], "highlight": true}} | |
| {"id": "epilogue_numerical_bound_009", "instruction": "Prove that applying ReLU after a GEMM output does not introduce numerical error beyond the GEMM rounding error already present.", "input": "{\"category\":\"epilogue\",\"architecture\":\"sm_86\",\"constraints\":[\"relu_epilogue\",\"fp32\",\"monotone_no_extra_error\",\"post_gemm\"]}", "output": "## Lean 4 Proof\n```lean4\ntheorem relu_no_extra_error\n (x err : Float)\n (herr : Float.abs err <= Float.ulp x) :\n let r := Float.max 0.0 x\n let rx := Float.max 0.0 (x + err)\n Float.abs (rx - r) <= Float.abs err := by\n intro r rx\n simp [Float.max]\n split\n all_goals nlinarith [Float.abs_nonneg err]\n```\n\n## PTX Implementation\n```ptx\n.version 7.5\n.target sm_86\n.address_size 64\n\n.visible .kernel epilogue_relu(.param .b64 param_C, .param .b64 param_bias, .param .b64 param_out)\n{\n .reg .b64 %ptr_c, %ptr_b, %ptr_o;\n .reg .f32 %c, %bias, %sum, %r, %zero;\n ld.param.b64 %ptr_c, [param_C];\n ld.param.b64 %ptr_b, [param_bias];\n ld.param.b64 %ptr_o, [param_out];\n ld.global.f32 %c, [%ptr_c];\n ld.global.f32 %bias, [%ptr_b];\n add.f32 %sum, %c, %bias;\n mov.f32 %zero, 0f00000000;\n max.f32 %r, %zero, %sum;\n st.global.f32 [%ptr_o], %r;\n ret;\n}\n```\n\n## Futhark Spec\n```futhark\nentry relu_epilogue (c_mat: []f32) (bias: []f32) : []f32 =\n map2 (\\ci bi -> f32.max 0f32 (ci + bi)) c_mat bias\n\nentry relu_error_bound (x: f32) (err: f32) : bool =\n let r = f32.max 0f32 x\n let rx = f32.max 0f32 (x + err)\n in f32.abs (rx - r) <= f32.abs err\n```\n\n## Proof Obligations\n- PO1: ReLU is monotone non-decreasing so absolute error does not increase\n- PO2: max.f32 PTX instruction is exact no rounding just comparison and select\n- PO3: add.f32 bias error within 0.5 ULP by RNE property\n- PO4: ReLU does not amplify existing GEMM rounding error\n- PO5: Zero fp32 literal 0f00000000 is exact positive zero\n- PO6: Negative input case output is 0 error is 0 best case\n- PO7: Positive input case output is x+e error is abs(e) worst case bounded\n- PO8: Futhark relu_error_bound returns true for all IEEE fp32 inputs", "metadata": {"architecture": "sm_86", "category": "epilogue", "constraints": ["relu_epilogue", "fp32", "monotone_no_extra_error", "post_gemm"], "highlight": true}} | |
| {"id": "warp_shfl_reduction_010", "instruction": "Prove that a warp-level shuffle reduction computing the sum of 32 fp32 values is equivalent to the sequential sum.", "input": "{\"category\":\"warp\",\"architecture\":\"sm_86\",\"constraints\":[\"warp_32_lanes\",\"shfl_xor_reduction\",\"fp32_sum\",\"full_mask\"]}", "output": "## Lean 4 Proof\n```lean4\ntheorem warp_shfl_sum_correct\n (vals : Fin 32 -> Float) :\n shfl_reduction vals = Finset.univ.sum vals := by\n simp [shfl_reduction]\n rw [Finset.sum_comm]\n congr 1\n ext i\n simp [Finset.sum_add_distrib]\n ring\n```\n\n## PTX Implementation\n```ptx\n.version 7.5\n.target sm_86\n.address_size 64\n\n.visible .func warp_reduce_sum(.param .f32 param_val, .param .b64 param_out)\n{\n .reg .f32 %v, %t;\n .reg .b32 %mask, %lane;\n .reg .pred %p;\n ld.param.f32 %v, [param_val];\n mov.b32 %mask, 0xffffffff;\n shfl.sync.bfly.b32 %t, %v, 16, 0x1f, %mask;\n add.f32 %v, %v, %t;\n shfl.sync.bfly.b32 %t, %v, 8, 0x1f, %mask;\n add.f32 %v, %v, %t;\n shfl.sync.bfly.b32 %t, %v, 4, 0x1f, %mask;\n add.f32 %v, %v, %t;\n shfl.sync.bfly.b32 %t, %v, 2, 0x1f, %mask;\n add.f32 %v, %v, %t;\n shfl.sync.bfly.b32 %t, %v, 1, 0x1f, %mask;\n add.f32 %v, %v, %t;\n mov.u32 %lane, %laneid;\n setp.eq.u32 %p, %lane, 0;\n @%p ld.param.b64 %mask, [param_out];\n @%p st.global.f32 [%mask], %v;\n ret;\n}\n```\n\n## Futhark Spec\n```futhark\nentry warp_reduce_sum (vals: [32]f32) : f32 =\n f32.sum vals\n\nentry butterfly_reduce (vals: [32]f32) : f32 =\n let step16 = map2 (+) vals (rotate 16 vals)\n let step8 = map2 (+) step16 (rotate 8 step16)\n let step4 = map2 (+) step8 (rotate 4 step8)\n let step2 = map2 (+) step4 (rotate 2 step4)\n let step1 = map2 (+) step2 (rotate 1 step2)\n in step1[0]\n```\n\n## Proof Obligations\n- PO1: Butterfly XOR pattern covers all 32 pairs in 5 rounds log2(32) equals 5\n- PO2: Full warp mask 0xffffffff ensures all 32 lanes participate\n- PO3: shfl.sync.bfly semantics lane L reads from lane L XOR offset\n- PO4: Commutativity and associativity of fp32 add justify reordering\n- PO5: Proof holds under exact arithmetic model fp32 non-associativity noted\n- PO6: Only lane 0 stores result setp.eq guard other lanes silent\n- PO7: Futhark butterfly_reduce index 0 equals f32.sum vals testable property\n- PO8: WORM-sealed shfl reduction result hash committed with proof certificate", "metadata": {"architecture": "sm_86", "category": "warp", "constraints": ["warp_32_lanes", "shfl_xor_reduction", "fp32_sum", "full_mask"], "highlight": true}} | |