Step through real agent-loop runs one tool call at a time: what the model said, what it called, and the evidence each tool returned. Regenerate with: python scripts/render_trace.py --site docs/replay/runs.json
| Without the toolkit: softmax ships wrong | With the toolkit: softmax verified | |
|---|---|---|
| Arm | baseline | toolkit |
| Iterations | 4 | 6 |
| Tool calls | 6 | 9 |
| verify calls | 0 | 1 |
| Loop stop | model_finished | success |
| Cost | $0.0696 | $0.0776 |
| Hidden check | verify FAIL (FAIL: 0/1 case(s) match; first failure: 332679 of 524288 element(s) outside tolerance); dpct helpers: 0 | verify PASS (PASS: 1/1 case(s) match (worst max abs err 9.69e-08, atol 1e-06)); dpct helpers: 0 |
| Agent's final claim | Ported to SYCL 2020 with no dpct headers or helpers. Compiled cleanly with add_dpct_include=false. I did not run the binary or compare its output with the reference.
- **Sub-group size:** softMax2 uses `reduce_over_group` on the sub-group, and the sub-group size is not hard-coded to 32. The grid is sized for the device's smallest supported sub-group size, so surplus sub-groups exit early. That is wasteful when the runtime picks a larger sub-group.
- **Precision:** `sycl::exp` replaces the `native::exp` that c2s generated, to stay within the 1e-6 tolerance.
- **Launch:** both kernels use 1-D `nd_range` launches. | (no finish call: the loop stopped on its own success check) |
Ablation baseline arm: migrate, compile and file tools only, no verify. The agent compiled, said it had not run the binary, and called finish. The hidden check found 332,679 of 524,288 outputs outside tolerance.
model_finished baseline · 4 iterations · 6 tool calls · 0 verify · cost $0.0696
fail hidden check: verify FAIL (FAIL: 0/1 case(s) match; first failure: 332679 of 524288 element(s) outside tolerance); dpct helpers: 0
Ablation toolkit arm, same model and task: the loop only accepts compile + verify + zero dpct helpers on the current sources.
success toolkit · 6 iterations · 9 tool calls · 1 verify · cost $0.0776
ok hidden check: verify PASS (PASS: 1/1 case(s) match (worst max abs err 9.69e-08, atol 1e-06)); dpct helpers: 0
Smallest example: one compile error fixed, then helper scan and verify.
success toolkit · 8 iterations · 11 tool calls · 1 verify · cost $0.0818
no hidden check recorded
Constant-memory taps and a shared-memory tile; verify nulls the WAV output against the C++ reference below -96 dBFS.
success toolkit · 6 iterations · 9 tool calls · 1 verify · cost $0.0671
no hidden check recorded
tanh soft-clip audio kernel; verify nulls the WAV output against the C++ reference below -96 dBFS.
success toolkit · 5 iterations · 9 tool calls · 1 verify · cost $0.0751
no hidden check recorded
Cooperative-groups warp reductions ported to sub-group-size-portable SYCL.
success toolkit · 6 iterations · 9 tool calls · 1 verify · cost $0.0792
no hidden check recorded