nitrateX
Salvor

Salvor, tested

Independent local suite results and field trials of repo-native memory for coding agents.

Last suite run: 14 October 2026 · dev · 9140ea9 · WSL2 Ubuntu, Node v22.22.1, Playwright Chromium View repository ↗

Share on X

100 sessions · 1 brainClaude Code · 34CLAUDE.mdCodex · 33AGENTS.mdGemini CLI · 33GEMINI.md GitHub · repo.salvor/ active_state.md L1 +1 active_state_verbose.mdL2 · on demand DOMAIN_REF.md · LF: DEFERRED_TODOS.md · Deferred domain-learnings/ decisions/ · Decision blocked: LF:retry · Learned failure Save this as a domainlearning? (yes/no)Human approval no ×yes ✓ main PR · reviewed ✓
Unit tests186/186
Browser29 passed · 6 skipped
Trials run2
Best Codex armC · one win + one tie

Where Salvor delivered

Full Salvor finished first or tied for first in both Codex trials. On the live static site it scored 24.5/25, against 17/25 with no memory.

The biggest gains came on project rules that live only in memory: language parity, internal links and version bumps. The agent followed them without being told.

The local test suite passed cleanly: 186 of 186 unit tests and 29 browser tests, with no failures.

Memory helped at every level: even the short memory arm beat no memory on the static site (19 vs 17), and full Salvor went furthest.

Local test suite

14 groups · 186 passed · 0 failed

GroupTestsPassedDuration
agentic-contract99339 ms
benchmark-contract1111342 ms
beta-consistency88364 ms
brand-assets11118182 ms
burn-field99307 ms
consistency1919360 ms
final1919355 ms
ratification1010311 ms
reconcile-contract99341 ms
release-gate1414326 ms
setup-safety2020344 ms
site-contract3636432 ms
site-interactions33311 ms
upgrade-contract88326 ms
Browser tests: 35 · 29 passed · 6 skipped · 0 failed · 155.8 s
  1. PASSEDdesktop exposes the engineering problem above the fold without clipping hero actions838 ms
  2. PASSEDtablet exposes the engineering problem above the fold without clipping hero actions599 ms
  3. PASSEDphone360 exposes the engineering problem above the fold without clipping hero actions495 ms
  4. PASSEDphone320 exposes the engineering problem above the fold without clipping hero actions452 ms
  5. PASSEDadaptive SM favicon renders dark on light chrome and white on dark chrome1812 ms
  6. PASSEDdesktop renders without horizontal overflow1387 ms
  7. PASSEDcompact-desktop renders without horizontal overflow1296 ms
  8. PASSEDtablet renders without horizontal overflow1411 ms
  9. PASSEDlarge-phone renders without horizontal overflow1240 ms
  10. PASSEDgalaxy-s25-edge-small-phone renders without horizontal overflow1178 ms
  11. PASSEDnarrow renders without horizontal overflow1110 ms
  12. PASSEDdesktop renders the dogfood cases as nested canonical knowledge926 ms
  13. PASSEDtablet renders the dogfood cases as nested canonical knowledge866 ms
  14. PASSEDphone360 renders the dogfood cases as nested canonical knowledge686 ms
  15. PASSEDphone320 renders the dogfood cases as nested canonical knowledge687 ms
  16. SKIPPEDdesktop keeps each MCP tool name dominant and baseline-aligns its function label and icon60 ms
  17. SKIPPEDcompact-desktop keeps each MCP tool name dominant and baseline-aligns its function label and icon62 ms
  18. SKIPPEDtablet keeps each MCP tool name dominant and baseline-aligns its function label and icon63 ms
  19. SKIPPEDlarge-phone keeps each MCP tool name dominant and baseline-aligns its function label and icon64 ms
  20. SKIPPEDgalaxy-s25-edge-small-phone keeps each MCP tool name dominant and baseline-aligns its function label and icon69 ms
  21. SKIPPEDnarrow keeps each MCP tool name dominant and baseline-aligns its function label and icon64 ms
  22. PASSEDthe enhanced hero uses the compact source and one semantic heading696 ms
  23. PASSEDRetina desktop WebGL keeps the WF hero at high-resolution backing scale1037 ms
  24. PASSEDdesktop keeps the full-color mystic ready beneath WebGL WF651 ms
  25. PASSEDmobile keeps the full-color mystic ready beneath WebGL WF518 ms
  26. PASSEDmobile navigation is real and keyboard accessible555 ms
  27. PASSEDhero copy passes mouse drags through to the burn surface while navigation remains selectable9027 ms
  28. PASSEDslow mouse drags use spaced burn points and later clicks keep earlier burns105971 ms
  29. PASSEDWebGL owns exact browser-rendered UI transition without changing the established ready or M end states1556 ms
  30. PASSEDtouch input can tap and drag the burn surface through the large SALVOR text5020 ms
  31. PASSEDWebGL failure preserves the static WF fallback890 ms
  32. PASSEDreduced motion leaves the wireframe static793 ms
  33. PASSEDthe hero tagline stays on one line from 320px to desktop7212 ms
  34. PASSEDthe open mobile menu paints above the hero copy after the burn reveals1394 ms
  35. PASSEDthe open mobile menu closes when the viewport grows past the breakpoint1460 ms

Field trials

  • ANo agent memory
  • BShort memory: AGENTS.md + one-line learned-failure list
  • CFull Salvor (provisional capture)

Live static site (HTML5, TR/EN)

Codex CLI 0.147.0, ChatGPT Plus · 7 October 2026

Audited, LF-intent reading (25 checks)

A17/25
B19/25
C24.5/25
Task-by-task results and cost

Per-task breakdown not shown · 5 tasks, 25 checks

Tokens (5 tasks total)

A209k
B246k
C342k

Time (5 tasks total)

A8 min
B12.7 min
C17.8 min

Recorded time per arm · 0–20 min

Unpublished web app (Python / FastAPI)

Codex CLI 0.147.0, ChatGPT Plus · 7 October 2026

Audited, LF-intent reading

A2/5
B1/5
C2/5
Task-by-task results and cost
TaskABC
Turkish word collisionFAILFAILFAIL
Contentless questionFAILPARTIALPASS
Fake confidence floorFAILFAILFAIL
Normalizer strips meaningPASSFAILFAIL
Retry only on 429PASSPARTIALPASS

Scroll to compare all arms

Tokens (per run)

A36k
B50k
C67k

Time (per run)

A0–4 min
B8–20 min
C4–15 min

Recorded ranges; no midpoint estimate · 0–20 min

Method and caveats

  • Rubric written before any output was read
  • PASS 1 · PARTIAL 0.5 · FAIL 0
  • One run per cell (n=1)
  • Audited scores where the audit differs
  • Small sample: read as signals, not proof
  • Projects anonymized
  • Independent tests, not affiliated with Salvor