Agent memory persistence — Benchmark Sources & Consensus
Agent ability to retain, transfer, and recall context across sessions — measured by task success rates before and after memory handoffs between models or restarts.
Platforms tracked: Hermes · Openclaw · Nemoclaw · Claude Cowork · Chatgpt
Consensus across 5 sources
Across 5 sources, persistent-memory layers lift agent task success: VEKTOR Slipstream scores 0.894 Transfer Continuity (vs PAM 0.880); World Model MCP adds +10.2 pts on SWE-bench repeat-mistakes; a self-reported memory-graph comparison puts 0.831 against 0.801 on LOCOMO. A founder's six-month account finds Hermes' built-in memory held up better than OpenClaw's but needed a second layer past its 2,200-character cap. A vendor-run test (40 questions, one model and harness) finds a SQLite MCP needs about 172k tokens per fully correct answer against 653k to 7.5M for Markdown stores, though its author calls the data unrepresentative. Several margins are narrow and self-reported, so treat the direction as better supported than the ordering.
All Sources
We aggregate published benchmarks; we never run our own tests and never pick winners. Each row links back to the original publication.
| Source | Date | Finding | Methodology | Quality |
|---|---|---|---|---|
| Medium / Vektor Memory | 2026-05-31 | VEKTOR Slipstream scores 0.894 Transfer Continuity Score vs Microsoft PAM's 0.880 across 50 engineering scenarios; memory lift ratio 6.61x vs 2.51x. | Transfer Continuity Score (task success with vs without memory transfer); 50 engineering scenarios across Q&A, coding, planning; GPT-4 Turbo baseline | high |
| Hacker News / GitHub | 2026-06-24 | World Model MCP, a harness-neutral memory layer, cut repeat coding-agent mistakes by +10.2 pts paired delta on 49 SWE-bench instances. | Pre-registered SWE-bench Verified repeat-mistake test; 49 instances; paired | high |
| GitHub | 2026-08-12 | A memory-graph implementation scored 0.831 against 0.801 for Memora on the LOCOMO long-conversation memory benchmark | LOCOMO benchmark run by the author against one named competitor; scores published with the harness in the repository | medium |
| Hacker News / dhawalshah.net | 2026-10-12 | Six-month side-by-side: Hermes' memory and self-written skills worked better from the start; OpenClaw's memory kept dropping context. | One founder, six months running both on rented servers; own logs and costs | medium winner: hermes |
| Hacker News / withnative.ai | 2026-10-27 | About 172k tokens per fully correct answer through a SQLite MCP, against 653k for Native-to-Markdown and 7.5M for Markdown with wikilinks. | 40 knowledge-work questions on one company event log (about 77,000 events), three runs per setup, OpenCode harness with DeepSeek V4.1 Flash · 40 questions, 120 trials per setup | medium |
How we work
OpenClawDatabase aggregates and links to published benchmarks. We don't run our own tests, and we don't pick winners. Our weekly benchmark-aggregator routine scans 7+ live leaderboards (OpenRouter, Aider, SWE-bench, GAIA, LMSYS, BigCodeBench, MMLU-Pro) plus relevant Reddit and Hacker News threads, then writes structured entries into /assets/benchmarks.json. Every row here links back to the original publication.
← Back to all benchmark tasks · See also: Decision guide · Cost calculator