What codebase-memory-mcp says, vs. what it scored.
Tree-sitter-based persistent knowledge graph (SQLite-backed) across 158 languages -- the most-starred tool in this comparison.
“The fastest and most efficient code intelligence engine for AI coding agents.”
“Evaluated across 31 real-world repositories: 83% answer quality, 10x fewer tokens, 2.1x fewer tool calls vs. file-by-file exploration.”
| Tool | MRR | p95 | p100 |
|---|---|---|---|
| ★ Atelier +semantic (BGE) | 0.727 | 390ms | 1057ms |
| ★ Atelier lexical (default) | 0.676 | 134ms | 319ms |
| codebase-memory-mcp | 0.502 | 541ms | 1817ms |
The 83%/10x/2.1x numbers are real and peer-reviewed (arXiv:2603.27277) -- but they're Codebase-Memory vs. reading files one by one, never against another code-search tool. On the same 7,213-query set as everyone else here, it scores 0.502 MRR: 31% behind Atelier's default channel, 44% behind Atelier's semantic channel.
The true story
Every tool in this comparison, codebase-memory-mcp included, has been through the exact same 14 repositories and 7,213 query/gold pairs that score Atelier — no cherry-picked queries, no separate corpus. Full methodology, every raw number, and the other 9 tools →