What Serena says, vs. what it scored.
LSP-wrapped semantic code toolkit -- symbol-level navigation and refactoring via real language servers, 40+ languages.
“Serena provides essential semantic code retrieval, editing, refactoring and debugging tools that are akin to an IDE's capabilities, operating at the symbol level and exploiting relational structure.”
“Practically, this means that your agent operates faster, more efficiently and more reliably, especially in larger and more complex codebases.”
| Tool | MRR | p95 | p100 |
|---|---|---|---|
| ★ Atelier +semantic (BGE) | 0.727 | 390ms | 1057ms |
| ★ Atelier lexical (default) | 0.676 | 134ms | 319ms |
| Serena | 0.401 | 3834ms | 269001ms |
Serena is real code intelligence, not a grep wrapper -- LSP-based symbol navigation across 40+ languages. That architecture has a cost: a cold language-server spin-up on the first query, documented in Serena's own issue tracker (oraios/serena #634 repeated LSP re-initialization, #890 an 11-hour index on a large repo, #470 gitignore-parsing hangs). It shows up directly in our numbers -- 3.8s p95, 269s p100 on the slowest query of 7,213 -- and its own MRR (0.401) trails both Atelier channels.
The true story
Every tool in this comparison, Serena included, has been through the exact same 14 repositories and 7,213 query/gold pairs that score Atelier — no cherry-picked queries, no separate corpus. Full methodology, every raw number, and the other 9 tools →