The load & analysis harness
Embedding, vectors & search
Three pillars an agent hits on every question: turning text into vectors, storing them, and fusing / reranking / synthesising / measuring a query. Retrieval here is framed as token efficiency — a missed file forces an expensive grep-and-read detour, so every recovered hit is tokens the agent never spends. This page covers the local embedder, the two named vectors, three-lane hybrid fusion, the reranker, the synonym and answer-synthesis layers, and the eval harness that gates every change.
Three lanes, and how a question is routed between them
Beyond the semantic lane this page mostly covers, two more exist for questions semantic search
structurally cannot answer completely: an exact-coordinates compiler lane for C#, and a
deterministic SQL lane for tables and stored procedures. docs/ROUTING_GUIDANCE.md
is the single source of truth for which lane a question should hit, propagated byte-for-byte into every
repo's CLAUDE.md: orienting and "why" questions go to query_ledger; the definition,
every caller, or every implementation of a named symbol goes to symbol_lookup /
find_definition / find_references / find_implementations; an exact
literal (a config-key spelling, a log message) goes to grep.
The compiler lane (Services/PreciseReferenceService.cs) wraps an in-process
Roslyn MSBuildWorkspace per resolved repo root, registered in DI only behind
Precise:Enabled (code default false, appsettings.json:285). One
WorkspaceHandle is kept per root under an LRU of at most Precise:MaxWorkspaces
(default 4, :83) with the configured Precise:DefaultRoot pinned so it is never
evicted. A root that reports "loaded" with zero usable projects — or whose global.json pins an
SDK the host never installed — is flagged degraded rather than silently answering empty:
EvaluateLoadOutcome (:436-461) names the reason (no csproj found, restore missing,
an uninstalled SDK pin, or an MSBuild error) so a caller can never mistake "the compiler never ran" for "the
symbol doesn't exist" (todo 72936320, shipped as PR #191, deploy #6 baf8df4). A
query against an unloaded root always answers loading on its first call rather than blocking for
the 4-22s cold load.
Precise:Enabled=true on the live host (docs/LEDGER_SCALE_PLAN.md §12,
"2026-09-13 07:00–07:10 UTC: user-authorised flips") — a judgement call on capacity (RSS 5.1 → 8.1 GiB with
the agent workspace resident, roughly 8% of host RAM worst case), made explicitly before WP3.1's own
"enable after a week of flat RSS" gate was met (the 2026-09-12 RSS investigation had found the sawtooth still
rising, not settled) — the plan records the flip as an authorised exception to that gate, not evidence the
gate was satisfied. Confirmed live: the first find_definition MessageOrchestrator call resolved
to MotMasterAgent/Services/MessageOrchestrator.cs:23 with the agent root loaded and 3 csprojs
open. find_definition/find_references/find_implementations are
advertised on the MCP tool list only while PreciseReferenceService is in DI, i.e. only while
Precise:Enabled is true (McpService.cs PreciseTools/CoreTools,
:264-322) — including the SQL kind bypass below, since
CallFindReferencesAsync checks for a live PreciseReferenceService before it ever
inspects kind (McpService.cs:2958-2960). A root that cannot load no longer answers
with an empty result: find_definition/find_implementations against a degraded root
return {"error": "...", "truth": "none", "hint": "fall back to reference_set (text) or grep"}
(PreciseUnavailableJson, McpService.cs:3067-3073); find_references
instead falls back to its existing text/SQL lane and attaches a precise_unavailable_reason field
so that fallback stays distinguishable from an ordinary "no such symbol" (McpService.cs:3115) —
PR #191's fix for todo 72936320.The SQL lane (Services/SqlReferenceService.cs) is a deterministic reference
finder over every .sql/.cs/.razor/.cshtml/.json/.yaml/.yml file on disk — no Qdrant, no LLM,
unconditionally registered in DI (unlike the compiler lane, it is a cheap on-disk index, not a per-checkout
Roslyn workspace). It classifies each hit as a definition, call, read, write or config reference, and is
reached via find_references(kind:"sql"), which bypasses Roslyn entirely
(SqlOnlyReferenceResult, McpService.cs:2996-3011) — subject to the same
tool-visibility gate noted above, so it is live now that Precise:Enabled=true. FindReferences(name,
kind) and the find_references kind:"sql" tool are unchanged by the rewrite below.
2026-09-13: the .sql-file core is now a real T-SQL parser, not regex. PR #197
(6c79e80) replaces the old regex sweep over .sql files with
Microsoft.SqlServer.TransactSql.ScriptDom 180.107.0 (MIT) — TSql160Parser plus a
TSqlFragmentVisitor. The C#/config literal scan is untouched and is labelled source:"literal";
ScriptDom hits are labelled source:"scriptdom" (a `"regex-fallback"` label covers the rare file
ScriptDom cannot parse). Re-run against the same 20 verified objects as the original WP3.4a evaluation: the
.sql lane is byte-identical on 16 of 20 objects, and the literal lane is identical on all 20
(92/92 hits, unchanged). Totals moved from 322 (230 sql / 92 literal) to 308 (216 sql / 92 literal) — the
entire delta is one bug class the rewrite closes (below). Real T-SQL parsing costs about 1011 ms per
1,000 .sql files versus the old regex's ≈171 ms per 1,000 (~5.9× slower)
— roughly 5.1 s to parse the whole of motmaster-sql, paid once per Refresh(), never
per query, with 0 parse errors across 5,085 real files. 57 dynamic-SQL sites
(EXEC(@sql)/sp_executesql) are detected and deliberately labelled not-resolvable
rather than guessed at.
DefRegex also
matched ALTER TABLE … ADD CONSTRAINT — schema evolution on an already-defined table — and counted
it as a table Definition: 14 such hits, already marked "decision": "accept" in WP3.4a's
own ground truth (docs/evidence/sql-lane/verified-20.json). True precision on that population is
293/307 = 95.4%, not 100%. Recall (94.9%) is unaffected — this is a false positive, not a
false negative — and ScriptDom closes it categorically: it parses that statement as a distinct AST node type
the visitor has no case for, so it now correctly produces no reference at all.Semantic search cannot enumerate callers and will not tell you when it missed one; a compiler or a deterministic sweep answers those questions exactly. Naming the right lane up front, rather than treating the ledger as the only tool, is the difference between a confident wrong answer and a verified one.
The SCIP lane — a fallback for find_definition / symbol_lookup
PR #199 (6a25052) adds Services/ScipIndexService.cs + ScipIndexHostedService.cs: one SCIP
index per (repo, commit), built into /opt/arch-ledger/scip/<repo>/<sha>/ off the
existing ingest-completed signal — out-of-band, background-triggered, and never on a query path. Routing for
find_definition / symbol_lookup is Roslyn → SCIP → text, and every
result is labelled with a truth field: "compiler", "scip", or
"text". On the frozen-30 symbol set (docs/evidence/precise-spike/symbols.json)
definition recall is 25/25 = 1.00 on the resolvable (non-Razor) set; the 5 Razor components
score 0/5, which is a known-unresolvable case for every C# tool tried so far, Roslyn and ast-grep included —
not a SCIP-specific gap. Precise:Scip:Enabled defaults true, gated on the
measured build time coming in under a 2-minute threshold.
find_references
stays Roslyn-only for C# — SCIP is never consulted for it. The WP3.0 spike measured SCIP's
raw occurrence lookup at 0.310 average reference recall against Roslyn truth, driven by a
systematic interface-dispatch gap (an interface-dispatched call is recorded against the interface method's
own symbol, not the implementing class Roslyn cascades to for free). The find_references tool
description states this figure so an agent is not misled into trusting it for a rename or signature-change
decision.agent@022de380f47d50e71bf9f6f1300db7f18a75844f (csharp): 52139 definitions, 91463ms, 39424404 bytes,
peakRss=790568960 — 38 MB on disk. The builder's own isolated measurement during development was
43.04 s / 37.6 MB for the same repo; production under load took 91.5 s — still inside the
2-minute gate, but much closer to it than the development figure alone would suggest. Both numbers are worth
keeping in view: the fast one shows the mechanism works, the slow one shows the margin that is actually left.A repo Roslyn cannot open — an unrestored checkout, an SDK pin the host lacks — no longer means
find_definition goes silently empty; SCIP recovers exactly that degraded case, measured on the
real FixtureBadSdk scenario, while find_references stays on the tool that is
actually accurate for it.
Local ONNX embeddings
EmbeddingService.cs:8-20 wraps SmartComponents.LocalEmbeddings.LocalEmbedder (a
bundled ONNX model). Dimensions = 384 is a public constant (:10) the Qdrant schema
keys off. Embed(string) is synchronous and allocation-light, and the singleton is reused across
ingest and query. There is no API, no network, and no per-token cost.
Embedding is free and offline, so the ledger can re-vector thousands of chunks at ingest and embed every incoming query at $0. In the blueprint cost model the local embedder is why the whole LEDGER path is $0 — only the DIRECT (grep-and-read) fallback burns agent tokens.
Two named vectors — code_vector + design_vector
Each chunk is embedded twice: what the code literally is (identifiers/syntax) and what it
means architecturally (intent/layer). At ingest (IngestionEngine.cs:484-487)
CodeVector = Embed(text) and DesignVector = Embed($"[{repoId}/{layer}] {intent}"). At
query time (McpService.cs:1732-1733) designVec = Embed(DesignQuery(query)) and
codeVec = Embed(query), where DesignQuery (:4453-4458) optionally prepends
Retrieval:DesignQueryPrefix. The key lesson (:1727-1731): embedding both axes
identically made the code axis pure noise under fusion — the asymmetric framing is deliberate.
One query matches both "the file whose text contains pr_customer_otp_verify" and "the thing
that handles OTP verification", combining evidence — and it backs find_similar's code / design /
hybrid modes.
Qdrant collection shape and is_latest filtering
The collection motlogs_unified_architecture (QdrantService.cs:157) has a
VectorParamsMap of two named vectors, both 384-d Cosine (:203-207). Keyword payload indexes on
repo_id, file_path, layer, pattern, concept_id, chunk_type make facet pre-filters server-side
(:213-218). Bitemporal versioning (Versioning:Enabled, OFF by default,
:129) governs is_latest: off means one deterministic point-id per (repo,file,chunkIndex),
re-ingest overwrites, a single version, byte-identical to the pre-versioning path; on means prior versions are
retained (is_latest=false, :607-623) and search Must-filters is_latest=true
via WithLatestOnly (:272-289, applied :866), with superseded versions
GC'd on RetentionDays.
Named vectors serve both lanes without duplicating storage, and keeping versioning off in production protects the eval baseline while leaving the door open to "what did this look like before" without polluting default retrieval.
Hybrid retrieval — RRF fusion of three lanes
HybridSearchAsync (QdrantService.cs:859-989) over-fetches
max(limit*4,16) candidates (:861), runs two dense SearchAsync calls (one
per named vector) in parallel (:869-871), and fuses with Fuse(id, lane, rank0, weight)
adding weight/(k+rank0+1) per lane (:880-889) — defaults RrfK=60, DesignWeight=1.0,
CodeWeight=0.6 (:115-117). The raw fused score is normalised 0–1 against
RrfMaxScore=(design+code)/(k+1) (:138,923-924,950), which is why MCP shows
high (72%) not the raw 0.026 (ConfidenceLabel,
McpService.cs:4462-4469).
Vectors catch paraphrase and keywords catch the exact identifiers the embedder fuzzes — fusing both means a query lands whether the agent knows the concept or the exact symbol name.
BM25 lexical lane — the recall win
LexicalIndex.cs is a self-contained in-memory BM25 (k1=1.2, b=0.75, :21-22) with a
code-aware tokenizer (:87-100): it lowercases the raw alphanumeric token AND emits
camelCase/PascalCase/snake_case sub-tokens, so VehicleTestStation matches
vehicle/test/station and pr_customer_otp_verify matches its parts. It is built lazily
from a cached corpus scroll (no re-ingest), with a 5-minute TTL under a build lock
(EnsureLexicalAsync, QdrantService.cs:1305-1331), indexing
file_path+intent+content per chunk. It is RRF-fused as a third list with LexicalWeight=1.0, only
when there is no server-side pre-filter and a raw queryText is present (:906-921). It is
best-effort — a lexical failure never fails the query.
This is a measured recall lift: the class doc and blueprint §8 attribute the M1 recall@5 improvement to it. It is the difference between finding a sproc by name and falling back to grep.
Cross-encoder reranker stage
An optional precision pass re-scores the top of the fused pool by feeding actual (query, passage) pairs
through a cross-encoder — where RRF only ever saw ranks. CrossEncoderReranker.cs runs
bge-reranker-base ONNX (XLM-Roberta, SentencePiece) shared from the wiki/agent host models
(:14-23), framing pairs as <s> query </s></s> passage </s> in a
512-token budget, and self-disables to identity order if the model or tokenizer is missing (:49-98)
so it is always safe to call. It reorders the top RerankPool=24
(QdrantService.cs:927-942).
Retrieval:CrossEncoderRerank=false) because the current
wiring regressed recall@1 from 0.545 to 0.227 (a suspected tokenizer / output-tensor mismatch,
under investigation). A simpler lexical-overlap Rerank also exists in McpService behind
Retrieval:RerankEnabled.A working cross-encoder is the standard top-1 precision squeeze, but the honest current state is a documented regression — a live tuning lever kept deliberately dark until it is fixed rather than shipped making results worse. It has now been measured twice, in two different ways, with the same verdict.
Lane attribution / explain
Opt-in observability: for each hit, which lanes surfaced it, at what pre-fusion rank, and each lane's RRF
contribution — without changing ranking or the default wire shape. LaneAttribution +
HybridSearchExplain + the canonical RetrievalLanes (dense-design, dense-code, bm25)
live in Retrieval/LaneAttribution.cs, populated only when explain is passed
(QdrantService.cs:876-889,953-986) and rendered by FormatExplain
(McpService.cs:4284-4302) behind explain:true.
It closes blueprint §8's "tuning is blind" gap: the eval harness can see why a hit ranked where it did and tune the weights and k from data instead of guesswork.
Synonym expansion
On weak results the query is rewritten with approved domain synonyms
(VRN → vehicle registration, reg number) and re-searched.
SynonymStore.ExpandQuery (Synonyms/SynonymStore.cs:341+) returns the original plus up
to 4 whole-word case-insensitive rewrites (cap 5), from an approved-synonym cache with a 5-minute TTL
(GetApprovedCachedAsync :290). query_ledger triggers this only on weak results
(hits.Count==0 || hits[0].Score<0.20), fans out ≤3 alternates to bound latency, and merges + re-ranks
by score (McpService.cs:1758-1804), flagging SynonymsExpanded. New synonyms are proposed
by SynonymSuggester (reading failed-query clusters, asking Groq) and land as Approved=false
in motlogs_synonyms until an admin vets them.
It is a human-in-the-loop learning loop: real miss clusters generate candidates, an admin approves, and future queries with the same vocabulary gap succeed — with no re-ingest.
ledger_ask — synthesised, cited answers
ledger_ask takes a natural-language question and returns synthesised prose plus structured
file/node citations — a mini-agent over the ledger's own tools. AskService.cs runs a bounded
tool-calling loop (max 8 hops, 60s budget, :29-30) against GroqService's chat client, exposing the
read tools (query_ledger, node_neighbours, find_similar, cluster_cohort, cohort_consistency, refactor_plan,
count_sprocs, orient_me, why_query, tests_for, :220-441), pre-classifies via QueryRouter
(:82-93), and parses each result's trailing ```json cited``` block to light up cited
nodes in the graph UI (:482-524). The system prompt hard-forbids fabricated counts or paths
(:443-477).
It returns the answer plus verifiable citations, not a file dump — and it backs both the graph
viewer's Ask box and the ledger_ask MCP tool.
Ask — the one human front door
/ask.html is a single natural-language question box over
GET /api/search — the same hybrid retrieval described above, no separate index and no separate
ranking. It is vanilla JS with no framework and no CDN, and every figure it displays is fetched at render time
from /api/search or /api/stats and hidden — not defaulted, not estimated —
when the endpoint does not supply it.
Three calls back the page (wwwroot/ask.js):
| Call | Purpose | Access |
|---|---|---|
GET /api/search?query=…&limit=8&source=ask | The hits | Public |
GET /api/symbols?prefix=…&limit=20 | Symbol type-ahead — resolves a partial name to real indexed symbols with their repo, file and line span | Loopback, tailnet, or bearer token |
GET /api/stats | Corpus size and index freshness, shown beside the box — omitted entirely if the endpoint does not return the field | Public |
There is exactly one Ask surface, and that is the point. The public shell
(app.html) once embedded its own Ask screen; it was a second, untagged search surface that sent
no source=ask, so a human question asked there was indistinguishable from agent traffic. It has
been removed, and the shell's rail entry, its ⌘K palette and every "re-run this query" control now link out
to the one audited page.
source is an allow-list, not a label (Services/QuerySource.cs).
/api/search is unauthenticated and logging it performs a vector write, so a call with no
source — or an unrecognised one — is not logged at all. Ask rows are also
excluded from the 24h/7d aggregates, which have always described agent traffic: the rows survive,
they are simply not the agent numbers.The metric this exists to make measurable — time-to-first-correct-answer for a human — is specified, not
yet baselined, in
docs/HUMAN_ACCESS_METRIC.md.
That spec is deliberate about its own limits: the retrieval leg (question submitted → hits returned) is
observable as latency_ms; the judgement leg (hits returned → the human recognises the right
file) logs no click, no dwell and no confirmation, and is almost certainly the larger of the two. No baseline
or target is quoted there, because the source field only started being written when the Ask page
shipped and any number quoted before a window of real traffic exists would be invented.
Humans and agents now ask the same index through one measurable door. Everything the tuning work bought for agents — the lexical lane, the priors, span-first citations — a developer gets by typing a question into a box.
Relevance evaluation harness
A golden-set regression gate runs through the exact query_ledger path.
GoldenSetProvider holds about 44 hand-authored questions, each naming expected file(s) plus an
optional repo gate (RelevanceEval/GoldenSetProvider.cs). RelevanceEvaluator.RunAsync
(RelevanceEval/RelevanceEvaluator.cs:47-84) embeds each query the same asymmetric way, calls
HybridSearchAsync with the real queryText/lexical/rerank flags, and scores with
RelevanceMetrics: recall@1, recall@5, MRR, plus a Misses watch-list
(RelevanceMetrics.cs:97-114). It is exposed at GET /api/admin/eval/relevance
(Program.cs:2814+) and wired into the deploy gate. The baseline was recall@5 0.458 / MRR 0.414
(blueprint §8 M1); the token-optimization waves lifted recall@5 to about 0.68; after the 2026-08-05
method-level re-chunk it reached 0.8636 (see the tuning log for that history). The
question set and corpus have both moved since, so that figure is superseded — the current dated snapshot is
recall@1 0.5125 / recall@5 0.75
(2026-09-12, docs/evidence/baseline-2026-09-12/README.md, run against the live
/api/admin/eval/relevance endpoint).
It is a no-regression contract for every retrieval change — TOON, rerank, weight tuning and embedder swaps all A/B against it before any default flips.
Retrieval at scale — the efcore trial
To find out whether hybrid search holds up on a corpus bigger than the whole MOTMaster estate, WP5.1
ingested dotnet/efcore (pinned commit, 6,108 files) into an isolated second Qdrant container and
ran the golden-set eval against it. Dated 2026-09-13, docs/evidence/efcore-trial-2026-09-12/RESULTS.md:
79,633 chunks — 32% more than all nine live MOTMaster repos combined (~60,451) — recall@5
0.950, recall@1 0.825, MRR 0.8865, query p50 32.2 ms, query p95 43.8 ms (34×
inside its 1.5s target), at 1.543 GiB RAM per 100k chunks and ≈$62 Groq per 100k fresh chunks. Every sampled
chunk carried intent_source: "groq", confirmed single-provenance before the recall number was
trusted. The trial also found the compiler lane's cost at scale is unmeasured: efcore was never
restored with its pinned preview SDK, so Roslyn loaded zero projects, and — at the time of this trial, before
the fail-loud fix below shipped — the failure was silent: find_definition returned an empty match
list indistinguishable from "the symbol doesn't exist" instead of a degraded-root reason.
The semantic lane's recall and latency hold up an order of magnitude past today's corpus size at a
predictable cost; the compiler lane's cost and behaviour on a codebase anywhere near that size remain
untested — production's Precise:Enabled=true today covers the MOTMaster estate, which this
host always keeps restored, not a foreign repo the size of efcore.
Query analytics — hit-rate, latency, miss clustering
MetricsService.RecordQueryOutcome (MetricsService.cs:133-171) tags each query
zero (no hits) or low-relevance (top score below threshold), keeps a bounded
RecentMisses ring plus ZeroHit/LowRelevance counters and per-file HitStats. Every MCP call is persisted as an
AgentQueryRecord with LatencyMs, ReturnedCount, TopHitScore and SynonymsExpanded
(McpService.cs:1815-1830). FailedQueryClusterJob clusters records with TopHitScore <
0.20 or zero hits by cosine over stored query vectors; LearningLoopService joins those clusters with
useful=false feedback into ranked human-review actions; and TeamMetricsService surfaces the
LEDGER-vs-DIRECT ratio. An in-process LRU also caches query_ledger results themselves — keyed on
query+limit+depth+edgeKinds+includeDeprecated, 256 entries, 60s TTL
(McpService.cs:46-51) — so a repeated identical call in that
window skips re-embedding and re-searching entirely.
It closes the loop REBUILD_PLAN flagged as broken (feedback written but never read): miss clusters feed synonym suggestions and golden-set additions, so the system learns from its own failures — and the query cache means a burst of identical calls (a re-run, a retry) costs one real search, not several.
TOON dense serialisation, and the config-driven defaults
An opt-in output format: one legend line plus one packed row per hit, versus decorated Markdown.
FormatResultsToon (McpService.cs:4261-4280) writes
# legend: repo|layer|pattern|path|Lstart-Lend⟫snippet once then one line per hit. It is selected via
format:"toon" on query_ledger, is a serialisation boundary only; the projection is a
30–50% reduction of serialisation overhead at limit 5–10.
What an absent format/limit argument resolves to is itself config-driven.
ResolveDefaultFormat/ResolveDefaultLimit (McpService.cs:3829-3849) read
Mcp:DefaultFormat (must be markdown or toon) and
Mcp:DefaultLimit (clamped 1-20); an explicit argument on the call always wins, and an invalid
config value falls back to the code default (markdown, 5) and logs once rather than per call. The
code default reproduces yesterday's behaviour exactly, so a deploy that never sets the keys is a no-op — but
the live value is not the checked-in one:
appsettings.json:33 ships "markdown"/5, while production was hand-flipped to
Mcp:DefaultFormat=toon, Mcp:DefaultLimit=3 at 19:08 UTC on 2026-09-12
(docs/LEDGER_SCALE_PLAN.md §12, "Compact-answer default flipped"), verified live by a
no-limit call returning 3 toon rows.
The agent spends its context budget on snippet content, not scaffolding — with the discipline "measure before defaulting" (recall@5 and MRR must stay flat) — and ops can flip the default shape for every caller that doesn't ask otherwise, without touching a single call site.
One snippet renderer, and Mcp:FullTopHitBody
Every hit body — full, skeleton, or expanded — used to be selected and capped independently by each of the
three renderers (plain markdown, rich markdown, span-first), and the span-first path had silently hand-rolled
its own copy of the 600-character cap and never read Mcp:FullTopHitBody at all. Snippet()
(McpService.cs:5439-5440) is now the one place a renderer gets a hit's body: SelectSnippet
picks full vs. skeleton vs. expanded, then SnippetForRank applies the 600-char SnippetCap
— except at rank 0, under detail:"full", when Mcp:FullTopHitBody=true (default
false), which prints the top hit's whole body uncapped. 62.5% of live chunks exceed 600 characters
(sampled: mean 1,650, median 977), and 6 of 20 recorded read_span calls in the real-session bench
followed a query_ledger for exactly this reason — one extra turn to see the rest of the answer
that was already returned. This is a length change only: every admission predicate (ingest-time file filters,
query-time path exclusion) runs upstream and decides on whole hits, so nothing is un-truncated into visibility
that would otherwise have been excluded.
A future cap or FullTopHitBody change lands once and applies to every renderer, and a
turn that used to be spent re-reading the same file via read_span is now spent on the next
real question instead.
Per-file staleness marker — ⚠ file changed since indexed
Every WP4.1a chunk carries indexed_sha — the repo's git HEAD at ingest time. The span-first
renderer's ComputeStaleFlagsAsync compares that against the repo's current HEAD as a fast
path (repo unmoved ⇒ nothing in it can have changed, skip the git call); when the repo has moved, it asks git
directly whether this hit's file differs between the two shas via
git diff --quiet <indexed> <head> -- <file> (exit 1 = changed), caching the head
sha once per repo and the diff result once per (repo, sha, file) triple for the life of one request — never
per hit. A hit whose file changed appends · ⚠ file changed since indexed to its meta line
(SpanFirstHitHeader:5395); anything unresolvable — no indexed_sha, not a git
checkout, an unknown sha — is silently left unflagged rather than guessed.
Comparing the whole repo's HEAD alone would over-fire on every busy repo's daily commits; per-file comparison means the marker only ever fires when the specific answer you're reading might actually be wrong.
The checkout overlay — searching a worktree that hasn't merged yet
The main collection mirrors main. CheckoutOverlayService lets a caller register a
git worktree or checkout root (/root/code/<repo>/.claude/worktrees/<id>,
/root/code/ws-*/motmaster-*, or /root/code/wt-* — WorktreePattern/
WsPattern/WtPattern) via POST /api/admin/overlay/register (loopback),
computes its delta against main — committed-but-unmerged diff ∪ working-tree status — and indexes
changed/added files (deleted files as tombstones) into a separate motlogs_overlay Qdrant
collection, never touching the main one. Passing that root as the root argument to
query_ledger then serves overlay hits marked ⊕ from your checkout (base <sha>)
(SpanFirstHitHeader:5404-5406, from the point's base_sha payload key). Every file is
swept by ISecretsScanner (GitleaksSecretsScanner) before indexing, and a fixed
never-index glob list (*.env, appsettings.*.json, *.pfx/*.pem/*.key,
id_rsa*) is excluded regardless. Defaults: Overlay:Enabled=true,
MaxFilesPerRoot=500, MaxRoots=60 — of 180 candidate checkouts on the eval host, 60
registered and 120 were refused (HTTP 413) at that cap (docs/LEDGER_SCALE_PLAN.md §12,
2026-09-12 evening; open todo eb98a1ca).
An agent working in an unmerged worktree can ask about its own uncommitted changes instead of planning against whatever the main collection last indexed — the gitleaks gate means that convenience never becomes a secrets leak.
Token-saving benchmark (bench_*)
An A/B measuring the ledger's actual token cost versus naive DIRECT (grep-and-read) for the same question
set — the evidence behind "recall is token efficiency." The bench_* MCP tools plus
BenchRunner/BenchmarkCollection drive identical questions two ways (LEDGER vs DIRECT, or
markdown vs toon) and record the token deltas; TeamMetricsService.EfficiencySummary rolls up the
system-wide ratio. It is tokens-only — $0, since embeddings are local — and it gates TOON and rerank changes
alongside RelevanceEval. The full protocol is on the MCP tools page.
It converts the central claim — "the ledger saves tokens" — into a measured number rather than an assertion.
Retrieval-tuning levers
The Retrieval block in appsettings.json is config-driven — most of these change with
no redeploy.
| Lever | Key | Default | Effect |
|---|---|---|---|
| RRF constant | RrfK | 60 | rank-decay steepness |
| Design weight | DesignWeight | 1.0 | intent-space influence |
| Code weight | CodeWeight | 0.6 | literal-code influence |
| Lexical lane | LexicalEnabled/LexicalWeight | true/1.0 | BM25 recall lane |
| Over-fetch | OverFetchMultiplier/OverFetchMin | 4/16 | candidate pool size |
| Design query cue | DesignQueryPrefix | "" | push query toward intent |
| Cross-encoder rerank | CrossEncoderRerank/RerankPool | false/24 | precision pass (regresses — off; re-tested 08-03) |
| Prose-document prior | DocPrior | 0.6 (shipped in the repo) | demotes .md/.txt/.pdf in fusion; recall@5 0.614→0.727 (sweep) |
| Test / CI prior | TestPrior | 0.6 | demotes test artefacts; rejected on the class-level corpus, reversed after method-level chunking made test files competitive — worth +4.5 pts recall@1 (correction) |
| Per-file diversity cap | MaxPerFile | 1 | one hit per file, so a fragmented file cannot occupy the whole result set |
| Confidence band | LowConfidenceThreshold | 0.45 | label calibration |
| Versioning | Versioning:Enabled | false | is_latest filtering |
Deeper levers — an embedder swap or a fixed reranker — require a full re-ingest / re-vector and are gated by
RelevanceEval + bench_* A/B before any default flips.
Retrieval quality can be tuned from data without a redeploy, and every lever is guarded by a measured no-regression gate — so tuning is empirical, not a leap of faith.
Open Brain — the MOTMaster Architectural Ledger · generated from source 2026-07-07