Interfaces & ops

MCP tools & deployment

The Open Brain is a single self-contained .NET 10 service that exposes the team's entire semantic code index to AI agents over the Model Context Protocol at https://brain.motlogs.com, backed by Qdrant. This page catalogs the MCP tools agents call — 46 declared (McpService.cs's FullToolList), of which 25 are advertised today — the response shapes they return, and the deployment, Qdrant, re-ingest, and enforcement machinery that keeps the service running.

Transport — streamable HTTP + stdio

The primary transport is streamable HTTP — POST /mcp, JSON-RPC 2.0, protocol version 2024-11-05 (McpService.cs:26), wired at Program.cs:1586; GET /mcp (Program.cs:1631) is a discovery/health card. JSON-RPC methods are dispatched (McpService.cs:197-203): initialize, ping, tools/list, tools/call, resources/list, resources/read; a null response (notifications) returns HTTP 204. Bearer auth (Program.cs:1586-1616) resolves a presented Authorization: Bearer <token> against (a) the shared Mcp:BearerToken → identity "shared", or (b) a registered per-developer token via TokenRegistry → real {developerId, team}; a registered token always wins over the shared token, a mismatch returns 401, the compare is constant-time (HttpAccessGate.ConstantTimeEquals, Services/HttpAccessGate.cs:132-139), and with no shared token configured the service is public but registered tokens are still honoured. Note that /mcp does not waive the bearer for a loopback/tailnet peer the way /api/* does (see the access gate below) — every caller, on-host or remote, presents the same header. stdio mode (Program.cs:40, --mcp) binds nothing in Kestrel, routes logging to stderr so stdout is a clean JSON-RPC channel, runs RunStdioLoopAsync() (McpService.cs:131), then exits.

Connecting — the recipe

Add the server as an http-type MCP entry (Claude Code: ~/.claude.json's mcpServers, or claude mcp add). Remote, off-host:

{
  "mot-ledger": {
    "type": "http",
    "url": "https://brain.motlogs.com/mcp",
    "headers": { "Authorization": "Bearer <your token>" }
  }
}

On-host — a session running directly on the Hetzner box, or tunnelled to it — talks to the service on its loopback port, skipping nginx and TLS entirely: http://localhost:5005/mcp (or 127.0.0.1), same JSON shape, same bearer header (the port and lack of TLS are the only difference — the bearer requirement above still applies even to loopback). GET /mcp without a body returns the discovery/health card (server name, transport, protocol version) and needs no auth, so it is the quickest way to confirm the service is up before wiring the full config.

MCP resources

Two read-only resources are read at session start (McpService.cs:1128-1173): ledger://manifest (what's indexed and freshness per repo) and ledger://global-arch (the GLOBAL_ARCH.md system spec).

The root argument — searching your own worktree

The main index mirrors main. query_ledger, read_span and symbol_lookup each accept an optional root: the absolute path of a registered checkout overlay (a worktree/workspace, e.g. /root/code/motmaster-openbrain/.claude/worktrees/<id>, or a ws-*/wt-* checkout). When set and registered, a file that differs between that checkout and the indexed mirror is served from the checkout instead — its uncommitted or unmerged edits become findable within minutes — and overlay hits are marked ⊕ from your checkout. symbol_lookup's symbol projection is still built from the main index only, so a registered root there gets a note that overlay symbols aren't in that projection yet and matches keep coming from the main index. find_definition / find_references / find_implementations take the same idea via root too, but there it names a known repo id or any registered worktree path, defaulting to Precise:DefaultRoot — not the overlay mechanism, the precise lane's own workspace resolution.

A root must be registered first: POST /api/admin/overlay/register {root} (loopback-only). An unregistered or unaccepted root, or the overlay being disabled, is ignored with a one-line hint rather than an error — a missing registration never turns a good query into a failure. A PostToolUse hook (motmaster-infra/claude-hooks/overlay-refresh.sh) fires this automatically after an Edit/Write/ MultiEdit inside an accepted checkout shape, debounced to one refresh per root per 30s — see Governance layer for the hook wiring.

What tools/list advertises — 25 of 46, today

The ledger declares 46 tools. Measured over 7,657 instrumented calls (the original pre-WP0.8 baseline), eight accounted for 98.9% of use — query_ledger alone was 84.8% — and find_similar had never been called once, in those 7,657 calls or in 69 local transcripts.

That surface is not free: every advertised schema is carried in the model's context for the whole session, and before its first call the model must fetch those schemas (see schema preload below). The tail was costing more than it earned, so tools/list trims to a CoreTools set (McpService.cs:264-279) — but that set itself grew on 2026-09-12 (WP0.8): eight reference/refactor-safety tools that were previously hidden were un-hidden after the change-safety suite found they were never being exercised while invisible (docs/LEDGER_SCALE_PLAN.md §10 WP0.8). Two independent gates decide the final advertised list (McpService.FilterTools, McpService.cs:312-323): Mcp:AdvertiseAllTools bypasses the CoreTools trim, but the three precise-lane tools are hidden whenever Precise:Enabled=false regardless of that flag — an advertised-but-non-functional tool is worse than an absent one.

The full MCP tool catalog

All 46 are registered in tools/list and dispatched by name in McpService.cs; the ones not in the advertised set above are callable but not advertised. A read-focused subset is also exposed to the in-app chat loop via ExecuteToolForChatAsync.

Span-first hits — every result leads with where the code is

A query_ledger hit used to open with its intent sentence and a labelled metadata row, and on the default path carried no line coordinates at all: an agent had to read to the bottom of a block to learn it could not navigate from it. Span-first reverses that order — path L<from>-<to> becomes the hit heading, the intent drops to the first bullet, and the metadata collapses to one unlabelled separator-delimited row whose schema is stated once in a legend line at the top of the response.

Nothing is dropped: repo, layer, pattern, chunk type, confidence band and percent, build sha, graph link, snippet, graph footer and the ```json cited block all survive. Removing the repeated labels more than pays for the coordinates and the legend, so the response is smaller — replaying the 20 frozen token queries over the t0 snapshot gives −3,902 bytes (−3.14%), about −49 estimated tokens per query, assuming a pessimistic 11-character coordinate on every hit. Config-gated on Mcp:SpanFirstLayout, so the flag flip is the single attributable change.

The cited block — ledger_id, start_line, end_line, indexed_sha

Every rendered response that cites files closes with a fenced ```json cited block — one entry per distinct file, deduplicated. Beyond the always-present repo/file pair, an entry carries whichever of these the call site actually had a real hit object for (a SearchResult or SymbolMatch, never fabricated): ledger_id (the chunk's stable identity — survives a file rename), start_line/end_line (the coordinates actually shown in the rendered heading, so read_span lands on the same lines the agent read), and indexed_sha (the commit the chunk was indexed at — SymbolMatch hits never carry this field). This is what lets follow-up tooling — the audit script's per-hit follow rate, the Ask drawer's "View code" button — survive a file rename instead of matching on a bare path.

Compact defaults — Mcp:DefaultFormat / Mcp:DefaultLimit (toon is live)

An explicit format/limit argument on a query_ledger call always wins, but absent one, it resolves through two optional config keys instead of a hardcoded "markdown"/5: Mcp:DefaultFormat ("markdown" or "toon") and Mcp:DefaultLimit (1–20, clamped). An invalid value (unparseable, or not one of the accepted formats) falls back to the code default and logs once rather than failing the call.

Both keys are no-op in the shipped appsettings.json code defaults — but the live production config is not the code default: at 19:08 UTC on 2026-09-12 (deploy #4, docs/LEDGER_SCALE_PLAN.md §12), the live values were flipped to Mcp:DefaultFormat=toon, Mcp:DefaultLimit=3, and verified live by a query_ledger call with no limit argument returning 3 toon rows. So today, any caller that omits format/limit gets the dense toon layout at 3 rows, not 5-row markdown — still overridable per call. The change is being judged by the nightly per-hit follow rate.

symbol_lookup hands over the definition, not an errand

On an exact match symbol_lookup already knew the file and the line range — and then printed a literal instruction, Read it: read_span(...), telling the agent to spend its next turn fetching what the ledger could have handed over in the one it was already paying for. Seven of the twenty recorded read_span calls in the real-session benchmark followed that line. The tool was manufacturing its own follow-up turn.

With Mcp:InlineSymbolSpan on, an exact match inlines the definition itself — read from the source file, not the indexed chunk, so it is immune to the chunk cap that leaves large members internally truncated in the corpus — padded by Mcp:InlineSymbolSpanContextLines (default 20) on each side and bounded by the same Retrieval:ReadSpanMaxLines cap (default 400) that read_span would have applied. The whole inline section of a response is further bounded by one read_span's worth of lines, so a 20-match lookup cannot cost more than the turn it removes.

Preloading the tool schemas

Claude Code defers MCP tool schemas by default: the tools arrive as names only, and before its first call the model must spend an entire turn calling ToolSearch to fetch a schema. Only then can it make the call it wanted to make. Measured over the 21 recorded ledger-arm sessions in the real-session benchmark:

The control scores zero because it loads no MCP server at all, so it has nothing to defer — the deferral was a fixed startup tax on one side of a two-arm comparison, measuring the harness rather than the ledger. Every session that opens by consulting the ledger, which is what CLAUDE.md instructs every session to do, pays that turn. Setting ENABLE_TOOL_SEARCH=false — as an environment variable in front of claude, or as an env key in ~/.claude/settings.json — removes it. Full guidance and accepted values: docs/TOOL_SCHEMA_PRELOAD.md.

Benchmark answer keys are excluded server-side

The benchmark's reference answers were once excluded only as a caller argument: BenchRunner passed exclude_paths on its own calls, nothing else did, and exclude_paths is unadvertised, so no real session ever sent one. On the 2026-08 real-session run, 6 of 21 ledger sessions were served answer-key prose as ranked query_ledger hits. Worse, the documents had already been deleted from the sources tree — ingest is upsert-only and nothing walks Qdrant asking whether a file still exists, so about 19 chunks outlived their files by weeks and kept being retrieved.

Retrieval/BenchDocGuard.cs makes it a server rule instead, applied inside the retrieval path regardless of what the caller passed: hybrid search merges the server list into its exclusions (a union, so the bench keeps its own), symbol_lookup applies it per point because it never went through hybrid search at all, and read_span refuses by path because it answers from disk where no index change can reach it. Retrieval:ExcludeBenchDocs defaults true — this is an integrity fix, not an experiment, so the safe state is the default state, the flag exists only for emergency rollback, and an unparseable value degrades to on rather than throwing inside search. A localhost-gated POST /api/admin/qdrant/purge-bench-docs then deletes every matching point, all versions and deliberately not is_latest-filtered, because a superseded version still carries the full answer-key text in its payload.

PR pre-flight — asking the ledger what a diff disturbs

tools/pr-preflight.sh takes a PR, asks the live ledger what the diff touches, and prints a report: affected callers with real file:line anchors, cross-repo impact, invariants touched, and a call table recording every request it made with its outcome. It is print-only by default — posting a comment requires an explicit --post — costs $0 and calls no model. A run where a ledger call had failed exits non-zero and renders that section as "Could not ask the ledger", never as a clean empty. The acceptance capture against a real merged PR is docs/PR_PREFLIGHT_ACCEPTANCE.md.

policy_check — the risk-tiered gate

Given a proposed action (tool + optional path + intent), policy_check returns a machine-readable allow | deny | review verdict an MCP gateway or hook consults before the action runs. It is backed by the pure, testable PolicyService (Services/PolicyService.cs): ReadOnly → allow (every discovery tool; unknown tools default here); Write → allow-but-flagged (feedback, mark_visited, bench_record, fix_mark_status); HighRisk → review (sync_ledger + configured). The tier map is config-overridable and UNIONed onto defaults; ToWire emits stable lower-case strings; and the verdict attaches the relevant invariants for the target path. It is documented in full on the Team analysis page.

Token-efficiency benchmark (bench_*)

Proves and measures that answering via the ledger costs fewer tokens than reading files directly, as a real A/B. Benchmark/BenchRunner.cs runs each pending discovery question through two isolated agentic loops with HARD-separated tool sets: (A) LEDGER = a single query_ledger tool hitting the local MCP endpoint (http://127.0.0.1:5005); (B) DIRECT = grep/read over /opt/arch-ledger/sources/current. It records each path's real token usage (input/cache_read/cache_write/output) from Anthropic API usage blocks, with loops capped at 40 turns / 4096 tokens. An opt-in correctness gate (BenchGate, off by default via Benchmark:Judge:Enabled) enforces that a cheaper answer only counts as a win if it is BOTH strictly cheaper AND at least as correct (an LLM judge). Results persist in the Benchmark Qdrant collection, and the system-wide ratio is surfaced by EfficiencySummary in TeamMetricsService. The in-session drain flow (bench_list → bench_get_task → run both ways in fresh contexts → bench_record) is $0 and deterministic; the real-API A/B spends money, is gated on Anthropic:ApiKey, and only runs on an explicit trigger (model default claude-opus-5).

Deployment topology

An agent posts to https://brain.motlogs.com/mcp (HTTPS + JSON-RPC + Bearer) → nginx (TLS terminator) → 127.0.0.1:5005 .NET 10 service. The publish is a self-contained linux-x64 binary, shipped as a tarball via scp over SSH. The systemd unit (Deploy-Hetzner.ps1:573-591) sets WorkingDirectory=/opt/arch-ledger, ExecStart=/opt/arch-ledger/MOTLogs.ArchitecturalLedger, Restart=always, ASPNETCORE_URLS=http://*:5005; the service name is arch-ledger. The nginx vhost (:120-181) redirects 80→443, terminates TLS 1.2/1.3 with a wildcard cert, and proxies to http://127.0.0.1:5005 with WebSocket upgrade + forwarded headers, while location /api/admin/ { return 404; } blocks admin at the proxy (loopback/SSH-tunnel only). Deploy flags include -NoNginx, -ConfigureNginxOnly, -RestartOnly, plus source-sync-only and transcript-sync-only modes; the graphify CLI is installed on-host via uv tool.

Public API access gate (WP0.7)

Every /api/* and /hook/* request now passes one gate (Program.cs, registered beside the audit-trail middleware, delegating its actual decision to the pure, unit-tested HttpAccessGate.IsApiRequestAuthorized in Services/HttpAccessGate.cs — trusted-peer loopback/tailnet detection, public-path matching, and bearer/registry resolution are each a standalone testable predicate there, not inline logic only a booted host could exercise): a small Security:PublicPaths allow-list (/api/health, /api/ask, /api/symbols, /api/gate-health by default — the latter two already self-gate or were already public, so this just leaves them alone) is let through with no check; everything else needs a trusted loopback/tailnet peer (Security:TrustLoopback / Security:TrustTailnet, both default true) or a bearer — either the shared Mcp:BearerToken or a TokenRegistry-registered per-developer token, the same rule POST /mcp already applies — presented as Authorization: Bearer … or ?token=…. This closed the previous hole: GET /api/todos had no gate of its own and returned the full cross-repo findings list to anyone on the internet. Per-route checks that already existed (IsLocalhost/IsGraphAuthorized) are unchanged and still run afterwards. Because /api/stats is deliberately not on the allow-list, the dashboard (index.html, script.js, admin.html, app.html) now needs a token to load unless it arrives over an SSH tunnel/tailnet session (trusted peer, no token needed). Per decision D8 the dashboard is bearer-only rather than exempted, so every page routes its /api/* calls through wwwroot/token.js's authFetch: it attaches Authorization: Bearer … once a token is known, accepts one the same way the graph viewer always has (?token=<Mcp:BearerToken>, stored to localStorage and stripped from the URL on read), and on a genuine 401 shows a small one-time banner asking the viewer to paste the token, then retries the failed call once. The token itself is never embedded in any served page or script. Security:Enforce=false in the runtime settings file is the documented rollback if this gate ever needs to come out without a redeploy.

Qdrant operations

The image is PINNED to qdrant/qdrant:v1.15.4 (1.16.x panics on scout_vehicles.location — an empty geo index unwraps None at gridstore.rs:333), runs with --ulimit nofile=1048576 (the default 1024 panics), and uses two bind mounts — storage (/var/lib/qdrant) and snapshots (/var/lib/qdrant-snapshots, else snapshots vanish on docker rm) — with loopback-only ports 6333/6334 and restart-always. The host needs 125 GB RAM and ≥16 GB swap (scout_vehicles peaks ~118 GB). The ledger collection motlogs_unified_architecture (~2,241 points, named vectors code_vector/design_vector, 384-dim Cosine) is tiny next to scout_vehicles (135M points / 686 GB).

Re-ingest / backfill runbook

A cache-preserving rebuild re-chunks and re-embeds without paying for the Groq analysis cache again.

1. Pre-flight — verify the commits on the host binary, that repo paths point at motmaster-*, that Qdrant is healthy, and take a snapshot.
2. Rebuild — POST /api/admin/qdrant/rebuild (guarded — NEVER clear-collection; returns 202).
3. Monitor — /api/admin/ingest-status (2–4h at MaxParallelism:2, watching BudgetSkippedChunks).
4. Post-passes — POST /api/admin/concepts/run + /api/admin/cross-stack/run (materialises the G1 controller→sproc edges).
5. Measure — GET /api/admin/eval/relevance (the 24-question golden set) as the no-regression gate.
6. Snapshot — take a fresh snapshot.

Byte-identical content stays Groq-cold-free (~$0). The high-risk sync_ledger MCP tool triggers the background re-ingest programmatically.

Enforcement hook

Every agent is nudged or forced to consult the ledger before touching code, via a PreToolUse hook. As of 2026-09-13 08:19 UTC this is a single, user-level, versioned wiring — not per-repo copies — installed from motmaster-infra/claude-hooks/. Full detail, including the current gate mode, the A/B under way, and /api/gate-health, is on the Governance layer page; this section states only what's relevant to the MCP surface itself: the ledger MCP tools are always exempt (mcp__mot-ledger__* never gates, or the gate would be unsatisfiable), and every non-ledger MCP tool is gated the same as Read/Grep/Bash/Edit/Write (pattern ^mcp__(?!mot-ledger__).+) — so a different MCP server's tool is subject to the same consult requirement as a file edit.

Open Brain — the MOTMaster Architectural Ledger · generated from source 2026-07-07