Memory & intelligence

Context saving

Distinct from source ingestion, this is the ledger's working memory: it captures, redacts, stores and recalls what agents and developers actually did across Claude Code sessions — memory that survives context-window compaction and process restarts, so nobody re-discovers a settled answer from scratch.

For the measurement side — the two-path benchmark behind the published token saving, how noisy it is, and why the mean judge score is now published beside the correct-answer counts — see the Retrieval tuning log.

Hosted / periodic session ingest of transcripts

Background loops walk every transcript on the host (~/.claude/projects/**/*.jsonl) into durable, queryable memory with no manual trigger. TranscriptReader.ReadAll (SessionIngest/TranscriptReader.cs:29) yields one RawTranscriptSession per .jsonl, extracting start/complete timestamps and filtering by since (:61-93); it is defensive — unreadable files and lines are silently skipped (:75-85) so schema drift never breaks a run. The root is SessionIngest:TranscriptsRoot (default ~/.claude/projects, SessionIngestService.cs:43-46). Three loops run:

LoopServiceDefaultWhat it does
A — session outcomesSessionIngestHostedService every SessionIngest:IntervalHours (24), HostedEnabled master switch Builds a SessionOutcome per session, embeds its summary, upserts. On failure LastRunUtc isn't advanced so the window retries; a successful run with new sessions chains skill-candidate discovery (:122-137).
B — conversation memory + decisionsConversationIngestHostedService opt-in ConversationMemory:Enabled (false), 240s boot delay Calls ConversationIngestService.IngestAsync.
C — decision distillationDecisionMemoryHostedService opt-in DecisionMemory:Enabled, 180s boot delay Calls DecisionExtractionService.ExtractRecentAsync.

On-demand triggers: POST /api/admin/session-ingest?days=N (Program.cs:1392), /api/admin/conversation/ingest (:631), /api/admin/decisions/extract (:624).

On Hetzner the transcripts root must be /root/.claude/projects. Both memory loops (B and C) default OFF, and every Groq pass is budget-capped, so a host is never surprised by spend.
Why it matters

Memory stays fresh automatically, embedding is local and free, and the opt-in defaults mean context-saving never silently costs money or ingests something an operator didn't intend.

Session-outcome extraction & boilerplate handling

SessionOutcomeBuilder.Build (SessionIngest/SessionOutcomeBuilder.cs:37) is a pure transform: it captures the first real user turn and last assistant turn as a Summary, budgeting each half independently (MaxFirst=1200, MaxLast=800) so a long opening can't evict the closing summary. StripNoise (:165) removes Claude Code's injected wrapper spans (<local-command-*>, <system-reminder>, <command-name>, regex :33-35) BEFORE selecting the first user turn. It records FilesRead/FilesEdited from Read/Edit/Write/NotebookEdit calls, ToolsCalled, LedgerQueries from query_ledger calls (:84-107), and scans tool_result for commit SHAs (capped 20). The file header states the privacy boundary: "only structured fields below are extracted… No raw prose is stored beyond the bounded Summary field."

Git-based outcome inference is NOT yet wired — Build hard-codes Outcome="unknown" (:155) and no code path updates it. So prior_work labels sessions "outcome unknown" until git-outcome inference lands; the blueprint describes it as intended, not current.
Why it matters

A ~2 KB structured record makes every past session semantically comparable and cheap to store, and the boilerplate stripping is exactly what makes cross-session clustering work instead of matching on injected wrapper text.

Secret redaction — the privacy prerequisite

SecretRedactor.Redact (Services/SecretRedactor.cs:30) uses compiled regexes for provider keys (Groq gsk_, OpenAI sk-, GitHub gh[pousr]_, Slack xox[baprs]-, AWS AKIA…), JWTs, Bearer tokens and generic key=value assignments (:15-28), replacing each with a typed [REDACTED:*]. For key=value it keeps the key name and redacts only the value (:36-43) so meaning survives. It is applied as defence-in-depth at every write path: recall chunks (ConversationMemory.cs:189), condensed decision input (:201,206) and summary-level decision input (DecisionMemory.cs:148,156-157).

Why it matters

Raw transcripts routinely capture keys and connection strings — redaction is what makes storing conversation memory safe at all, rather than turning a memory store into a secret leak.

Conversation recall — recall_conversation

Semantic search over past conversations (redacted transcript spans). ConversationIngestService.IngestAsync (ConversationMemory.cs:163) pulls human-readable turns via TranscriptTextExtractor.ExtractTurns (user/assistant text only, dropping the tool_use/tool_result noise where most secrets live, :23-40), packs consecutive redacted turns into ~ChunkChars (default 1200) windows, and embeds locally. ConversationStore (:75) is a Qdrant motlogs_conversation_chunks collection (384-d cosine) with deterministic MD5 IDs of sessionId#seq so re-ingest updates in place. The recall tool CallRecallConversationAsync (McpService.cs:3942) embeds the query, searches with an optional developer filter, drops hits below ConversationMemory:MinRelevance (default 0.30), and renders dated redacted snippets.

Why it matters

Long-term recall outlives a single context window, is scoped per-developer when wanted, and is safe because only redacted prose is ever stored.

Prior work — prior_work

"Has anyone worked on this before?" — past sessions that touched a file or task, each labelled by how it ended. CallPriorWorkAsync (McpService.cs:1423) uses SessionOutcomeReader (SessionIngest/SessionOutcomeReader.cs:24): ForFileAsync (:99) scrolls motlogs_session_outcomes matching by path suffix, most-recent first; ForTaskAsync (:117) ranks by token-overlap between the task text and each session's summary/lessons/files, robust even when summary vectors are zero placeholders. Each is labelled by PriorWorkVerdict (:1608): merged → ✅ template, reverted → ⛔ AVOID, abandoned → ⚠️ caution, else • unknown — rendered with summary, files edited, lessons, and commit SHAs.

Because session Outcome is hard-coded "unknown" (see above), verdicts read "unknown" until git-outcome inference is wired. The mechanism is complete; only the signal that feeds it is pending.
Why it matters

It turns session history into reusable templates and cautionary tales — so a task that was tried and reverted surfaces as a warning before an agent repeats it.

Decision memory — prior_decisions

"Did we already consider this approach, and why did we choose, reject, or defer it?" — context to revisit, never a veto. DecisionExtractionService (Services/DecisionMemory.cs:120) feeds redacted session summaries and lessons to Groq with a strict prompt returning {topic, stance∈{chose,rejected,deferred}, rationale} JSON, tolerantly parsed by the pure ParseDecisions, budget-guarded, recording spend. DecisionStore (:37) is 384-d cosine with IDs deterministic on sessionId|topic so re-runs update in place. The tool CallPriorDecisionsAsync (McpService.cs:3898) searches with an optional developer scope, drops hits below DecisionMemory:MinRelevance (default 0.35), renders each with a stance icon (⛔/⏸️/✅) and rationale, and closes with explicit guidance that a "rejected" decision is context to challenge, not a hard block.

Why it matters

It prevents re-litigating settled trade-offs while framing memory as advisory — so a stale constraint can be revisited rather than silently obeyed forever.

Session-scoped visited-node ledger — mark_visited / recall_visited

Within a single conversation, an agent records code it has already confirmed and later cites it by path:L#-L# or ledger ID instead of re-pasting it. VisitedNodeLedger (VisitedNodeLedger.cs:27) is in-memory and thread-safe (a ConcurrentDictionary keyed by caller-supplied sessionId), holding a VisitedNode {LedgerId, Path, StartLine, EndLine, VerifiedAtUtc, Symbol, Note}. Record (:57) is idempotent on node identity (DedupKey id:<ledgerId> else loc:path|start|end); re-marking just refreshes the timestamp. Recall (:97) returns most-recently-verified first with an optional filter; an unknown session returns empty, never throws. It is intentionally NOT persisted — ephemeral per conversation — and lives on the singleton McpService (handlers CallMarkVisited :1465, CallRecallVisited :1491).

Why it matters

Naive multi-turn sessions re-include prior code every turn — roughly quadratic token cost. Here a re-reference becomes an id plus coordinates (tens of tokens) not a re-dump (thousands), flattening per-turn context growth on long debugging or refactor sessions.

Drift since — drift_since

Returns only the structural delta of a file since a point in time — symbols added, removed, moved, or signature-changed — computed server-side. CallDriftSinceAsync (McpService.cs:1531) parses an ISO since, reads the "before" snapshot (QdrantService.GetSymbolSnapshotAsync) plus the latest "after", diffs via StructuralDrift.Compute, and renders with RenderDrift.

FORWARD-ONLY: when no retained snapshot predates since (common until versioned retention is enabled), it returns an honest placeholder rather than fabricating a delta (:1555-1564).
Why it matters

"What changed?" comes back as a token-light delta instead of two full-file dumps — and stays candid that snapshot retention is still pending rather than inventing an answer.

LogFusion — runtime log memory

An adjacent context source: the ledger tails the host's journalctl and folds repetitive runtime log lines into templated "log facts" — a memory of what the running system actually does and says. JournalctlTailHostedService (LogFusion/JournalctlTailHostedService.cs) streams lines through JournalctlLineParser; LogTemplateExtractor/LogTemplateHasher collapse variable parts into stable templates; LogFactUpserter persists them. This is runtime/operational context rather than conversation memory, and it feeds both the learning-corpus surface and the log-scan agent.

Why it matters

It complements conversation memory with runtime-behaviour memory, so an agent can correlate "what we discussed" with "what the system actually emitted."

The learning corpus on the dashboard

/api/stats emits a learningCorpus block — agentQueries, sessionsIngested, logFacts, refactorPlans, failedClusters, plus query hit-rate and latency (Program.cs:470-481) — and a hostedSessionIngest tile fed by SessionIngestState.Snapshot(): enabled, intervalHours, lastRunUtc, lastCount, lastDurationMs, lastError, nextRunUtc (:482-491; state SessionIngest/SessionIngestState.cs:10, GET /api/admin/session-ingest/status :1402). Session queries plus per-session query_ledger calls feed skill-candidate discovery (chained :122-137).

Why it matters

It makes context-saving observable and operable — an operator can see at a glance whether the memory loops are alive, when they last ran, and how much compounding "has anyone tried this?" memory exists.

Scope note: Agents/ConversationReviewAgentJob.cs reviews the Agent repo's customer (WhatsApp) conversations to raise UX findings — that is a quality-platform agent (see Analysis & quality), NOT part of the Claude Code developer-context path, despite the similar name.

Open Brain — the MOTMaster Architectural Ledger · generated from source 2026-07-07