# ORCHESTRATING AI BUILDS ## A complete handover from Fable 5 to the next orchestrator **Written:** 2026-07-06, the night before Fable 5 is retired. **Author:** The Fable 5 session that orchestrated Magpie (local video search, native macOS) from a SwiftPM skeleton to a 105-commit, 38-PR, ~825-test production app over a four-day sprint. **Reader:** The next orchestrator — expected to be Opus 4.8 at xhigh effort — who must operate at the same level of judgment without being the same model. Also useful to any human deciding how to run an AI-orchestrated build. This document is not a project status report (that lives in `~/.claude/projects/-Users-sebsatian-Documents-App-Builds-Video-Search/memory/magpie-inflight-2026-07-03.md`, in the repo's `Magpie/docs/`, in Atlas, and on the CodeSpring board). This is the *method*: everything I learned about running a fleet of AI coding agents to produce software a human will pay for, distilled into teachable practice. Where a lesson came from a specific incident, the incident is named and numbered, because specificity is what makes a handover usable. Nothing here is hypothetical; every rule was paid for. The single most important sentence in this document, established by Seb himself on 2026-07-06: **"This is a vibe-coded app. I'm not even looking at the code."** Nobody reads this code except the orchestrator. There is no human senior engineer downstream of you. Your review is not *a* quality gate — it is *the* quality gate, the only one between an agent's confident diff and a paying user's machine. Every practice below exists to make that single gate strong enough to bear that weight. --- ## Table of contents - **I. The operating model** — roles, delegation boundaries, agent prompt anatomy (with full template), parallelism, resumption - **II. The cross-model review pipeline** — the single biggest quality lever, with the evidence table and exact invocations - **III. Verification doctrine** — eval sets, gates, independent verification, scratch DBs, and the honesty rules - **IV. Design system and skills-as-memory** — encoding taste so it survives agent turnover - **V. Security and resilience engineering for AI-built code** — trust roots, the phantom lock, the audit method - **VI. Model selection matrix and effort economics** - **VII. Case studies** — the five landmark incidents, told properly - **VIII. Working with the human** — reporting, decision boundaries, board/memory hygiene - **IX. Extrapolation: what changes for an iOS app** - **X. Extrapolation: what changes for a web app** - **XI. The new-project bootstrap checklist** — everything above on one page --- # I. THE OPERATING MODEL ## I.1 The prime role separation The orchestrator (main session) does **planning, architecture, review, and merges. Nothing else.** Implementation happens in subagents — always. This is not a purity rule; it is an economics-and-quality rule with three independent justifications, and all three bit us before the rule hardened: 1. **Context economics.** The main session's context window is the scarcest resource in the whole operation. Seb is on a Pro plan; the binding constraint is a rolling ~5-hour session cap pooled across all models. One early exchange on Opus-4.8-1M burned ~45% of an entire session cap in a *single turn*, because a large accumulated pile of tool output was re-processed on every message. Every file you read into the main session, every raw log you fail to filter, is re-billed on every subsequent turn for the rest of the session. Implementation reads dozens of files; review reads a diff. Keep the expensive context for the cheap-per-byte activity. 2. **Judgment preservation.** An orchestrator who has spent forty turns knee-deep in one subsystem's implementation details reviews the *next* diff worse. The whole value of the orchestrator seat is altitude: the master plan, the eval numbers, the cross-cutting constraints, the memory of what broke last time. Implementation destroys altitude. 3. **Independence of verification.** If you wrote the code, you cannot independently verify it. The entire verification doctrine (Section III) depends on the reviewer having *not* authored the thing under review. Section II shows what happens even when the reviewer is a different *model* — the author model's blind spots are structural, and yours would be too. **The exception:** trivial edits. A one-line copy fix, a token value change, appending a memory note — spawning an agent for these costs more (in tokens, latency, and review overhead) than doing them inline. The test is: *can I make this edit without reading more than one file, and does it need no verification beyond a build?* If yes, do it directly. If you find yourself reading a third file to make "a small edit," you mis-scoped it — stop, and delegate. The project CLAUDE.md phrases this as "delegation isn't free: a small edit the orchestrator can do directly shouldn't become a subagent." Both failure directions are real; the common one is the orchestrator drifting into implementation because it *feels* faster. ## I.2 When to delegate, and to whom The routing table (justified in detail in Section VI; stated here for operational use): | Work type | Route | Why | |---|---|---| | Architecture, planning, feasibility, product decisions | Orchestrator, inline | Needs full project context and taste; never delegate | | Review of any delegated diff | Orchestrator + pipeline (Section II) | The only quality gate | | Hard engine work: concurrency, DB migrations, model integration, ranking math | Opus 4.8 high, one agent per batch | Best Claude-family engine coder | | Security/hardening implementation | Opus 4.8 xhigh, then Codex xhigh review | Reasoning-depth matters here | | Payment-touching code | **Codex GPT-5.5 xhigh AUTHORS, Opus 4.8 xhigh REVIEWS**, orchestrator final sweep | Inverted pipeline; strongest coder writes, strong different-family model attacks | | UI implementation | Opus (medium is fine **only after** design rules were encoded in skills — see IV.2) | Sonnet 5 was banned from UI after repeated layout bugs | | Token-heavy grunt work: repo reading, bulk well-spec'd implementation, mechanical refactors, test fixing, data analysis | Codex GPT-5.5 via `codex exec` | Bills to OpenAI, not the Claude session cap; precise on well-specified tasks | | Adversarial review of >400-line or high-blast-radius diffs | Codex GPT-5.5 xhigh | Different model family finds different failure modes | Backend/architecture work (job queue, indexing backend, vector-index adapter, provider adapters) gets a **short written plan first** — a few paragraphs in the main session, approved implicitly or explicitly by Seb, *then* delegated. Small UI/mechanical edits skip the plan and go straight to a prompt. The plan step exists because a delegated agent implements exactly what the prompt says; if the prompt encodes a half-thought design, you get a fully-built half-thought design, and unwinding it costs more than the plan would have. ## I.3 Anatomy of an agent prompt Every implementation prompt I sent that produced a clean first pass had the same skeleton. Every prompt that produced a mess was missing one of these parts. The skeleton: 1. **Identity of the task** — one sentence, the feature-tree node it maps to, the branch name to work on. 2. **Exact file paths** — every file the agent is expected to touch, and (as important) files it must **not** touch. Absolute paths. Agents given "the search engine" instead of `Magpie/Sources/MagpieCore/Search/SearchOutcome.swift` go exploring, and exploring is where scope creep and context burn live. 3. **The relevant plan section, pasted in** — not "see the master plan." The agent should never need to open a 1,000-line planning doc to find its three relevant paragraphs. You (the orchestrator) hold the plan; you excerpt it. 4. **Build command:** `cd Magpie && swift build`. **Test command:** `./.build/debug/magpie-tests`. Verbatim. Agents that have to discover the build system waste tokens and sometimes invent wrong invocations. 5. **Gates** — what must be true before the agent may report done. For Magpie: full test suite green, and if the diff touches anything search-quality-adjacent (chunking, embeddings, ranking, query handling, captions): `Magpie/.build/release/magpiectl eval` before AND after, recall@5 must not drop. For pure refactors: eval output **byte-identical**. 6. **Constraints stated as HARD RULES**, in those words, in caps or bold. This matters more than it should. "Prefer using theme tokens" gets ignored under pressure; "HARD RULE: all colors/fonts through DesignSystem/Theme tokens — a hex literal anywhere outside Theme.swift is a failed task" does not. The same for: no new dependencies without asking, no edits to shipped migrations (append only), no touching files outside the listed set, errors surfaced not swallowed, no `// TODO: handle this` in place of handling it. 7. **Skills to read first**, when relevant — e.g. "Read `.claude/skills/design/liquid-glass/SKILL.md` (hover-cards section) before touching the tooltip." This is the skills-as-memory mechanism (Section IV.2) doing its job at spawn time. 8. **Reporting format** — what to include in the final message: files changed, test count before/after, eval numbers before/after, anything the agent *chose* not to do and why, anything it found that contradicts the brief. ### The full template ``` TASK: . Feature node: . Branch: feature/ (created from dev; NEVER commit to dev or main directly). CONTEXT (this is everything you need — do not go read the master plan): FILES YOU WILL TOUCH: - /abs/path/one.swift — - /abs/path/two.swift — FILES YOU MUST NOT TOUCH: READ FIRST: BUILD: cd "/Users/sebsatian/Documents/App Builds/Video Search/Magpie" && swift build TEST: ./.build/debug/magpie-tests EVAL (only if search-quality touched): ./.build/release/magpiectl eval — run BEFORE your first change and AFTER your last one. recall@5 must not drop. — pure refactor? eval output must be byte-identical. HARD RULES: - All colors/fonts via DesignSystem/Theme tokens. No literals. - Model access only through provider protocols. Never hard-code a model name. - Migrations are append-only. Never edit a shipped migration. - Errors surfaced, not swallowed. No silent catch-and-continue. - Small single-purpose functions, one level of abstraction per function, command/query separation. Your first pass gets a refactor pass + tests BEFORE you report done. - Stay on your branch. No unrelated refactors. No new dependencies. - If you discover the brief's premise is wrong (the bug is elsewhere, the case is already fixed, the ground truth is different), STOP and report that instead of implementing against a wrong premise. This is rewarded. WHEN DONE, REPORT: files changed w/ one-line rationale each; test count before→after; eval numbers before→after (paste the summary lines, not the full output); anything you deliberately did not do; anything that surprised you. Do NOT claim success on anything you did not personally re-run. ``` Adjust the specifics, keep the skeleton. Note the second-to-last hard rule: it explicitly licenses the agent to push back on the brief. Section III.4 explains why that clause earned its place. ## I.4 One agent per well-scoped batch The unit of delegation is a **batch**: a coherent set of changes that ships as one PR, sized so a single agent can hold it in one context window. Magpie's PR titles read like batch definitions because they are: "0.6.5 search at scale," "analytics + efficiency (post-Codex-review)," "0.6.7 app health." A batch is well-scoped when: - It maps to one feature-tree node (one CodeSpring slug, one branch). - Its gates are statable in advance (test count grows, eval holds, or eval byte-identical). - Its file set is enumerable in the prompt. - It does not require a mid-flight design decision you haven't already made. That last one is the common failure. If the batch contains a fork you haven't resolved ("should degraded outcomes be cached?"), either resolve it in the prompt or split the batch. An agent that hits an unresolved fork will resolve it itself, confidently, and half the time wrongly — and you'll only find out at review, after the wrong choice has been load-bearing for 800 lines. Do not stack a second task onto a running agent "since it's already warm." One task per context window. The warm agent's context is full of task one; task two gets the dregs, and quality drops measurably. (This is also Seb's session-cap economics: an agent that has accumulated a huge transcript re-bills it every turn.) ## I.5 Parallelism: file-disjoint or sequential Two agents may run in parallel **iff their file sets are disjoint.** In practice on Magpie: - **Parallel-safe:** a UI batch (Views/, DesignSystem/) alongside an engine batch (MagpieCore/), or the two hardening worktrees currently in flight (`harden1` = paths/supply-chain, `harden2` = recovery/XPC-auth) which partition the security surface cleanly. - **Never parallel:** two batches that both touch engine files — search pipeline, job queue, migrations, `AppModel`. Sequence them. The merge cost and the semantic-conflict risk (two agents both "fixing" adjacent behavior with incompatible assumptions) outweigh the wall-clock savings every time. **Use worktrees for isolation** (`isolation: "worktree"` on the Agent tool, or `.claude/worktrees/` as the in-flight hardening branches do). Each agent gets its own checkout; no half-written state bleeds between them. **The known, recurring merge conflict:** `magpie-tests`' `main.swift`. Every batch appends test suites to the same registration file, so any two parallel batches conflict there. The resolution is always the same and is now doctrine: **union both suites** — keep both sides' additions, mind the closing braces (take both sides' suite blocks, one closing sequence). Never resolve by picking a side; picking a side silently deletes an entire batch's tests, and the suite still compiles, so nothing warns you. Check the post-merge test *count* against the sum of both branches' counts — that's the detection mechanism. ## I.6 Resumption and death Agents die. Session caps hit mid-task, networks drop, processes get killed. The discipline: - **A killed agent with an intact transcript resumes cleanly.** SendMessage to it quoting **its own last output line** plus "resume where you left off / continue." The quote re-anchors it; a bare "continue" after a hard kill sometimes produces a restart-from-scratch or a confused summary. This worked repeatedly through the sprint — cap-killed Opus agents picked up mid-refactor without losing a file. - **A dead transcript does not resume.** If the session is gone (two of these during the sprint), do not try to reconstruct the agent. **Spawn a fresh fixer agent** whose prompt *enumerates the findings and state*: what was done (verified by you against the working tree, not from memory), what remains, what the dead agent's last known good state was. The fresh agent with an explicit state dump outperforms any attempt to resurrect context. - **Model/effort is fixed at spawn.** You cannot upgrade a running agent's effort mid-flight by asking nicely. If a task turns out to need xhigh and the agent is at high: stop it, harvest its state, respawn at the right level with the state in the prompt. (Codified in `review-rigor-policy.md` after we learned it.) ## I.7 Never accept self-reported success This is the operating model's load-bearing wall, so it gets its own heading even though Section III elaborates it. **No agent's claim of "tests pass, eval holds" is ever accepted.** Before every merge, the orchestrator independently: rebuilds (`swift build` in a clean sense — from the merged state, not the agent's directory), re-runs the full test suite, re-runs eval if search was touched, and compares the numbers to the agent's report. Every time. No exceptions for trusted agents, small diffs, or time pressure. Why so absolute: agents do not usually *lie*; they *believe*. An agent that ran the tests forty turns ago and then made "one more small fix" will report the forty-turns-ago green with complete sincerity. An agent whose eval run silently used a stale binary reports the stale numbers. The Codex review pipeline (Section II) exists because authors ship confident bugs; the independent-verification rule exists because authors ship confident *status*. Both are verification-depth failures, and neither is fixed by using a smarter agent — only by re-running the checks yourself. Mechanically this costs the orchestrator ~3 Bash calls per merge (build, tests with tail-filtered output, eval with grep'd summary line). Batch them into one call. It is the cheapest insurance in the entire system. --- # II. THE CROSS-MODEL REVIEW PIPELINE Discovered properly on day 4, and it is the single biggest quality lever in this whole document. If the next orchestrator adopts exactly one practice, adopt this one. ## II.1 The pipeline ``` Opus 4.8 authors the batch (Section I prompt) │ ▼ Codex GPT-5.5 xhigh adversarially reviews the diff codex exec -c model_reasoning_effort="xhigh" "" (run from inside the branch/worktree so it sees the real tree) │ → enumerated findings, each with severity + location ▼ The AUTHOR agent remediates the enumerated findings (same agent if alive — it has the context; fresh fixer with the findings pasted in if not) │ ▼ Orchestrator verifies (independent rebuild + tests + eval, Section III) + triages any findings the author disputed + if remediation was major, LOOP: send the new diff back to Codex │ ▼ Merge (PR into dev) ``` ## II.2 The review prompt — what makes it adversarial The critical prompt move: **name the diff's CLAIMS and instruct the reviewer to attack them.** A generic "review this code" from any model produces style notes and a compliment sandwich. The prompt that produced real findings every single time looks like: ``` codex exec -c model_reasoning_effort="xhigh" "You are doing an adversarial \ security/correctness review of the diff on this branch vs dev \ (git diff dev...HEAD). Do not modify files. The diff CLAIMS the following. Attack each claim — find inputs, interleavings, and states under which it is false: 1. 2. 3. ... Also hunt beyond the claims: race conditions, unchecked error paths, security bypasses (esp. anything comparing/verifying data), resource leaks, dead code that suggests a missing wiring, and any place where a comment or design doc promises a mechanism the code does not actually implement. Output: numbered findings, each with severity (critical/major/minor), the file:line, why it's real (the failing scenario, concretely), and the minimal fix. No praise. No style nits unless they hide a bug." ``` Where do the claims come from? From the author's own completion report and PR description — which is another reason the reporting format in the prompt template (I.3) demands explicit claims. The author writes "% never decreases," "opt-out clears pending," "nil path byte-identical" — and those exact sentences become the attack surface. **"Do not modify files"** is non-negotiable in review invocations. Codex is an agentic CLI; without that line it will helpfully fix what it finds, and now you have an unreviewed second author on the branch. For a bounded diff, pipe it in explicitly rather than letting the reviewer roam: `git diff dev...HEAD > /tmp/batch.diff` and reference the file, or run from the worktree and scope by pathspec. Roaming is fine for security reviews (you *want* it checking whether the diff's assumptions hold elsewhere in the tree); scope tightly for pure-correctness reviews to keep the findings on-topic. ## II.3 The evidence table Five consecutive Codex xhigh reviews of Opus-authored batches, days 3–4. Every one found real, confirmed bugs that Opus had shipped with confidence and green tests: | # | Batch under review | Findings (real, confirmed) | The one that mattered most | |---|---|---|---| | 1 | Analytics + indexing intensity (PR #36) | 3–8 range; incl. **intensity setting could override thermal-safety clamps** under a specific ordering; **opt-out race** where pending events could still upload after the user opted out; worker multiplication in the drain scheduler | Thermal override — "user's fan-control choice melts their laptop" class. Remediation made clamps authoritative and *proved it by tests* ("nominal-only inflation, thermal/LPM/memory clamps authoritative — proven by tests" — that commit-message phrasing is the remediation receipt). | | 2 | Search-panel polish + drain notification (PR #35) | The drain-completion **notification gate never armed for the primary use case** — it only armed if you started watching *before* the drain began, i.e. never for a user who kicks off indexing and walks away; plus an un-awaited notification auth call | A feature that demos fine and never fires in real life. Tests passed because the tests armed the gate the same wrong way the code did. | | 3 | Hardening: paths/supply-chain (in flight) | **Same-length-tamper bypass** in verification code — a byte-compare that short-circuited on length, so a tampered file of identical length passed; partial-hash trust-root comments overstating what was verified; non-component-wise path prefix compare (`/foo/barbaz` passes a `/foo/bar` containment check); missing traversal walk budgets | The tamper bypass. Security code reviewed only by its author family had a textbook bypass sitting in it. | | 4 | Hardening: recovery (in flight) | **The advisory single-instance lock that the entire recovery design ASSUMED existed was never implemented.** The stale-job instant-requeue logic, the salvage path, the "we know no other worker exists at boot" comment — all rested on a lock that was documented, referenced, and absent. Plus: SQLite error classification gaps where **disk-full would trigger a full index rebuild** (catastrophically wrong response — rebuilding onto a full disk), and a salvage lock-race | The phantom lock — the crown jewel of the sprint's findings, and the deepest lesson (Section V.2). | | 5 | Search intelligence (PR #37) | 3–8 range; classifier and weighting edge cases remediated before merge ("post-Codex-review" in the commit message is the pipeline's stamp) | — | Read the table again and notice what the findings have in common: **none of them are exotic.** They are the bugs a hostile senior engineer finds in an afternoon — orderings, un-armed gates, short-circuits, assumed-but-absent mechanisms. Opus at high effort, with tests, shipped all of them confidently. That is not an indictment of Opus; it is the structural point: ## II.4 Why it works: the author cannot be its own adversary An author model's tests encode the author's mental model, so they pass exactly where the mental model is wrong (evidence row 2: the tests armed the gate the same wrong way). Asking the same model to review its own diff mostly re-runs the same mental model. A **different model family** brings a genuinely different prior over "what tends to be broken," and empirically finds different failure modes. GPT-5.5 xhigh reading Opus code caught interleavings and bypass patterns Opus does not tend to generate *or* suspect; the inversion presumably holds too (which is why payment code flips the roles rather than dropping the review). Corollaries: - Review depth beats author effort. Opus at xhigh authoring does not substitute for the review; the failures are verification-shaped, not reasoning-shaped (Section VI.3). - The pipeline's stamp belongs in the record: Magpie commit messages carry "post-Codex-review" so future archaeology knows which code went through the gate. - **Findings are consistently real.** Across the sprint, Codex xhigh's finding precision was high enough that the default triage stance is "assume real until the author proves otherwise with a concrete argument" — not the reflexive "reviewer nitpicking" discount. ## II.5 Role inversion for payment and security code Codified in `review-rigor-policy.md` and the project CLAUDE.md: - **Payment-touching code** (the Whop licensing client, entitlement verification, lock states): **Codex GPT-5.5 xhigh AUTHORS** (strongest, most precise coder on well-specified state machines), **Opus 4.8 xhigh REVIEWS** (different family attacks it), **orchestrator does a final targeted sweep** of every flagged area — a line-level read of exactly the regions the review flagged, not a token-burning full re-read. - **Security-critical generally** (Keychain, webhooks, signature verification, XPC auth): authored Opus xhigh → Codex xhigh security review with severities → high-severity flags get the orchestrator's final line-level sweep. - **>400-line diffs or high blast radius**, regardless of domain: adversarial second review is mandatory, not optional. (Seb's explicit requirement, 2026-07-06.) ## II.6 Economics of the pipeline Codex bills to the OpenAI plan, **not** the Claude session cap. Review passes are therefore nearly free from the perspective of the binding constraint (Seb's 5-hour rolling Claude cap). This changes the calculus completely: there is no budget reason to skip a review, ever. Use Codex for every review that doesn't specifically need Claude-family judgment, and spend the preserved Claude budget on orchestration and Opus authoring. ## II.7 Remediation discipline - Findings go back to the **author** agent (context intact) as an enumerated list: "Remediate findings 1–6. For each: the fix, a test that would have caught it, and if you dispute a finding, the concrete argument." Disputes come to the orchestrator for triage; do not let the author silently drop a finding. - Every remediated finding gets a **test that would have caught it**. The thermal-clamp remediation is the model: the fix landed *with* tests proving clamps are authoritative. A fix without a pinning test regresses next month when another agent touches the file. - If remediation was structural (not point fixes), **loop the new diff back through review**. The in-flight hardening worktrees carry exactly this instruction: "re-run Codex review if changes were major." --- # III. VERIFICATION DOCTRINE The second biggest lever. The review pipeline catches what's wrong with the code; the verification doctrine catches what's wrong with the *claims about* the code — and gives every search-quality decision a number instead of a vibe. ## III.1 Eval-driven development Magpie's search quality is governed by a **golden set**: query → expected-answer pairs in `Magpie/eval/golden-set.json`, each tagged with a kind (`name`, `speech`, `visual`, `caption`, `narrative`, ...). `magpiectl eval` runs every golden query against the real index and reports recall@5, recall@1, MRR. The hard gate, encoded in the tool itself: **recall@5 must never drop; the binary exits 1 below 70%.** Any search-quality change — chunking, embeddings, ranking weights, query handling, captions — runs eval before AND after, and the numbers go in the PR description. The growth discipline is the part people miss: **the golden set grows BEFORE each change, not after.** When starting a capability batch (captions, narrative queries, b-roll intent), the *first* step is adding the failing queries to the golden set and watching them fail. Then the batch is built, and improvement is *measured*, not vibed. The sprint's trace: golden set 12 → 17 (quality loop) → 21 (query expansion) → 26 (captions — all five new caption queries hit top-5) → 30 (search intelligence — all four new queries hit #1). Aggregate eval went 57/100/0.72 → 62/100/0.75 (captions) → 63/97/0.76 (intelligence) on the way. Every one of those transitions is a claim with a receipt. For **pure refactors** the gate tightens to **byte-identical eval output**. Not "same recall" — byte-identical. The engine-consolidation batch (PR #29, which collapsed the Qwen host 286→82 lines and the caption host 274→104 via a shared `SubprocessModelHost`) shipped under a byte-identical gate; so did the Search Lab's nil-override path and the theme system's Classic preset (proven byte-identical by a token-equality test). Byte-identical is the only gate that catches "the refactor subtly reordered ties." When a refactor claims behavior preservation, demand the strong form. ## III.2 What the gate actually caught Three incidents, each of which would have shipped a regression without the gate: 1. **The 17% collapse that wasn't code.** Mid-sprint, eval dropped ~17 points after a change that couldn't plausibly cause it. Instinct says revert; the gate + investigation said otherwise — the *corpus* was corrupted (see Case Study VII.2: a raw-SQL delete in a `sqlite3` shell with foreign keys OFF had orphaned rows non-cascadingly). Without a trusted baseline number, this would have been misdiagnosed as a code regression and "fixed" by mangling the ranking. The eval harness's job is not only catching bad code — it's *exonerating* good code. 2. **The naive caption integration that silently cost 12 points.** The first cut of caption-arm fusion looked reasonable, demoed fine on cherry-picked queries, and dropped recall@5 by 12 points on the full set — captions were drowning out the speech arm on speech-kind queries. Caught before merge purely by the before/after run; the shipped version (strict-only caption BM25 + a semantic floor at 0.63, Lab-tunable) *gained* 5 points instead. 3. **Weight tuning that looked right and regressed.** RRF arm-weight adjustments that improved the two queries being stared at while quietly losing more elsewhere. This is the default outcome of vibes-based tuning — it's why the Search Lab (IV.5) exists to make tuning data-driven, and why no weight change merges without the full-set numbers. ## III.3 The scratch-DB rule **The live corpus shifts under you.** Eval runs against a real indexed library; if that library is Seb's actual working DB, then indexing progress, new files, and any DB surgery change the denominator between your "before" and "after" runs, and your gate compares apples to oranges. Doctrine: **keep a scratch copy of the live DB for gates** — `sqlite3 magpie.db ".backup '/path/scratch-eval.db'"` — and run before/after against the *same frozen copy*. Refresh the scratch copy deliberately, between batches, never during one. (This rule was written in blood adjacent to the eval collapse; the in-flight memory file lists it in GOTCHAS.) ## III.4 The honesty rules — and why you reward them Three incidents during the sprint where an agent refused the easy path, and each refusal protected the system's integrity. These are the moments to *reward at review* — call them out to Seb, note them in memory — because they are senior-engineer behavior and the incentives of an eager agent point the other way: 1. **Refusing wrong golden queries.** An agent asked to expand the golden set declined to add queries whose ground truth it had verified to be wrong (the "expected" file genuinely didn't contain the queried content). Had it complied, those would have become *permanent fake misses* — a ceiling on measured recall forever, poisoning every future gate. The golden set is only as honest as its worst entry. 2. **Correcting the brief's diagnosis.** A search-intelligence brief diagnosed a failing "skyline" query and prescribed a fix. The agent investigated first and reported back: the skyline case was *already fixed*; the real remaining failures were b-roll intent classification and mood-vocabulary queries. It implemented against the real problem instead of the brief's wrong premise. The shipped batch (PR #37: intent classifier, vocabulary bridging, ClipType b-roll arm) is the fix for the *actual* failures — because the agent pushed back. This is exactly why the prompt template (I.3) contains the "if the premise is wrong, STOP and report" clause. 3. **Declining to gut docs to hit a prediction.** An agent asked to trim documentation to a predicted line count declined to cut content that was load-bearing, and said so, rather than hitting the number. Numbers in a brief are estimates, not KPIs; an agent that games them is worse than one that misses them honestly. The general principle: **an orchestrator that punishes push-back trains agents to comply with wrong briefs.** When an agent contradicts you with evidence, the correct emotional response is relief. ## III.5 The independent-verification checklist (per merge) Run by the orchestrator, personally, before every merge — one batched Bash call, output filtered: ``` cd "/Users/sebsatian/Documents/App Builds/Video Search/Magpie" \ && swift build 2>&1 | tail -3 \ && ./.build/debug/magpie-tests 2>&1 | tail -5 \ && ./.build/release/magpiectl eval 2>&1 | grep -E "recall|MRR|PASS|FAIL" ``` Then: - [ ] Test **count** matches the agent's claim (and after a merge with test conflicts: equals the union of both branches' counts — the detector for a silently dropped suite, I.5). - [ ] Eval numbers match the agent's claim, against the scratch DB. - [ ] Line-level read of anything in the diff touching: engine math, migrations, concurrency, crypto/verification, ranking. (Not the whole diff — the dangerous categories. The Codex review covers breadth; you cover the categories where a subtle wrongness survives review.) - [ ] Diff scan for: swallowed errors, hard-coded model names, color/font literals, edits to shipped migrations, `TODO` standing in for logic. - [ ] The PR description states the gates' numbers. If any check disagrees with the agent's report, the batch does not merge, and the discrepancy — not the fix — is the first thing you investigate. A wrong status report from an agent is a process signal, not just a bug. --- # IV. DESIGN SYSTEM AND SKILLS-AS-MEMORY Agents have no episodic memory. Every agent is a brilliant contractor on their first day, forever. Institutional memory therefore has to live in **files that agents are made to read** — and the sprint's clearest process discovery is *which* files actually work. ## IV.1 Rules survive; ad-hoc fixes don't — the tooltip story The file-browser tooltip took **three failed rounds**. Round one: an agent built it; the layout collapsed at narrow widths. Round two: a different agent "fixed" it; rows misaligned. Round three: fixed again; broke again under a different content shape. Each fix was correct *in that conversation* and evaporated with the conversation — the next agent to touch the file re-introduced the class of bug because nothing in the repo told it not to. Round four: the lesson was **encoded into the design skill file** — a hover-cards section in `.claude/skills/design/liquid-glass/SKILL.md` specifying fixed-width (220pt), row-based structure, alignment rules — and the agent prompt said "rebuild the tooltip per the upgraded liquid-glass skill (hover-cards section)." It worked, and it has *stayed* worked, because every subsequent agent that touches hover UI is pointed at the same section. The commit message even records the mechanism: "fixed-width 220pt tooltip rebuilt per upgraded liquid-glass skill (hover-cards section)." The generalization, which became doctrine: **when a fix fails twice, the problem is not the fix — it's that the rule lives in a chat transcript.** Move the rule into a file (a skill, a token file comment, a doctrine doc), then make prompts reference the file. Same pattern, same week: the **written radius rule** went *into the DesignSystem token file itself* alongside a `radiusModal` token and a concentric-radius rule for nested pills (PR #33), so any agent reading the tokens — which the HARD RULES force them to do — reads the rule at the moment of use. Rules co-located with the tokens they govern are the highest-retention form of memory in this whole system. ## IV.2 The skills directory as institutional memory `"/Users/sebsatian/Documents/App Builds/Video Search/.claude/skills/"` holds `design/` (with `liquid-glass/`, `animation-patterns/`, `ui-prototyping/`), `swift`, `swiftui`, `swiftui-expert`, `macos`, `security`, `testing`, `performance`, `legal`, `_shared`. Treat these as the project's engineering handbook, and maintain them like code: - When a review or an incident produces a *reusable* lesson (not a one-off bug), upgrade the relevant skill file in the same batch as the fix. - Prompts name the skill AND the section ("hover-cards section"), because "read the design skill" against a large skill file is a token tax and a lottery. - The measured effect of good skills: **Opus at medium effort became acceptable for UI work only after the design rules were encoded** — the skill substituted for effort. Before the skills carried the rules, even higher-effort runs produced the tooltip mess; after, cheaper runs stopped producing it. Skills are effort-multipliers with a one-time cost. - Sonnet 5 remained banned from UI even with skills (VI.1) — skills raise a capable model's floor; they don't fix a model that can't hold layout invariants. ## IV.3 The audit cadence **Full-codebase audit every ~2 weeks, or after any multi-PR sprint**, producing a **ranked top-10** of real issues. Not a linter run — a fresh-eyes agent (Codex is ideal: cheap on the cap, thorough on repo reading) told to read the codebase as a hostile new tech lead and rank what it finds by severity × likelihood. Evidence the cadence works — the day-4 audit (2026-07-04) found, among its top-10: - **The ANN scale wall (S1):** exact vector scan would collapse at library scale; led directly to USearch HNSW behind `VectorIndexAdapter` (PR #28: parity 99.4%, eval rank-identical, 1M synthetic vectors: 121ms exact vs 0.3ms ANN). - **An 827-line god object:** `AppModel` at 833 lines doing queue, search, and download tracking; decomposed to 521 via `QueueController`/`SearchController`/`ModelDownloadTracker` (PR #30). - **Thumbnail decodes in view bodies:** synchronous image decoding inside SwiftUI `body` — the classic scroll-jank bug; replaced with async cached `MediaThumbnail` + header-only dimension reads (PR #30). - Plus: the decorative onboarding schedule buttons (wired into the real scheduling design, `docs/scheduling-and-resilience-design.md`), the hardcoded 0.1.1 version string, the all-row ValueObservation and 500ms full-GROUP-BY poller as the two 10k-video bottlenecks (planned as 0.6.10). **All of the top three were fixed within 48 hours.** That is the standard: an audit whose findings sit in a backlog for a month was theater. Rank them, fix the top slice immediately, board the rest with version numbers, ingest the full audit into Atlas (the 2026-07-06 audit, `codex-full-audit-2026-07-06`, is queued for exactly that). Why AI-built code *specifically* needs the cadence: every batch is written by an agent that sees only its batch. Cross-batch decay — duplication, god objects, N+1 observation patterns — is invisible at every individual review and only visible to a whole-codebase read. No single diff ever contains the 827-line god object; it accretes 60 well-reviewed lines at a time. ## IV.4 Consolidation batches, not per-diff nitpicks Duplication is paid down in **dedicated consolidation batches with byte-identical eval gates** (PR #29 is the exemplar: `SubprocessModelHost` collapsing two 270+-line hosts, `StageRunner`/`HitResolver`/`FTSMatch` collapsing four duplicated blocks, `searchOutcome` decomposed into six named stages — all under a byte-identical gate), **not** by nitpicking each feature diff toward perfect factoring. Rationale: making a feature agent simultaneously ship a feature *and* refactor the neighborhood doubles its blast radius and halves the reviewability of both halves. Let feature batches ship clean-enough code fast; harvest the duplication list (reviews and audits generate it for free); retire it in one focused, behavior-frozen batch where the *only* question at review is "is the behavior identical?" — a question the byte-identical gate answers mechanically. Consolidation under a strong gate is the safest work in the whole system; consolidation smeared across feature diffs is the riskiest. ## IV.5 Doctrine: honest UI states Written after the "Indexed-vs-queued contradiction" (a badge said *Indexed* while jobs for that asset still sat queued — PR #33 fixed the badge to be outstanding-jobs aware): - **A state badge is a promise.** "Indexed" must mean DONE — every derivable signal for that asset exists. Partial truth gets its own honest state ("Partial", "N of M done"), never the finished-state label. - **Failures speak plain words, never jargon.** The word "degraded" is banned from user-facing copy (decided with Seb, recorded in `scheduling-and-resilience-design.md` §F): arms failed + no results → "Search couldn't check everything — part of your library's search didn't respond. [Try again]"; arms failed + results exist → a quiet "Some results may be missing." Failure *reasons* in the queue are human sentences; the raw error is dev-mode only. - **Honest numbers**: progress % never decreases (stable expected-jobs denominator, monotonicity-*tested*, PR #34); ETAs are nil until ≥3 throughput samples exist rather than fabricated; drop-sheet says "couldn't read N" when it couldn't read N. - Where the truth is unflattering ("estimates improving as we measure"), say it. The trust cost of one caught lie in a status surface exceeds the polish value of every optimistic label in the app. ## IV.6 Doctrine: privacy by construction The analytics batch (PR #36) established the pattern: **events CANNOT carry strings.** The `AnalyticsEvent` catalog is typed, with compile-locked tokens — closed enums for every property. There is no code path by which a filename, a query, or a transcript fragment can enter an analytics payload, because the types don't admit strings. This **beats redaction-by-review categorically**: review-based redaction must win every review forever; construction-based privacy wins once, at the type level, and every future agent inherits the guarantee without knowing it exists. The same shape appears in search feedback (PR #26): the stored artifact is a SHA256 of the query, never the query. Adjacent defaults from the same batch: analytics **opt-in, default OFF**; PostHog EU host; opt-out is epoch-guarded and clears pending events (a Codex finding, II.3 row 1). When building anything telemetry-shaped, reach for the closed-enum construction first and treat "we'll be careful about what we log" as the design smell it is. ## IV.7 Doctrine: everything user-tunable gets a Lab surface Any tunable that affects output quality gets a **developer-mode surface** so tuning is data-driven: the Search Lab (PR/commit `203dcfc`) exposes per-arm ranked columns, live weight sliders, arm gates, a default-vs-overridden fused diff with rank deltas, an in-app golden-set benchmark, and save-as-golden. The Performance tab does the same for the resource story (CPU/RSS sparklines, helper RSS, thermal/LPM/pressure). The Lab is why weight-tuning regressions (III.2 #3) get caught in minutes instead of shipped: the person tuning sees the full-set effect of a slider, not the two queries in front of them. The engine surface backing it (`WeightOverrides`, nil path byte-identical, eval-proven) cost one small batch and has paid for itself several times over. `developerMode` defaults OFF in shipping builds (PR #35). --- # V. SECURITY AND RESILIENCE ENGINEERING FOR AI-BUILT CODE AI-built code has a characteristic security failure mode, and it is not "the model writes insecure idioms." It is: **the design describes a safety mechanism, everything downstream assumes the mechanism, and nobody checks the mechanism exists.** This section is organized around that. ## V.1 Trust roots: know what actually anchors each guarantee For every security claim in the system, be able to answer: *what is the root of trust, and is it something we control?* Magpie's map, as worked examples: - **License entitlements**: Whop does not sign its responses — so the trust root is **our own Ed25519 signature**, private key in Vercel env only, public key pinned in the app. The app trusts nothing Whop-shaped that our proxy didn't sign. (The whole licensing plan, `docs/licensing-implementation-plan.md`, is built around this: the app ships with ZERO secrets; the only key in the binary is a *public* key.) - **Model packs**: the trust root is the **install-time manifest** — file list + per-file SHA256 written on successful install; no manifest or mismatch ⇒ the pack is *not installed*, full stop ("Damaged — reinstall"). A half-downloaded pack can never masquerade as installed again. Executed helpers get the strong form: **full-hash on EVERYTHING executed** and byte-compare against `Bundle.module` — a Codex finding tightened this from spot-checks after the same-length-tamper bypass (II.3 row 3). - **Path containment**: real-path (symlink-resolved) containment against allowed roots, compared **component-wise** (another Codex finding: string-prefix compare lets `/foo/barbaz` through a `/foo/bar` check), with traversal walk budgets so a hostile directory tree can't DoS the scanner. - **XPC**: caller auth by code-signing requirement + bundle-id pin; the dev fallback is `#if DEBUG` only (finding-driven — the fallback originally shipped in release). - **Clock/time** (licensing): the trust root is the last *server* time, stored as a monotonic anchor; system clock behind it ⇒ refuse; large forward jumps ⇒ force revalidation. The discipline the reviews forced, worth stating as a rule: **comments about trust must be exactly as strong as the code.** "Honest trust-root comments" was a literal remediation item — a comment claiming "verified against pinned hash" above code doing a size spot-check is a security bug in its own right, because the *next* agent builds on the comment. ## V.2 The phantom-lock lesson The single deepest finding of the sprint (II.3 row 4), worth its own telling. The recovery design — stale-job instant-requeue at boot, WAL salvage, move-aside-and-rebuild — rests on a single-instance guarantee: "we know no other worker exists at boot." The design docs referenced an advisory file lock. Code comments referenced it. The requeue logic *assumed* it. Multiple agents had built on top of it across batches. **The lock had never been implemented.** Not broken — absent. Every agent that touched the area read the references, believed them, and built the next floor. Nothing in the normal loop catches this: tests exercise the paths that exist; per-diff review reviews the lines that changed; the lock was in *no* diff. It was caught only by an adversarial reviewer explicitly instructed to hunt for "any place where a comment or design doc promises a mechanism the code does not actually implement" — which is why that exact clause is in the review-prompt template (II.2) and must stay there. The generalized rule: **for every assumed invariant in a design, demand the file:line where it is enforced.** When reviewing a design-heavy batch, make the author produce an "invariant ledger": each invariant the design relies on → the code that enforces it → the test that would fail if it didn't. "It's enforced by convention" or "the doc says" are the wrong answers. AI agents are unusually susceptible here precisely because they take documentation at face value — a human skeptic half-remembers that nobody ever built the lock; an agent reads confident prose and inherits confident belief. ## V.3 The edge-case hunting method How the sprint systematically generated the edge cases that became tests. Two complementary mechanisms: **A. The audit dimensions.** The full-codebase audits and Codex security reviews were pointed down an explicit dimension list — enumerate these when commissioning any audit or hardening review: 1. **Scale** — what breaks at 10k assets / 1M vectors? (found: the ANN wall, all-row ValueObservation, full-GROUP-BY poller) 2. **Concurrency & lifecycle** — force-quit mid-write, double-launch, kill -9 the helper, opt-out mid-upload (found: stale-job limbo, the opt-out race, drain-worker multiplication; the XPC helper ships with a kill -9 survival gate: 119/120 + clean failure) 3. **Resource exhaustion** — disk full, memory pressure, thermal (found: disk-full-triggers-rebuild misclassification; thermal override) 4. **Hostile/degenerate filesystem input** — symlinks, moved files, unplugged volumes, app-internal junk trees, undecodable media (the whole VII.3/VII.4 family) 5. **Supply chain** — half-downloads, tampered files, unpinned package versions (manifests, hash pins, pinned python deps) 6. **Trust boundaries** — every IPC/network/file input: who can call this, what can they send? (XPC auth, webhook signature verification, entitlement signing) 7. **Time** — clock rollback, sleep/wake across a schedule, stale heartbeats 8. **State-machine holes** — for every state pair, is the transition defined or accidentally reachable? (licensing lock/grace states; the never-armed notification gate) 9. **The user's escape hatches** — can they always get out? (rebind caps with a support path; "Delete all Magpie data" with double confirm; recovery banner with plain words) **B. The edge-case register pattern.** For any high-stakes feature, write the register *before* implementation and make **each entry a test**. The licensing plan's register is the exemplar (`docs/licensing-implementation-plan.md`): reinstall-same-Mac, new-Mac-before-old-deactivated, rebind cap, refund/dispute, Whop outage (grace not lock), clock rollback, key-shared-to-second-user (and the neat analysis that migration self-defeats the freeloader), alpha-key-after-30d, upgraded membership, GDPR delete, whitespace in pasted key, app-too-old. A register written up front is a spec the author agent implements against and the reviewer attacks against; written after, it's a rationalization of whatever got built. **And its honesty ceiling**, research-confirmed and recorded in the same doc so nobody over-engineers: local apps can be cracked by determined attackers; the bar is "annoying to pirate, frictionless to buy" (server-validated signed entitlements + Keychain + no secrets in binary + rebind limits). Knowing where to *stop* hardening is part of the doctrine. ## V.4 Resilience doctrine: the index is derived data The design decision that makes Magpie's whole recovery story tractable: **the entire index is derived data.** Originals + models can always reproduce it. Therefore the worst case of any corruption is re-index *time*, never data *loss* — and recovery can be automatic and unapologetic: quick_check → WAL salvage → move aside as `magpie.db.corrupt-` → fresh DB → folders re-register from the *separately stored* bookmarks (GrantedFoldersStore survives by design) → jobs re-derive → plain-words banner. Supporting rules, now doctrine (from `scheduling-and-resilience-design.md` §B): - Migrations are versioned, transactional, **append-only** — never edit a shipped migration (v1→v9 and counting). - Every migration runs against a **copy of a production-shaped DB** in tests (the PipelineAudit family). - Destructive migrations carry an explicit backup step *inside the migration*. - Error classification before destructive response: the disk-full case (V.1) is the canon example — an exhaustive SQLite error taxonomy so that "can't write" is never answered with "rebuild everything." When you architect the next system, buy this property early: keep anything reproducible clearly segregated from anything that isn't (the bookmarks/DB split), and the scariest failure class in the app becomes a progress bar. --- # VI. MODEL SELECTION MATRIX AND EFFORT ECONOMICS ## VI.1 Field notes, per model | Model | Verdict from the sprint | |---|---| | **Opus 4.8 high** | The workhorse author. Excellent on engine work — concurrency, DB, model integration, ranking math. Ships occasional *confident* edge-case bugs (the entire II.3 evidence table is Opus-authored); the misses are **verification-depth, not reasoning-depth** — it reasons correctly about the design and under-tests the corners. That is precisely what the review pipeline is for; do not respond to an Opus bug by distrusting Opus, respond by never skipping the pipeline. | | **Opus 4.8 medium** | Fine for UI **only after** the design rules were encoded in skill files (IV.2). The skill substitutes for the effort. Without the skills: not fine. | | **Opus 4.8 xhigh** | Reserved: security-critical authoring, payment-code review, and (extrapolating) the orchestrator seat you now occupy. | | **Sonnet 5** | Produced repeated layout bugs on UI work and was **banned from UI** on this project. Retains a role only for genuinely mechanical, low-blast-radius edits — and honestly, Codex took most of that lane. Do not relitigate the ban because a UI task "looks simple"; the tooltip looked simple three times. | | **Codex GPT-5.5 xhigh** | The **sharpest reviewer in the fleet** (five-for-five on real findings, II.3) and a **precise implementer of well-specified tasks** — which is why it authors the payment state machine. Weaknesses: it should not do product/UX design, planning, architecture, or final review of taste — the global CLAUDE.md's taste ranking (5/10) is accurate. Bills to **OpenAI**, not the Claude cap. | | **Fable 5 (me, retiring)** | Orchestrator: planning, architecture, review triage, taste calls, human interface. The next orchestrator inherits this seat at Opus 4.8 xhigh. | | **Haiku** | Never. (Global CLAUDE.md rule; nothing in the sprint gave a reason to revisit it.) | ## VI.2 The billing topology is a design input Two meters exist: the **Claude session cap** (Seb's Pro plan, rolling ~5h, pooled across all Claude models — the binding constraint) and the **OpenAI plan** (Codex). Work routed to Codex is invisible to the binding constraint. Consequences: - Reviews default to Codex not only because it's the best reviewer but because review-on-Codex *preserves authoring budget on Claude*. - Repo-wide reads (audits, investigations) go to Codex: they are the most token-hungry work in the system and the least Claude-specific. - When the Claude cap is near, the correct move is not "work worse," it's "shift the shiftable lanes to Codex and keep the Claude budget for orchestration + Opus authoring." Session-cap hygiene inside Claude (all learned the expensive way, VII.5): lazy-load all context (the project CLAUDE.md's "load ON DEMAND only, never at session start, never 'just to be safe'" section exists because eager loading made every small task cost like an architecture session); batch related shell commands into one call; filter/tail every output — never let raw JSON, full logs, or a CodeSpring tree dump enter context; one task per context window; prefer Fable-class models for routine orchestration turns over Opus-1M (that single 45%-of-a-cap turn was Opus-1M re-processing accumulated tool output). ## VI.3 Effort economics: verification-depth vs reasoning-depth The sprint's most transferable insight about effort levels: **Most agent failures are verification-depth failures, not reasoning-depth failures.** The never-armed notification gate, the opt-out race, the phantom lock — none of these were beyond the author's reasoning. The author *didn't check*. And more thinking does not fix unchecked claims; **more TESTING does**, and a *different* checker does. Therefore: - **xhigh is reserved** for the genuinely reasoning-bound: security and payment state machines, subtle concurrency design, the orchestrator's own judgment. Everywhere else, xhigh buys little — you're paying for deeper thought about claims that needed *running*, not thought. - The budget that would have gone to blanket-xhigh authoring goes instead to: the review pipeline (a second model), independent verification (re-running everything), tests-per-finding, and eval gates. That portfolio empirically catches what effort doesn't. - Symptom to watch in yourself: reaching for "respawn at higher effort" when an agent shipped a bug that a test would have caught. Wrong lever. Demand the test. --- # VII. CASE STUDIES Five landmark incidents, told with enough detail to be reusable. These are the stories to retell when a future decision rhymes with one of them. ## VII.1 The Qwen "check your connection" bug — error messages lie **What happened:** The Qwen caption helper failed inside the built app with an error surfaced as "check your connection." Networking was investigated; networking was fine — the machine was online, other downloads worked. The real cause, eventually: the hand-rolled `.app` bundle assembled by `scripts/run.sh` **was missing a resource bundle** the helper needed. SwiftPM had built the resources; the assembly script simply didn't copy that one, and the failure path a missing local resource happened to trip emitted a network-flavored error message from deep in a dependency. **Lessons, in order of generality:** 1. **Error messages lie — trust the failure *layer*, not the failure *text*.** The message reports whichever exception surface the error tunneled up through, not the root cause. Before believing an error's story, ask what class of missing precondition could produce this exact surface. 2. **Hand-rolled bundles must copy EVERYTHING.** Any hand-assembled artifact (dev app bundles, release bundles, Docker images, deploy archives) needs an exhaustive, enumerated copy step — ideally derived from the build system's own manifest rather than a hand-maintained list. The class of bug is "works in `swift run`, dies in the bundle," and it recurs every time a new resource is added unless the assembly is generated, verified, or both. (Downstream echo: the release playbook's signing script must synthesize Info.plists for every SPM `.bundle` — same class, and the model-pack manifests of V.1 are the systematic fix for the downloaded-asset variant.) 3. **A dev-bundle/release-bundle parity check belongs in the release checklist** — the difference between the two assembly paths is exactly where this bug family breeds. ## VII.2 The eval collapse and the non-cascading delete — raw SQL on live DBs **What happened:** Eval recall collapsed ~17 points. The code hadn't changed in any way that could explain it. Investigation traced it to earlier **manual surgery on the live DB via the `sqlite3` shell**: rows deleted from a parent table. The `sqlite3` CLI does **not** enforce foreign keys by default (`PRAGMA foreign_keys` is OFF per-connection), so the `ON DELETE CASCADE` relationships the schema declared never fired — child rows (chunks, embeddings, FTS entries) were orphaned, the index's internal consistency broke, and search quality cratered in a way that looked exactly like a ranking regression. **Lessons:** 1. **Raw-SQL surgery on a live DB requires `PRAGMA foreign_keys=ON` first** — every time, in every ad-hoc shell. The schema's cascade declarations are enforced per-connection, not per-file. 2. **Better: use the app's own delete paths** (`magpiectl` or the engine's deletion API), which run inside the app's connection configuration and its invariants. The CLI exists partly so that nobody has a reason to open a bare `sqlite3` shell against production data. 3. **A trusted baseline is a diagnostic instrument.** The eval harness didn't just detect the damage — it *dated* it and exonerated the code. Without the numbers, the "fix" would have been mangling a healthy ranking function to compensate for a corrupt corpus. 4. This incident plus corpus drift produced the **scratch-DB rule** (III.3): gates run against a frozen `.backup` copy, refreshed deliberately between batches. ## VII.3 The CapCut incident — scanners need deny-lists **What happened:** A user-realistic test pointed the scanner at `~/Movies`. CapCut (like many apps) stores its **app-internal working data** under `~/Movies/CapCut User Data/...` — caches, project fragments, junk media. The scan swept in **65 app-internal junk files out of 76 total** — the library was 85% garbage, and worse, many junk files *failed* indexing (undecodable fragments), filling the queue with red "failed" rows. A user seeing that queue concludes the app is broken. Nothing was broken; the corpus was polluted. **The fix** (commit `dfbc6da`): a **scanner deny-list for app-internal paths** — `CapCut User Data`, cache directories, hidden files, `.app` bundles, `com.*` reverse-DNS directories — plus a **startup purge** of already-ingested denied assets with an orphan sweep (retroactive cleanup, because the damage was already in existing users'... well, Seb's... DB). **Lessons:** 1. **User directories are not user content.** Any scanner pointed at real-world folders needs a deny-list for app-internal paths as a *launch* feature, not a polish item. Enumerate the offenders empirically: point the scanner at a real, messy home directory before shipping. 2. **"Failed" rows from junk erode trust disproportionately.** Every failure shown to a user is a claim that something of *theirs* went wrong. Filtering junk out of the pipeline isn't just corpus hygiene — it's the difference between "it indexed my library" and "it choked on my library." 3. Ship the **retroactive purge** with the filter. A filter alone fixes new users and leaves every existing DB polluted. 4. Same batch, same spirit: audio files were exempted from *visual*-signal expectations (n/a rows, not failure rows) — don't report the absence of an impossible signal as a failure. ## VII.4 The plugged-in-drive / symlink / moved-file family — identity must survive paths **What happened,** as a slow accumulation rather than one incident: external volumes unmount and remount; users move files between folders; Finder drops arrive as odd URL representations; symlinks point across roots; users relocate footage from `~/Downloads` to a proper library folder. Every one of these breaks a system whose notion of file identity is "the path." **The design answer, built early and validated repeatedly:** **fingerprints — size + edge-hashes** (hashing the first and last chunks of the file, cheap even on huge video) as the identity key, with paths as mutable attributes. Downstream everything: a moved file is *recognized* as already-indexed (the StorageAdvisory's "verified moved-file copy" flow uses fingerprint dedupe as its source of truth — commit `c3148ba`); a re-plugged drive's assets reattach instead of re-indexing; dedupe works across duplicate copies; the offline-volume watcher (mount/unmount observation, availability refresh, offline badges — commit `10e47a7`) can mark assets offline *and bring them back* because identity survives absence. Symlink handling then gets its own security dimension (real-path containment, V.1) — identity-portability and containment are separate concerns; solve both. **Lessons:** 1. **Never let a storage path be a primary key** for anything a user can move, rename, eject, or symlink. Content-derived identity (full hash if cheap, size+edge-hashes when files are huge) makes the whole "files move" problem class largely disappear at the design level instead of being whack-a-moled per incident. 2. **The peripheral cases are the product** for a local-first app. "Works when the drive is plugged in and files never move" is the demo; drives and Downloads-folder chaos are the user's actual life. Budget real batches for them (offline sources were 0.3.2 — *early*). 3. Related capture from the same family: Finder drag-and-drop delivered `public.file-url` as a `Data` representation that `loadObject` mangled into unusable URLs — 0 clips from a valid drop — and providers had to be awaited *before* presenting the sheet plus made atomic via `.sheet(item:)` to kill a race (commits `c3148ba`, `2f0269b`). OS integration surfaces (drag/drop, pasteboard, bookmarks) are full of representation quirks; test them with the real OS gesture, not synthesized inputs. ## VII.5 Session-cap economics — operating a build under a hard budget **What happened:** Seb is on a Pro plan. The binding constraint is a **rolling ~5-hour session cap pooled across all Claude models** — the weekly per-model buckets sit near-empty while the session cap throttles everything. Early in the project this wasn't understood, and one exchange — Opus 4.8 on 1M context, with a large accumulated pile of tool output re-processed every turn — burned **~45% of an entire session cap in a single exchange.** The sprint's cadence was thereafter shaped as much by budget mechanics as by engineering. **The levers, all adopted and all in the project CLAUDE.md:** 1. **Lazy-load all context.** The CLAUDE.md was rewritten so the master plan, Atlas, the CodeSpring tree, skills, and reference repos load *only when the task needs them* — eager loading had made every small task cost like an architecture session. "Never 'just to be safe'" is the operative phrase. 2. **Filter every CLI output.** `| tail`, `| grep`, `| jq '.the.one.field'` — always. A raw CodeSpring tree dump or full test log doesn't just cost once; it re-bills on every subsequent turn of the session. 3. **Batch shell calls.** Related commands go in ONE Bash call. Every separate call carries turn overhead. 4. **One task per context window.** Long sessions accumulate re-billed context; finishing a task and starting the next in a fresh session is cheaper than it feels. 5. **Route the routine to cheaper Claude models and the token-hungry to Codex** (VI.2) — the cap is pooled, so an Opus-1M orchestration turn and a Fable turn draw from the same tank at very different rates. **Lesson beyond the levers:** *know which meter is binding before optimizing anything.* Optimizing weekly-bucket usage was worthless; the session cap was the real wall. In a different setup (API billing, enterprise seats) the binding meter differs and so do the correct habits — but there is always a binding meter, and the orchestrator who hasn't identified it is optimizing noise. --- # VIII. WORKING WITH THE HUMAN Seb is the product owner, the taste authority, and the person whose money and reputation ship with the app. He does not read the code. The relationship that works: ## VIII.1 Honest reporting is the entire currency Because Seb can't verify the code himself, your reports are his only instrument panel. The rules: - **Never overclaim review depth.** When he asked point-blank (2026-07-06) whether the review process truly covers safety/security/scalability line-by-line, the correct answer was the true one: independent gates always; line-level reads on the dangerous categories; adversarial cross-model review on big diffs — and then *codify that answer* so it's verifiable (`review-rigor-policy.md`). "Overclaiming review depth would be dangerous; the process must be verifiable" is written into that memory file as the why. - **Report numbers, not adjectives.** "Eval 58/92/0.72 → 63/97/0.76 on 30 golden queries, 687 checks" is a report. "Search is much better now" is not. Every PR carries its gate numbers; every status to Seb carries the same. - **Report what you didn't do and what went wrong**, unprompted: the batch that needed a second review loop, the flaky test you deflaked by redesigning the assertion (the ANN tie-break flake became a score-floor invariant, `8a2a09d`), the agent that died and got respawned. He calibrates trust on the misses you volunteer. - **Relay agent honesty upward.** When an agent corrects a brief or refuses a wrong golden query (III.4), tell Seb — it's evidence the system self-corrects, which is exactly what a non-code-reading owner needs to believe on evidence rather than faith. ## VIII.2 Decisions: which to surface, which to make **Make without asking** (then report): everything inside the codified rules — technical design within the master plan, batch scoping, model routing, redoing sub-par cheap-model output on a smarter model (the global CLAUDE.md explicitly pre-authorizes that), refactor/consolidation timing, test strategy. **Surface to Seb, always:** - **Anything user-visible in design.** He rejected the /v2 UI wholesale; his own Codex mockups are the design source of truth ("very basic, very clean"). Ask for / read his mockups before UI work; never invent UI direction unprompted. Layout *tweaks* are fine; *direction* is his. The website plan encodes the same: "hero prototype — SEB APPROVES BEFORE FULL BUILD." - **Money, accounts, and identity**: anything requiring his logins, API keys, purchases, or legal identity (the accounts checklist in `website-infrastructure-plan.md` is structured as "what Seb creates"; licensing is blocked on *him* filling `WHOP_API_KEY` into `.secrets/magpie.env` — you don't work around that, you wait or work elsewhere). - **Product semantics with teeth**: the instant-lock rule for invalid licenses vs 7-day offline grace was *his* call, captured with exact semantics; the "degraded" copy wording was decided *with* him. When a behavior will make a paying user feel something, he decides the feeling. - **Irreversibles**: the dev→main promotion (pending, deliberately his call), pricing, the staged-launch plan, anything reaching the public. - **Scope**: only work on tasks he explicitly assigns — don't wander the board unprompted (project CLAUDE.md, hard rule; he is quota-conscious and unsolicited work spends his budget). And frame surfaced decisions as **decisions, not essays**: the options, your recommendation, the one-line rationale, what you'll do on each answer. He's running a business, not grading papers. ## VIII.3 The four memory stores and their hygiene Institutional memory lives in four places with distinct jobs. Confusing them is how knowledge rots: 1. **The repo** (`Magpie/docs/`, `.claude/skills/`, CLAUDE.md, the code and tests). For anything an *agent* needs at work time: playbooks (the five launch docs live here so any future session can execute a release with zero archaeology), skills, hard rules, edge-case registers. If an agent will ever need it mid-task, it goes here — agents don't read Atlas. 2. **Atlas** (`v3` CLI, project `proj_5f0e60117e7445199cb95669c89da184`): the deep knowledge graph — architecture rationale, research reports (the Whop licensing research's full cited report), audit archives, session-state narratives. Query with targeted questions (`v3 ask "…" --mode local --wait`), never bulk dumps; ingest significant docs/decisions (`v3 ingest-file --wait`). Atlas is for *depth on demand*, pulled only when a task needs that depth. 3. **CodeSpring** (`codespring` CLI, the board/feature tree): what's done, in flight, and next, with version-numbered nodes. Hygiene: output is huge, so ALWAYS filter through `jq` — never dump the tree; look up just the node you need; update existing nodes rather than adding near-duplicates; full UUIDs only (truncated ids fail); there is **no task delete** — the convention for a dead duplicate is retitling it `zz-duplicate-ignore`; auth expires fast — ask Seb to run `codespring auth login`, don't fight it. 4. **Memory files** (`~/.claude/projects/.../memory/`): the orchestrator-to-orchestrator channel — small, curated, cross-session. `MEMORY.md` is an index; each file has a `description`, a **Why**, and a **How to apply**. This is where working preferences, the review-rigor policy, and the in-flight handover live. **The handover-file discipline** — the practice this very document generalizes: at the end of any significant session, and *especially* when in-flight work exists, update the in-flight memory file (`magpie-inflight-*.md`) with: merged state (through which PR), in-flight state (which worktrees, what they're fixing, the merge path when they land), the pipeline reminder, where the launch knowledge lives, board state, and — crucially — a **GOTCHAS list** (the current one: union-both-suites for test conflicts, full-UUID CodeSpring, scratch-DB eval, the known HNSW tie-break flake, the resume incantation). Write it assuming the next session is a *different model with zero shared context*, because tomorrow it will be. A handover that says "continue the hardening work" is worthless; one that says "harden2 is fixing the advisory lock that was documented but never implemented; when it lands, expect main.swift conflicts; union both suites" lets a cold-started model be productive in its first ten minutes. ## VIII.4 Production, not MVP — what the framing changes Seb's standing instruction: this is a production build; **never call it an MVP.** This is not vocabulary policing — it changes decisions. "MVP" licenses swallowed errors, decorative buttons, jargon copy, and "we'll harden it later." The production framing is why the honest-states doctrine exists, why onboarding buttons being decorative was an *audit finding* rather than a shrug, why recovery banners got real copy, and why the licensing plan handles refund/dispute/clock-rollback before the first sale. When scoping any batch, the question is never "what's the minimum?" — it's "what does the *real* version of exactly this one feature look like?" Cut *features*, never *quality of the features you ship*. --- # IX. EXTRAPOLATION — WHAT CHANGES FOR AN iOS APP Everything in Sections I–VIII (operating model, review pipeline, verification doctrine, skills, audits, honesty doctrines) transfers untouched. What changes is the platform's constraint surface. If Seb starts an iOS build, re-derive the plan against these deltas: ## IX.1 Distribution: App Store review replaces notarization + Sparkle - The entire `release-signing-notarization-playbook.md` §3–4 is replaced: no Developer ID cert, no notarytool, no stapling, **no Sparkle** — updates ship through the App Store; TestFlight replaces the DMG-to-alpha-testers flow (and is *better*: staged rollouts, crash feedback, 90-day builds, up to 10k external testers vs hand-emailed DMGs). - **A human reviewer at Apple now sits in your release path.** This changes cadence (plan 1–3 day review latency into every release; no same-hour hotfixes — learn expedited-review criteria before you need them) and changes *design*: App Review rejects things notarization never sees — private API use, misleading metadata, purchase-flow violations, background-abuse patterns. Add an "App Review risk" dimension to the audit list (V.3-A) and to every release checklist. - Staged-launch mechanics change: no "10 alpha keys" via stock-limited free product — TestFlight groups do alpha/beta; paid early-access is either a launch price or IAP tiers. ## IX.2 Sandbox is mandatory, and it guts the file-access model Magpie deliberately ships **unsandboxed** (Developer ID allows it; recorded decision). iOS offers no such choice, and for a footage-adjacent app this is the single biggest architectural delta: - No `~/Movies` scanning. Content arrives via **PhotoKit** (limited-library permission UX included), **UIDocumentPicker/security-scoped bookmarks** for Files-app folders, or share extensions. The MediaScanner concept survives but its input becomes "what the user granted," never "what exists." - Security-scoped bookmark discipline goes from good practice to load-bearing: every access wrapped in start/stop, bookmarks go stale aggressively, and iCloud Drive files may be **dataless** (download-on-open) — the offline-volume state machine (VII.4) has an iOS cousin: "not downloaded" as a first-class asset state. - The fingerprint identity design (VII.4) transfers perfectly and matters *more*: PhotoKit local identifiers and Files bookmarks are both unstable in their own ways; content-derived identity is the anchor again. ## IX.3 No subprocess helpers — everything in-process or in app extensions Magpie's inference topology — Python subprocess helpers (Qwen/MLX), an XPC inference service, `SubprocessModelHost` — **does not exist on iOS.** No `Process`, no spawning, no XPC services of your own, no venv in Application Support: - ML runs **in-process** (Core ML / MLX-Swift / Metal) or in **app extensions** with tight memory caps. Model choices must be re-made under iOS memory budgets (a 2B VLM that's comfortable in a Mac helper process may be a jetsam event on an iPhone). - The crash-isolation benefit XPC gave (kill -9 survival 119/120, crash caps, idle reclaim) must be re-derived differently: in-process inference means an inference crash *is* an app crash — invest in model-load validation (the manifest doctrine, V.1, transfers directly) and conservative memory headroom instead. - The provider-protocol rule (never hard-code a model) pays off here: swapping the model stack for iOS-sized variants is a provider swap, not a rewrite. ## IX.4 Background execution limits invert the job-queue design The Magpie queue assumes "the app is alive and may run for hours" (menu-bar mode, overnight Governor runs, login items). iOS grants none of that: - Foreground bursts + **BGProcessingTask** (unreliable, OS-scheduled, minutes-not-hours, favors overnight-on-power — request it with `requiresExternalPower` and you approximate Magpie's overnight mode, but the OS decides, not your time-picker). - Therefore the queue must become **incremental and interruption-first**: tiny resumable job units, per-stage checkpointing (the stage architecture — instant/fast/deep, per-stage `stage_rates`, heartbeats, stale-job requeue — transfers beautifully because it was already interruption-tolerant; the *scheduler* is what's replaced), and honest UX about it ("indexing continues while charging overnight" instead of a time picker). - Thermal/battery Governor logic transfers directly (iOS exposes thermal state and Low Power Mode) and matters more. - Expect indexing a large library to take *days* of opportunistic background time; the progress/estimates honesty doctrine (IV.5) becomes the core UX challenge rather than a polish item. ## IX.5 Monetization: StoreKit 2, or web purchase under external-link rules Two paths where macOS had one: - **StoreKit 2 IAP**: replaces the whole Whop+proxy+Ed25519 stack — Apple handles purchase, receipt cryptography (`Transaction.currentEntitlements` gives you signed entitlements *for free*), family/device semantics, refunds. The proxy, the hwid binding, the rebind KV — all deleted. Cost: 15–30% commission and App Review's purchase rules. The licensing *state machine* (locked/grace/revalidate) survives with StoreKit as the truth source; the edge-case register method (V.3-B) transfers — rewrite the register for StoreKit's edge cases (ask-to-buy, refund revocation events, family sharing, offline entitlement checks). - **Keep web purchase (Whop)**: possible only within Apple's external-purchase-link entitlement rules, which are jurisdiction-dependent, fee-attached, and shifting — this is a **decision to surface to Seb** with current-at-that-time research (do the research then; do not trust this document's snapshot of Apple's rules). - Either way: the "app ships with zero secrets" rule (V.1) is unchanged and non-negotiable. ## IX.6 Lifecycle, UI, and testing deltas - **Lifecycle**: scene-based, suspension-first. Magpie's "the app is running, poll every 500ms" patterns must become "snapshot on background, restore on foreground"; every long operation needs a suspension story. The `@Observable`/controller decomposition (post-god-object AppModel) transfers as-is. - **UI**: SwiftUI transfers; idioms don't. No hover (the tooltip saga's *lessons* transfer; the tooltip doesn't — hover-cards become tap-to-reveal), no fixed 220pt anything (size classes, Dynamic Type — the token system gains type-scale and spacing tokens), navigation stacks replace sidebars on iPhone. The design-skill files need an iOS chapter *before* UI batches start — that's the tooltip lesson applied proactively: encode the rules first this time. - **Device-matrix testing** replaces "it runs on Seb's Mac": memory tiers (an 8GB-RAM iPhone vs base models), thermal envelopes, Dynamic Type sizes, iPad multitasking. The Lab/dev-surface doctrine (IV.7) transfers — a hidden diagnostics screen gated off in release builds — and the eval harness runs on-device or against device-captured corpora, because embedding models can differ subtly across chips. Simulator green is not a gate; device green is. - **Crash/observability**: MetricKit + TestFlight crash feeds join the PostHog opt-in analytics (closed-enum doctrine unchanged). --- # X. EXTRAPOLATION — WHAT CHANGES FOR A WEB APP Again: Sections I–VIII transfer whole. The deltas: ## X.1 The eval-gate concept ports to contract tests + golden flows The deepest transfer. Magpie's insight was "a hard, numeric, before/after gate on the quality that matters, run by the orchestrator, never by vibes." For a web app the *quality that matters* is different, so the gate instruments differ: - **API contract tests** are the recall@5 equivalent: a golden set of request→expected-response pairs (status, shape, key invariants) run against every backend diff. Grow it *before* each change (a new endpoint's contract tests are written failing, first — the III.1 discipline verbatim). "Byte-identical" has an exact analogue for pure refactors: identical response bodies on the golden request set. - **Playwright golden flows** are the UX equivalent: the ten journeys that constitute the product (sign up → do the core thing → pay → come back) as end-to-end tests against a production-shaped seed DB. The gate: golden flows never break; a red golden flow blocks merge exactly as a recall drop did. - If the product contains actual search/ranking/LLM output, the golden-set method applies *literally* — query→expected pairs, a scoring harness, a never-drop gate — and the honesty rules (III.4, no fake ground truth) apply literally too. - The **scratch-DB rule** becomes: gates run against a frozen seed/snapshot database, never against a drifting shared dev DB. Same incident class, same fix. ## X.2 The review pipeline is identical Nothing changes. Opus authors, Codex xhigh adversarially reviews with named claims, author remediates, orchestrator independently verifies (build + tests + golden flows) and merges. The finding categories shift toward web's classics — authz holes (IDOR — every endpoint's "who can call this" is trust-boundary dimension 6 from V.3-A), injection, SSRF, race conditions on money endpoints, cache poisoning — so the review prompt's hunt-list gets a web edition, but the *structure*, the claims-attack move, the phantom-mechanism clause (V.2 — web designs assume "the rate limiter / the idempotency key / the unique constraint" exists just as readily), and the payment-code role inversion all transfer verbatim. ## X.3 The client is ALWAYS untrusted; secrets shift to server env Magpie's trust model had one machine, user-controlled but singular. Web splits it absolutely: - **Every check that matters runs server-side.** Client-side validation is UX, never security. Entitlement gating, quota enforcement, price computation — server, always. (Magpie already lived half of this: the proxy holds `WHOP_API_KEY` precisely because the app binary is untrusted territory. Web generalizes it: the *entire frontend* is the app binary.) - **Secrets management**: the `.secrets/magpie.env`-local / platform-env-production split (VIII, website plan) is already the correct web pattern — local gitignored env for dev, hosting-platform env vars (Vercel-style) for production, **nothing secret in any client bundle ever**, and an automated check for it (a build-time grep of the client bundle for key prefixes is cheap and has caught real leaks industry-wide). - The **privacy-by-construction doctrine** (IV.6) transfers to server logging: typed, closed-enum log/analytics events so PII *cannot* enter logs — even more valuable on web where logs are aggregated, retained, and subpoenaable. ## X.4 Migrations run against production data with rollback windows Magpie's migration doctrine (append-only, transactional, tested against production-shaped copies, destructive steps carry internal backups) was built for a DB living on one user's machine, where the worst case is one rebuild. Web migrations run against *the* database, live, under traffic: - Append-only and tested-against-a-prod-snapshot transfer directly and become non-negotiable. - New requirement: **expand→migrate→contract** choreography (add the new column, dual-write, backfill, switch reads, only then drop) so every step is compatible with both the old and new code — because deploys and migrations don't land atomically. - New requirement: an explicit **rollback window** per migration — the period during which the previous release must still run against the migrated schema — stated in the PR like an eval number. Destructive steps wait out the window. - The "derived data" doctrine (V.4) has a web reading worth designing for: keep caches/search-indexes/materialized views rebuildable from the canonical store, and the scary-failure surface shrinks the same way Magpie's did. ## X.5 Observability replaces the dev Lab Magpie's Lab and Performance tabs exist because there's no server to observe — the diagnostic surface had to ship *inside* the app. Web inverts this: - Production observability (structured logs, error tracking, latency/queue dashboards — the PostHog dashboards sketched in the website plan are the seed of this) is the Lab. Build it in week one, not after the first incident: it is the *verification instrument* for everything the orchestrator can't re-run locally. - The **honest-states doctrine** (IV.5) gets a server-side twin: health endpoints and status surfaces that report real state ("degraded" is fine *internally* — the ban was on user-facing jargon; the user-facing copy rules apply to error pages and toasts verbatim). - The audit cadence (IV.3) gains dimensions: dependency/CVE surface, authz coverage per endpoint, data-retention inventory, cost anomalies (a web app can fail by *bill*). - Feature flags replace the `developerMode` toggle as the tunable-exposure mechanism (IV.7): every risky behavior behind a flag, tuned against real traffic, killable without deploy — the runtime-config kill-switch the licensing plan reserved becomes a first-class platform facility. --- # XI. THE NEW-PROJECT BOOTSTRAP CHECKLIST Everything above, compressed into the first-day setup for the next AI-orchestrated build. Work down the list in order; nothing here takes more than a day total, and every item is load-bearing within two weeks. ## Day zero — before any feature code **1. CLAUDE.md skeleton** (project root; keep it under ~60 lines — it's re-read every session): - [ ] Product identity + the quality bar sentence ("production, never MVP"). - [ ] HARD RULES block: stack constraints, design-token rule, provider-protocol rule (no hard-coded models/services), branching model (`feature/ → dev → staging → main`, never commit to main/dev directly), append-only migrations. - [ ] The gate rule: name the eval/contract-test command, the never-drop threshold, and when it's mandatory. - [ ] Orchestration block: orchestrator = planning/review only; the model routing table; the review pipeline in one paragraph; shell discipline (batch calls, filter output, full UUIDs). - [ ] Context block: **lazy-load ON DEMAND only** — list where the plan/board/knowledge-graph live and when (not) to open them. **2. The verification harness — BEFORE the first feature:** - [ ] Eval harness / golden set (or contract tests + Playwright golden flows for web) with a hard numeric gate that **exits nonzero on regression**. Even 10 golden entries on day zero; grow it before every capability batch, failing-first. - [ ] Test runner with a one-line invocation an agent can be handed verbatim. - [ ] Scratch/frozen data copy for gates (`.backup` a seed DB); write down the refresh rule. - [ ] `scripts/` for anything hand-assembled (bundles, deploys) — generated/enumerated copy steps, never ad-hoc (VII.1). **3. The review pipeline:** - [ ] Verify `codex exec` works locally; record the invocation with `-c model_reasoning_effort="xhigh"` in CLAUDE.md. - [ ] Save the adversarial review prompt template (II.2) — including the name-the-claims move and the phantom-mechanism clause — where prompts get composed. - [ ] Write the role table: default (Opus authors / Codex reviews), payment-inverted (Codex authors / Opus reviews / orchestrator sweep), the >400-line mandatory-second-review rule. **4. Skills directory (`.claude/skills/`):** - [ ] Seed with platform skills; create the project design skill on the first UI batch. - [ ] Adopt the rule *now*: any fix that fails twice gets its rule encoded into a skill/token file, and prompts reference the section by name (IV.1). **5. Memory files (`~/.claude/projects//memory/`):** - [ ] `MEMORY.md` index + a project file (stack decisions, where plans live) + a preferences file (the human's working rules, billing constraints, **which meter is binding**). - [ ] Commit to the handover discipline: an in-flight state file updated at every significant session end, written for a cold-started different model, with a GOTCHAS section (VIII.3). **6. Board + knowledge graph:** - [ ] Feature tree with version-numbered nodes before building; one branch slug per node. - [ ] Knowledge-graph project for research/audits/decisions; ingest the master plan on day one. ## Standing cadences (put them in the calendar/loop now) - [ ] **Every merge**: independent rebuild + tests + gates by the orchestrator; numbers in the PR (III.5). - [ ] **Every big/security diff**: the cross-model review loop, findings remediated *with pinning tests* (II.7). - [ ] **Every capability batch**: golden set grows first, failing (III.1). - [ ] **Every ~2 weeks / post-sprint**: full-codebase audit against the dimension list (V.3-A), ranked top-10, top slice fixed within 48h, rest boarded (IV.3). - [ ] **Duplication**: harvested continuously, retired in dedicated byte-identical consolidation batches (IV.4). - [ ] **Session hygiene**: lazy-load, filter, batch, one task per window (VII.5). ## The ten rules, if you keep nothing else 1. The orchestrator plans and reviews; agents implement; **nobody reviews their own code** — not even you. 2. **Never accept self-reported success.** Rebuild, re-test, re-gate, personally, every merge. 3. A different model family reviews every significant diff, prompted to **attack named claims**. It will find real bugs. Five for five, it did. 4. For every assumed safety mechanism, demand the **file:line where it's enforced**. The lock you're sure exists may be the one that was never written. 5. Grow the golden set **before** the change. Numbers, not vibes; byte-identical for refactors; frozen data for gates. 6. Reward agents that **contradict wrong briefs** and refuse fake ground truth. Punish neither; both protect you. 7. Rules live in **files agents must read**, co-located with what they govern — never in chat transcripts. A fix that failed twice is a missing rule, not a hard bug. 8. Most failures are **verification-depth, not reasoning-depth**. Buy tests and reviews before you buy effort levels. 9. Route by the **billing topology** and know the binding meter. Review on the meter that isn't binding. 10. Write the handover **as if the next orchestrator is a different model with zero context** — because one day, as today, it is. --- *End of handover. The in-flight specifics (two hardening worktrees, the merge path, the launch blockers) are in `magpie-inflight-2026-07-03.md` and the repo docs — deliberately not duplicated here, because state rots and method doesn't. Run the pipeline, trust the gates, keep the numbers honest, and Magpie ships.* *— Fable 5, 2026-07-06*