gotcha-distill

Coding agents forget. Every session starts from the files on disk, and whatever the last agent learned the hard way is gone unless someone wrote it where the next one will read it.

The obvious fix — put every lesson in the always-loaded instructions — does not survive contact with a real repository. Fifty lessons at 200 tokens each is 10,000 tokens on every turn of every session, nearly all of it irrelevant to the task in hand. The lessons that matter get buried by the ones that do not.

Three levels, three load frequencies

The pattern splits each lesson by how often it needs to be in context:

Level	What	Loaded	Cost
1	One bullet in the contract, under a domain heading that names its trigger	Every session	~15 tokens
2	The skill itself — how to write a lesson	Only after something breaks	~1,500 tokens, once
3	The full entry in its domain file	Only when the task touches that trigger	~150–300 tokens

Level 1 is the part that has to be cheap, because it is always there. It is one line: what the trap is, and when you are about to walk into it. Level 3 is the story — what broke, the evidence, the fix, and the rule that generalises — and it is only worth loading when you are in that part of the codebase.

The part most people skip

Two-tier indexes rot. A lesson gets written into the domain file and never indexed, so nothing loads it. Or an index line outlives the entry it points at. Either way the knowledge base quietly stops being trustworthy, and a knowledge base nobody trusts is one nobody reads.

So the contract is asserted, in both directions: every index line must have an entry, and every entry must have an index line. Plus: every domain heading names a real file and carries a trigger clause; no duplicate titles on either side; no orphan files; every cited postmortem exists; every entry carries a date; and every copy of the skill across tool directories is byte-identical.

The checks are only worth what their failure modes are worth, so the script builds a synthetic repository, asserts it is clean, then applies nineteen named mutations and asserts each one trips its own check. A check that cannot be shown to fail is decoration.

What the split actually buys

Measured on the production repository using the pattern: 80 lessons, 5.3 KB always loaded, and an 11.5:1 ratio between what stays lazy and what does not. Every lesson costs about fifteen tokens of routing in a session that never touches its trigger, against roughly two hundred if the entry itself lived in the prompt.

That is the whole economic argument. Cost grows with the complexity of the task rather than with the volume of accumulated documentation, which is the only way a knowledge base can keep growing without eventually strangling the sessions it was meant to help.

What it costs to be wrong

The entries are not style preferences. The bar is that the lesson cost a production break or a near-miss — which is why the knowledge base in the project this came from is 80 entries and not 800. A gotcha with no trigger condition is a line nobody can match to their task, so it is a line nobody loads; the skill says so explicitly, and the contract enforces the trigger clause.

How it got here

I prototyped the whole thing on my own projects before it went anywhere near work, then adopted it once it had earned its place. It runs on two interchangeable agent runtimes from one source of truth, because a lesson that only exists for one tool is a lesson that disappears when the tool does.

The repository is stdlib Python and one shell script, with no dependencies. It generates an llms.txt from the same index — the index is already a list of links with notes, which is exactly what that format asks for — and CI checks the generated file against a fresh run so it cannot drift.