Personal project, measured
agent-harness
A retrieval harness that routes agent questions through layered lookups instead of grepping whatever happens to be cloned — measured at 1,232× less context, and correct where grep was not.
A platform organisation has hundreds of repositories. Any one engineer has cloned a few dozen. Coding agents inherit that gap exactly, and the naive answer — grep the local clones — fails in two different ways.
It is expensive: reading enough files to answer “what required_version does
each Terraform root declare?” can push megabytes into a model’s context for a
one-line answer.
And it is wrong, which is worse. A miss is not an absence. If the repo that owns the answer was never cloned, no amount of local grepping finds it, and the agent reports with complete confidence that the thing does not exist.
Four lanes, cheapest first
| Lane | Source | Answers |
|---|---|---|
| A · Route | Org catalog | Which repo owns this; archived; cloned |
| B · Structure | Per-repo inventories | Version pins, module refs, CI wiring |
| C · Symbols | Code graph (opt-in) | Where a thing is declared, to file:line |
| D · Current truth | GitHub API at HEAD |
What the code says right now |
An agent consults the cheapest lane that can answer, then stops. The catalog covers the whole organisation, so Lane A answers routing questions that grepping structurally cannot.
The rule underneath it is that a local clone is not a fact source. A clone answers “can I run this”. The remote answers “what does it say”. Conflating those is how an agent ends up confidently describing code that was deleted upstream three months ago.
The bake-off
Six questions with known ground truth, each answered twice — layered lookups against a grep-and-read baseline — measured in bytes that would actually enter the model. Run on 2026-09-04.
| Question | Layered | Baseline | Ratio |
|---|---|---|---|
| Which repos own IAM as code, and which are cloned? | 158 B | 740,665 B | 4,688× |
| Which cloned repo is archived upstream? | 26 B | not answerable | — |
What required_version does each Terraform root declare? |
422 B | 3,010,633 B | 7,134× |
| Which legacy roots are wired to Atlantis? | 2,425 B | 153,828 B | 63× |
| Who consumes the shared modules repo, at what ref? | 767 B | 740,665 B+ | 966× |
| Which workflows and chart keys does a service use? | 1,296 B | no useful result | — |
On the four questions both approaches could answer: 3,772 bytes against 4,645,791 — a 1,232× reduction.
The ratio is not the interesting part. On two of the six the baseline was not merely expensive, it was wrong or unable to answer at all — the owning repo was not cloned, and archived status is invisible from a clone no matter how thoroughly you read it. The layered approach was correct on all six.
What I will and will not claim
A number like 1,232× invites scepticism, and it should. So the project keeps an explicit ledger of what the measurement supports:
| Claimable | Not claimable |
|---|---|
| Context efficiency on that question set | 1,232× as a standing property |
| Correctness where the baseline failed | Time saved — there is no human baseline |
| Commit attribution series | Task success rate |
The gate metric is skill-load adoption: until the skills are actually loaded in real sessions, efficiency claims are theoretical. The instrumentation exists, and the honest baseline when it was turned on was near zero. That is reported as near zero. A flattering compression ratio nobody has yet benefited from is not a result, and shipping it as one would be the easiest way to make every other number on this site worth less.
Policy, wired into the retrieval path
Lanes B through D can surface proprietary source, so they are opt-in behind an explicit environment gate. Lane A — catalog metadata only — stays available by default.
That places the data-classification boundary inside the retrieval stack rather than in a document beside it. An agent cannot casually pull restricted content, because the lane that would return it is switched off until someone turns it on deliberately.
Shape
Stdlib Python only in the core tools — a local operator tool that needs a dependency install before it runs is a tool people stop running. Org-specific configuration lives in adapters rather than core, so the retrieval machinery stays reusable while company-specific skills ship alongside it. Organisation snapshots are committed rather than regenerated, so history cannot be quietly rewritten by a later run.
Not open source. The architecture and the results stand on their own without the private adapter contents.