The Comprehension Gate: Making Human Understanding an Engineering Constraint

The last post treated comprehension as a measurement problem. This one is what happened when I stopped treating it as one and built the instrument. Three weeks after publishing the concept, here is the first comprehension-coverage map of a repository I own — boleto.ts, a small TypeScript library for Brazilian payment slips, developed predominantly through agent-mediated coding over the past nine months: of the 42 commits touching its source directory, 29 carry an agent's authorship — GitHub's Copilot coding agent, mostly, with Anthropic agents behind several more.

Four of six modules are dark: no human on record holds current, evidenced comprehension of them. The two that escaped are only AT_RISK. And the map is hiding its best finding in that word "comprehender," which I will get to — because the single entity the instrument currently recognizes as keeping the library's signature module out of the dark is not me, and it is not a person.

The instrument is comprehension-coverage, open source as of this week: a specification, a Python reference implementation, and a Kotlin CLI that reproduces the reference byte-for-byte. Everything quoted in this post is real output from recorded runs — the repository pins the commits, the configs, and the exact invocations.

What counts as evidence — and what is only activity

The map is built from evidence events, not from vibes. In v0.1 an event is one of three things extracted from plain git history: hand authorship, agent-mediated authorship (detected by co-author trailers and bot identities, and discounted — briefing an agent builds some theory, but the comprehension-gap research says nowhere near as much as writing it yourself), and explicit attestations — a signed record that a person re-walked a module.

What earns nothing: merging an agent's diff unread, watching tokens stream, having your name on the repository. The spec is blunt about what this means for the instrument's claims, and the sentence doing the philosophical work is constitutional decision C2: the tool is "an evidence detector with an explicit model of staleness, never a comprehension detector." It does not claim to know what anyone remembers. It computes how stale the available evidence of remembering has become. Every honest thing and every wrong thing the map says follows from that distinction.

How the evidence decays

Evidence rots on two clocks. The primary clock is churn: comprehension of a module that has been rewritten under you is stale no matter what the calendar says. The secondary clock is time itself, because the forgetting-curve research is unambiguous that theory fades even when code holds still. The score halves for every full rewrite's worth of churn since the evidence, with a wall-clock half-life as the floor.

The interesting part is that the first real run argued with the specification and won. The provisional spec set the wall-clock half-life at 180 days, flat. Run against a decade of expressjs/express history as a dress rehearsal, that floor produced an obvious false positive: it erased the standing of a contributor whose module had barely changed since his substantial work on it — old and static was being punished like old and overwritten, the exact opposite of the design intent. The fix that survived the experiment is a quiescence-scaled floor: the wall-clock half-life stretches up to threefold as a module goes quiet, and collapses back to the base rate as churn approaches a full rewrite. The alternative — just lengthening the flat floor to 540 days — was rejected for a reason the express data alone couldn't show: it also hands a 2.16× decay reprieve to evidence the module has already churned away, which quietly demotes churn from primary clock to suggestion. The whole experiment — variants, scores, the rejected option and why — is public in the repository's CALIBRATION.md, and the change landed as one commit citing the run that forced it. The parameters remain provisional. The process for changing them is not.

The dark module, traced

Take itf — the interleaved-2-of-5 barcode encoder, one of the two modules I marked critical, and the one that blocks the gate. The tool's own remediation output, verbatim:

Read that middle line again. Asked who last held evidence of understanding this module, the instrument answers with the name of a bot. The trace behind it checks out against raw history: the 137-day-old evidence is an April 2026 batch of commits by GitHub's Copilot coding agent that reworked itf.ts, helpers.ts, and svg.ts in one pass — which is why three modules share the same strongest-evidence line. The main module's strongest evidence is more recent, 83 days, and it belongs to a commit authored directly by an Anthropic agent, carrying its own co-author trailer. Both agent-detection paths — bot identity and trailer — are firing on real history, and what they reveal is a library whose entire recorded life in some modules is agent activity, discounted evidence decaying with nobody re-evidencing behind it.

And me? I directed every one of those agent sessions. I reviewed the pull requests. I merged them. The map records none of it — in boleto, the module this library exists for, there is no evidence event under my identity at all. Not decayed: absent. My actual contribution to this codebase — prompts, reviews, merge decisions — happens in evidence classes v0.1 does not ingest. Hold that thought.

Where the instrument is wrong

The spec's acceptance criteria required the first field test to produce at least one result the repository owner could refute from ground truth — not hoped for one, required one. It produced two, and they interlock.

The first: boleto is AT_RISK rather than DARK because it has exactly one comprehender — copilot-swe-agent[bot], scoring 0.7218 against a threshold of 0.5. A bot identity accumulated enough discounted authorship evidence to qualify as a comprehender. This is the model faithfully executing its own rules into absurdity: agent-mediated evidence is discounted because an agent holds no transferable theory a team can lean on at 3 a.m. — and yet nothing in v0.1 stops the agent identity itself from clearing the bar. The consequence is worse than cosmetic. The gate fails only on DARK by default, so if itf were not on the critical list, an agent change to my payment-barcode module would merge under a green gate whose sole load-bearing "comprehender" is Copilot. The failure is filed in the repository as an issue, deliberately marked do not fix until after this run was reported — the instrument's first field test deserved to be observed honestly, not sanitized into a success.

The second failure is the mirror image, and the concept post predicted it: I am the person who most plausibly holds theory of this library, and I score near zero everywhere except tests. The express dress rehearsal surfaced the same pattern first — current maintainers who demonstrably understand that codebase, because they review every consequential change and run the releases, score near nothing on core modules. The people who understand code today increasingly review; v0.1 only sees authorship. This is where "evidence of comprehension is not comprehension" stops being a caveat and becomes a design principle: both failures are missing evidence classes, not miscalibrated thresholds. The fix for the bot is to exclude agent identities from comprehender counts; the fix for me is a REVIEWED evidence class fed by the forge, whose interface the spec already reserves. Neither fix is a softer number. The tool can be wrong in ways the concept survives — as long as it is wrong out loud, in public, with the failure named.

The gate

What does a CI pipeline do with this map? The policy I run has three colors. GREEN: the touched modules have a current comprehender; nothing changes. AMBER: coverage is thin or stale; a designated comprehender joins the review — the map becomes a routing rule. RED: the module is dark; agent changes cannot merge until someone re-establishes comprehension — by genuinely re-walking the module and attesting it, the one deliberately manual evidence class v0.1 ships. On boleto.ts today the gate exits red on itf, which is exactly right: the next change to that encoder should cost someone a morning of actually understanding it.

Two complications, because a control that cannot be criticized is a control that gets bypassed. First, amber fatigue: a warning that fires on most changes is noise, and the thresholds that decide amber are judgment calls that will need the same calibration discipline as the decay parameters — the serial human stage this whole series has been tracking moves upstream again, into whoever tunes the gate. Second, the break-glass paradox. The module most likely to page you at 3 a.m. is precisely the dark one, and a gate that blocks the incident fix is a gate that gets deleted by Friday. So the gate ships an escape hatch: --break-glass "<incident-ref>" converts a red exit into a warning — and writes a signed, timestamped incident-diagnosis record into the repository, naming who overrode it and for what incident. Here is the twist I like most, and its honest limit. Whoever diagnosed a dark module under fire just performed the strongest comprehension-building act there is, so the escape-hatch record is genuine future evidence — the gate's own bypass feeds the map. But in v0.1 that record deliberately does not score: the reader recognizes the reserved evidence class and skips it, warning aloud. An escape hatch that instantly manufactured a comprehender would be a self-service comprehension faucet, and I verified the loop closes correctly — break the glass, re-run the tool, and the module is still dark until a human attests for real.

The series, resolved

Six posts ago this series started with a claim about engineering surviving the agents. Each post made one piece of the invisible work explicit. Intent, made explicit before inference spends a token. Delegation, made explicit as a designed handoff instead of a hopeful prompt. Attention, kept engaged instead of spectating. Judgment, compiled into fitness functions the build can enforce. Understanding, made observable as a map with dark regions on it. And now the last step, which is the smallest and the most consequential: the observable state made policy. The first five posts made the human layer explicit. The last one makes it enforceable.

I will end with commitments rather than conclusions, because this concept was published with refutation conditions attached and the instrument should be held to the same standard. The Substrate Collapse preprint stakes a falsifiable prediction — systems that look healthy on authorship metrics but measure low on comprehension should suffer disproportionately in novel incidents — and as incident data accumulates against these maps, I will run that protocol and publish the result either way. Every parameter change will keep landing in CALIBRATION.md as a public record of the instrument arguing with its own spec, starting with the two failures above: agent identities excluded from comprehender counts, and the REVIEWED class implemented, both now that this run is on the record. And in twelve months I will re-read the six claims this series made and score them in public.

The map of boleto.ts told me that the library I am most identified with is understood, on the evidence, by no human — including, on the evidence, me.

An instrument that can say that about your own work belongs in the build.

https://github.com/tiarebalbi/comprehension-coverage/issues/29