Zum Inhalt springen

Requirements coverage

How req coverage measures bidirectional traceability — per-requirement rollups, classified unjustified methods, the infrastructure exemption, and the opt-in tolerated-legacy gate.

codecharter req coverage and the MCP req_coverage tool report one coverage model from both directions of traceability, computed by a single shared aggregator so the two surfaces can never disagree on a number:

  • Requirement → code: does each requirement's evidence currently hold?
  • Code → requirement: which methods carry no [Satisfies] justification at all, and why?

Both halves reuse the evaluators and indexes Requirements and Requirements over MCP already describe — no criterion, verdict, or justification walk is re-implemented for this report.

Requirement → code: rollups

Every requirement's criteria are evaluated once, through the same pin evaluator req check --with-code and req_verify use, then rolled up to a single worst-of verdict by a fixed severity order:

Rank Verdict Why it ranks there
5 (worst) violated Proven broken against the current code.
4 stale A decayed positive claim: it once held, and the pinned evidence has since changed — actively misleading in an old traceability matrix, so it outranks an honest non-answer.
3 unknown Evaluation could not decide. An honest non-answer, not evidence of health or decay.
2 open A declared gap — the least alarming non-green outcome, since it is deliberate rather than discovered.
1 (best) holds Every criterion holds.

Never vacuously green. A requirement with zero criteria does not roll up to holds — it rolls up to open with an unbound reason of no-criteria. The same applies to zero anchors (no-anchors) and anchors that all failed to resolve (anchors-unresolved). This is the same "nothing ever reports Holds without having checked" invariant the criterion evaluators already apply, extended to the requirement level.

The frontmatter status key is never an input to the rollup — it is reported beside it. A requirement whose declared status disagrees with its rollup (status: implemented while the rollup is not holds, or status: open while the rollup is holds) is surfaced as a status divergence: reported for visibility, never used to flip the rollup.

Code → requirement: classified methods

Every method in the analyzed code model falls into exactly one of three disjoint buckets, reported as two headline percentages that are never merged into one ambiguous number:

  • justified — reached by at least one non-infrastructure requirement's [Satisfies] justification walk.
  • infrastructure-declared — see Infrastructure exemption.
  • unjustified, itself split into:
    • dead-candidate — no caller is visible to the justification walk, and the method is not a dispatch target (no interface implementation, override, or implementing method reaches it either). This is never proven unused — only not reachable through this walk's visible edges (no reflection, no delegate invocation, no dynamic dispatch is followed). The report always says "candidate", never "dead" on its own.
    • undocumented-requirement-candidate — has visible callers, but no anchor's justification walk reaches it. Carries a nearest_anchored_neighbour: the closest justified method found by a bounded reverse search over callers and dispatch predecessors (capped at four hops), together with the requirement ids that justify it — null when the cap is hit, never a guess. This hint is exactly what req_draft seeds a new requirement from.

justified_percent = justified / total_methods; accounted_percent = (justified + infrastructure_declared) / total_methods. If the corpus has no [Satisfies] anchors at all, both percentages report 0 with an explicit notice, and no classified list is produced — a repo that has not adopted anchoring yet must not receive ten thousand "dead candidates" on its first run.

Infrastructure exemption

A requirement with kind: infrastructure exempts exactly the methods its [Satisfies] anchors name directly — a method anchor exempts that method, a type anchor exempts that type's methods. This exemption is declared, never guessed: there is no heuristic that infers "this looks like glue code" on its own, so mislabeling ordinary business logic as infrastructure is always a visible attribute change in a diff, never a silent classifier decision.

It is also non-transitive by design. Anchoring Program or a dependency-injection registration class does not exempt everything those methods call — only the anchored declarations themselves. A transitive exemption would let one attribute on your composition root exempt half the solution, which is exactly the kind of mislabeling this report exists to keep visible. When a non-infrastructure requirement's justification walk reaches a method also directly exempted as infrastructure, the report emits an info-level req-infrastructure-overreach notice — a signal the infrastructure label may be misapplied to code that business logic actually depends on.

Diff scope

--git-ref/git_ref (or raw diff text over MCP) narrows the classified method list and the rollup list to what the diff touches. Every total and every percentage — including justified_percent and accounted_percent — stays corpus-wide; only the two lists shrink, with a scope: diff notice naming what happened. This mirrors graph_hubs's own scope field: a diff-scoped view never silently narrows what a summary number claims to cover.

The tolerated-legacy gate

Coverage reports a number from the moment you adopt it, even on a large, partially-anchored codebase — but that number should not have to be 100% before the report is useful, and it should not silently regress either. req coverage therefore ships a one-way ratchet, the same pattern the findings baseline uses:

  1. codecharter req coverage --write-baseline records every currently unjustified method's identity (Roslyn declaration id, or a graph-key fallback when the compiler produced none — a rename invalidates a fallback entry by design) to .codecharter/req/coverage-baseline.json.
  2. From then on, a baseline-listed method renders tolerated; a new unjustified method — one not in the baseline — renders new.
  3. The gate itself (non-zero exit on any new method) is opt-in, behind req.coverage.gate.enabled: true in .codecharter/config.yml (default false). The mechanics and the failing-gate exit code exist and are tested regardless of whether any repository has opted in.

Infrastructure-declared methods are exempt by design and never count as baseline debt — only dead-candidate and undocumented-requirement-candidate methods are tracked. The baseline never widens itself: only --write-baseline writes it, and repeated writes of the same set are byte-identical (sorted, de-duplicated, no timestamps).

Report first, gate later — a Goodhart note

Attributes are diff-visible and review-carried; the metric is not a target until the gate exists.

[Satisfies] anchors are something a reviewer sees in a pull request diff the same way any other code change is seen — not a score a tool infers behind your back. Coverage exists to make an honest number visible and to stop it from silently regressing once you decide it should, not to become a target chased for its own sake. Optimizing straight for the percentage — for example, anchoring requirements to code indiscriminately just to raise justified_percent — defeats the point: the number is only as trustworthy as the anchors a human actually reviewed.