codecharter test

codecharter test <path> [options]

codecharter test runs a rule against a set of curated code snippets and checks that it flags the snippets it should and leaves the rest alone. It is the regression-test harness for your .cgr rules: write the spec once, and every later edit to the rule is checked against the same expectations.

<path> can be:

• a single .cgr rule file (the matching .spec.md next to it is used),
• a single .spec.md file (the same-named .cgr rule file in the same directory is used — foo.spec.md tests foo.cgr), or
• a directory, in which case every spec under it is discovered and run. Rules without a .spec.md sidecar are skipped silently; they are not reported as failures.

The .spec.md format

A spec is a Markdown file that pairs a rule with example snippets. It has two top-level sections:

• ## hits — snippets that the rule must flag. If the rule produces no finding for one of these, the spec fails.
• ## misses — snippets that the rule must not flag. If the rule produces a finding for one of these, the spec fails.

A spec can also start with an optional title line (# ...) and an intent blockquote (> ...) describing in one line what the rule enforces; the scaffold generates both.

Inside each section, every example is a ### heading followed by a fenced csharp code block. The heading text — any short description — is the case label:

# repository-naming

## hits

### case: class without the Repository suffix
```csharp
public class UserStore
{
}
```

## misses

### case: correctly suffixed class
```csharp
public class UserRepository
{
}
```

The full text of each ### heading is the label the test output uses to point at the exact snippet that failed (the case: prefix in the example above is a convention, not a requirement). The fenced block must use the csharp language tag; blocks in other languages are ignored.

Examples

# Test one rule against its sibling spec
codecharter test .codecharter/rules/repository-naming.cgr

# Test a spec file directly
codecharter test .codecharter/rules/repository-naming.spec.md

# Discover and run every spec in a directory tree
codecharter test .codecharter/rules

Options

Option	Default	Description
--output json	console	Emit machine-readable results instead of the human summary. Each case reports its label, the expected outcome (hit/miss), and the actual outcome.
--no-color	off	Disable ANSI color codes in console output. Useful when piping logs.
--scaffold	off	Generate a starter .spec.md next to the rule instead of running the tests. The skeleton contains the rule's title, an intent line, and one placeholder case in each of the ## hits and ## misses sections. Existing specs are never overwritten.
--license <path>	none	Use a specific codecharter.license file instead of the default license discovery. See Licensing.

Exit codes

codecharter test is CI-friendly: the exit code tells a pipeline whether the specs passed without parsing any output.

• 0: every case in every spec matched its expectation.
• 1: at least one case failed (a missing hit or an unexpected hit on a miss). A rule that fails to parse also exits 1: all of its cases are reported as failed.
• 2: nothing to test — the given path does not exist or no .spec.md was found for it.
• 6: license error (missing, expired, or not valid for this CLI version).

With --scaffold the codes mean: 0 — skeleton written, 1 — a spec already exists (it is not overwritten), 2 — the rule file does not exist.

In VS Code

The VS Code extension (0.5.0 and later) wraps the same harness behind Command Palette commands:

• CodeCharter: Test rule spec — runs the spec for the rule in the active editor.
• CodeCharter: Test all specs — discovers and runs every spec in the configured rules directory (the codecharter.analyze.rulesDirectory setting; rules/ in the workspace root by default).
• CodeCharter: Scaffold spec for current rule — creates a starter .spec.md next to the current .cgr file, the editor equivalent of --scaffold.

Results appear in the editor so you can jump straight to the failing case.

Related

• codecharter validate checks that a rule parses; codecharter test checks that it flags the right code.
• MCP rule authoring drives the same hit/miss check from an AI assistant via the test_rule_spec tool.