CodeCharter is deterministic. When local and CI show different findings, there is a concrete cause. Here are the most common ones.
1. Different CodeCharter versions
The most frequent case. A different version runs in CI than locally.
Check:
codecharter --version
Compare locally and in CI.
Fix: bring both to the same version. Recommendation: pin in CI (see Versioning) and install the same version locally.
2. Different rule configuration
CI sees a different rules/ version than you do locally, because you are working on a branch that does not yet have the change.
Check: run git status locally and check which commit was checked out in CI.
Fix: make sure rules/ and .codecharter/config.yml are committed and that both sides see the same commit.
Also note how rules are resolved when no --rules flag is set: the CLI first looks for a rules/ directory in the current working directory and, if there is none, falls back to the sample rules shipped next to the binary. The GitHub Action behaves the same way: without a custom rules directory it uses the sample rules baked into its image. "No flags" can therefore mean different rule sets locally and in CI. When in doubt, pass --rules explicitly on both sides.
3. Solution state
CodeCharter uses MSBuild to load the solution. CodeCharter analyzes the files on disk: if CI works on a clean checkout with all NuGet packages restored, but you have uncommitted local changes or packages are not restored, the model will differ.
Check:
dotnet restore
dotnet build
Fix: run a clean dotnet restore && dotnet build locally before codecharter analyze.
4. Different --fail-on thresholds
Without --fail-on, codecharter analyze exits with code 1 as soon as there is any finding, regardless of severity. With --fail-on error, only error-level findings fail the run. The flagless default is therefore the stricter gate: if CI runs with --fail-on error and you run locally without flags, a warning makes your local run red while CI stays green.
Check: look at your CI config to see which flags are set.
Fix: run locally with the same flags:
codecharter analyze MySolution.sln --fail-on error
5. Different --severity thresholds
--severity controls the minimum severity that appears in the report (default: info). If one side runs with a higher threshold, for example --severity warn, info-level findings disappear from its output and the two findings lists look different.
Fix: use the same --severity value on both sides.
6. Diff-scoped analysis
With --diff or --git-ref, CodeCharter reports only findings on changed lines, and the --fail-on gate also applies only to those findings. The GitHub Action's diff: true input enables the same behavior on pull requests. CI can then legitimately show fewer findings than a full local run.
Check: whether CI uses --diff, --git-ref, or the Action input diff: true.
Fix: reproduce the CI scope locally, for example:
codecharter analyze MySolution.sln --git-ref main..HEAD
7. Baselines
If CI passes a --baseline file, all findings recorded in it are suppressed; output and --fail-on apply only to new findings. A local run without the baseline shows everything.
Fix: run locally with the same baseline file, or remove --baseline on both sides to compare full results.
8. Platform differences
CodeCharter is platform-deterministic. Same findings on Windows and Linux. If you still see differences, that is a bug. Write to us with a minimal repro.
Debug workflow
First check the exit code, not only the findings list. Exit code 1 means findings tripped the gate. Other non-zero codes mean the run itself did not complete normally: 2 (missing codecharter.lock.json when portal profiles are configured), 3 (solution could not be loaded, or the portal was unreachable during restore), 4 (lockfile drift), 6 (license error). A red CI without any findings usually points to one of these.
If nothing helps, run both with --verbose and diff the logs:
codecharter analyze MySolution.sln --verbose 2> debug-local.log
# CI: same, save log as artifact
diff debug-local.log debug-ci.log | less
In --verbose mode CodeCharter prints to stderr:
- Which path is being analyzed
- Which rules directory is used and how many rules were loaded from it
- How many types and methods the analysis found
The verbose log is only a few lines, so the diff immediately shows whether both sides use the same rules directory, load the same number of rules, and see the same code model size.
If you cannot find it
Write to us at [email protected] with your CLI version, the --verbose logs from both sides, and the diff of the findings lists.