---
url: >-
  https://thekevinscott.github.io/testing-conventions/reference/checks/mutation.md
description: >-
  The mutation check — a binary gate, not a score: no unexplained surviving
  mutant on a pull request's changed lines; the engines, timeouts, and the
  line-scoped exemption.
---

# `mutation`

Break the code, and a test has to fail. `mutation` mutates the lines a pull request touched,
runs the unit suite against each mutant, and fails on any surviving mutant not exempted with a
reason — **a binary gate, not a score**. This page is the complete record of the check.

## Why this check exists

Coverage proves a line executed; it says nothing about whether any test asserted on the result —
a test that calls the function and checks nothing still covers every line it touches. Mutation is
the [unit ladder](/explanation/#the-unit-ladder-exist-→-run-→-verify)'s top rung: does the test
**verify** the code? It is also the rung an agent can't game: an LLM can reach 100% coverage with
assertion-light tests, and mutation only passes when the tests actually pin behavior.

The gate is binary rather than a score because of **equivalent mutants**: some mutations are
semantically identical to the original, no test can ever kill them, and detecting them is
formally undecidable — so a score floor can be unreachable through no fault of your tests. The
rule is instead: **no unexplained surviving mutant on the lines a change touched** — zero
survivors except what you exempted with a reason. Diff-scoping is what makes a binary gate
tractable: whole-tree mutation is too slow to gate.

## What it enforces

A **mutant** is the code with one small, deliberate fault (`+` becomes `-`, `>=` becomes `>`,
`return x` becomes `return null`). The tool makes each mutant on a changed line and runs the unit
suite against it: tests fail and the mutant is **killed**; tests pass and it **survived** — and
an un-exempted survivor fails the check.

A **declaration-only module** — a file with no function or control flow, the same subject
[`colocated-test` uses](/reference/checks/colocated-test#what-it-enforces) — is never a mutation
subject: a mutant the engine finds there is dropped before judging, so editing one demands no
survivor explanation.

In Rust, a mutant inside an item a `#[cfg(not(test))]` gate keeps out of the test build is dropped
the same way. The unit tier runs `--lib`, which sets `cfg(test)`, so that code is not in the
binary the suite runs and no test can reach it — cargo-mutants mutates it anyway, because it reads
the source rather than the build, and reports a survivor nothing could have killed. The exclusion is
item-level and only covers a gate a test build genuinely cannot satisfy: `#[cfg(not(test))]` and
`#[cfg(all(not(test), unix))]` qualify, `#[cfg(any(not(test), unix))]` still compiles under
`cargo test` and stays a subject. This is what makes the
[binary entry point `colocated-test` documents](/reference/checks/colocated-test#a-rust-binary-s-entry-point)
mutation-clean.

In Rust, cargo-mutants' whole-body replacement is also dropped on a **subprocess seam**: a
**branch-free** function that runs a `std::process::Command`, directly or through another function
in the same file that does. Calling one spawns a process, so no unit test can observe that its body
was replaced by `Ok(String::new())` — and the tiers that can, `tests/` and the e2e suite, are
outside the unit-tier `--lib --bins` scoping. Nothing else is dropped: the mutant's genre must be
cargo-mutants' `FnValue`, so a mutation *inside* a seam is still judged.

Branch-free is the whole line. A body holding a decision — the comparison that picks an argument,
the `if` that adds a flag — keeps every mutant it has, including the whole-body one, because the
decision is assertable once it moves out: a pure function that returns the argv, and a seam that
runs it. That is the shape the mutation adapter's own `install_argv` + `execute` pair already
takes. The drop is for the irreducible last mile, not for everything that spawns.

In Python, a mutant inside a **type annotation** is dropped: a parameter annotation, a return
annotation, an annotated assignment's type, and a `type` alias's value. An annotation holds type
metadata, and the interpreter need never evaluate it — `from __future__ import annotations`
(PEP 563) keeps it a string, and from Python 3.14 on every annotation is lazy (PEP 649) — so the
rewritten operator in `def f(x: dict | None) -> int | None` executes no code, at import or at call,
and the suite has nothing to fail on. Stryker and cargo-mutants read a type as a type and offer no
mutant there; cosmic-ray matches operators over the token tree, so the adapter scopes the drop to
the annotation's own span and skips those mutants before the suite runs.

The drop is scoped to the annotation's span, not to its line. A default value beside an annotation —
`def f(x: int | None, n: int = 10 - 4)` — is live code evaluated when the `def` runs, sits outside
the annotation's span, and keeps every mutant it has. So does a `cast(int | None, x)` call, whose
arguments the interpreter does evaluate.

Each language wraps its standard engine; the tool drives the engine, and you provide the test
runner that runs your own suite:

| Language | Engine | You provide |
| --- | --- | --- |
| TypeScript | [Stryker](https://stryker-mutator.io/), driven via its Node API by an adapter bundled in the npm package | `vitest` |
| Python | [cosmic-ray](https://github.com/sixty-north/cosmic-ray), driven via its library API by an adapter bundled in the wheel, with a baseline run that requires the clean suite to pass; each mutant's run ends at its first failing test | `pytest` |
| Rust | [cargo-mutants](https://github.com/sourcefrog/cargo-mutants), provisioned on first use (a pinned `cargo install` into the tool's own cache) and run from there; concurrent invocations share one provisioning install rather than each racing to install their own | the cargo toolchain that builds and tests your crate |

A run lists each survivor with its file, line, and mutation — the engine's mutator and the source
that mutator produced — and fails on any un-exempted one. Survivor paths are reported relative to
the scanned path, so exemptions address the same paths every other check uses.

Each engine runs where its ecosystem expects: cargo-mutants at the crate root, cosmic-ray at the
scanned path, and Stryker at the **package root** — the nearest directory at or above the scanned
path holding a `package.json`. cargo-mutants addresses files relative to the crate's **cargo
workspace root**, so the tool rebases the diff it feeds the engine onto that root and rebases the
engine's reported paths back — a crate that is a member of a larger workspace is diff-scoped,
gated, and reported exactly as a standalone crate is. A [`[rust] features`](/reference/config#rust-features)
list rides on every cargo invocation the run makes, so the crate's test targets are **built** with
the features too: an integration test that names a feature-gated item compiles, and the unmutated
baseline the whole run rests on builds. All three mutate the tree **in place**: Stryker applies each mutant
to the package's real files (a backup lives under `.stryker-tmp` for the duration of the run, and
the run restores every file when it ends), so everything a run touches resolves exactly as it does
in your editor — a source that reaches above the scanned path (`import pkg from '../package.json'`,
a shared `../tsconfig`) resolves through the real tree, the package's `tsconfig.json` is read where
it lies, and tooling resolves from the package's own `node_modules`. Mutation stays scoped to the
scanned path, and the scanned path's colocated unit suite is what judges each mutant; the package's
other suite tiers (`tests/`) stay out of the run. Rust reaches that scope with
`--cargo-test-arg --lib`: binary unit tests and the `tests/` tree do not judge library mutants.
A survivor under `--lib` means the library unit tier does not assert the mutated line. An
integration or e2e test may legitimately assert it; a narrow line-scoped exemption can record
that test and reason. The scan path narrows **which** of your test
files judge the mutants, and your runner keeps deciding what counts as a test and where it is
rooted — so a `vitest.config.ts` at the package root whose `include` names `src/**/*.test.ts`
resolves that pattern against the package root, exactly as it does when you run vitest yourself.
The run mutates production source alone and leaves your test files intact: a `*_test.py`,
`test_*.py`, or `conftest.py`; a `*.test.*` or `*.spec.*`, and anything under `__tests__/`; a
Rust `#[cfg(test)]` module — at any depth beneath the scanned path, not just its top level.
In-place mutation is built for the ephemeral CI
checkout the workflow runs in; run it locally on a clean working tree, since the sources hold
mutants while the run is live.

Each engine's own configuration is the tool's, and the test runner's configuration is yours.
cosmic-ray's whole config (module paths, excludes, timeout, test command) is rendered in-process
by the bundled adapter, so a cosmic-ray config file in the tree plays no part in the run; Stryker
is driven through its Node API with the adapter's options — the runner, in-place execution, the
mutate scope — taking precedence over a `stryker.conf.json`, which still supplies anything the
adapter leaves unset. The runner judging each mutant resolves its configuration exactly as in
your own runs: pytest finds its rootdir, config, and `conftest.py` files with its own upward
search from the scanned path, and vitest resolves the package's `vitest.config.*` with its own
upward search from Stryker's package-root, in-place run.

### Timeouts

A mutant whose run outlasts its budget is **inconclusive** — neither killed nor survived, dropped
from the survivor set. A **baseline** timeout (the clean, unmutated suite) is a loud error: a
suite that can't finish in its budget is untrustworthy as a judge of any mutant, exactly as a
baseline that fails is. Each engine scopes the per-mutant timeout to the clean suite's own
measured runtime, so a legitimately slow suite keeps a proportional budget.

### Reporting

A failing run lists each survivor as `<file>:<line>: <mutation>`, where the mutation names the
engine's mutator and the source that mutator produced:

| Language | Survivor line |
| --- | --- |
| TypeScript | `index.ts:6: EqualityOperator (-> n >= 0)` |
| Python | `calc.py:6: core/ReplaceComparisonOperator_Gt_GtE (-> return n >= 0)` |
| Rust | `src/lib.rs:7: replace > with == in is_positive` |

Stryker and cosmic-ray name their mutator and then the source it replaced; cargo-mutants states
both in one description of its own. A mutant whose engine reports no replacement is listed by its
mutator name alone.

A passing run states which fact made it green:

* `unit mutation: no surviving mutants — every mutation was caught (6 mutant(s) tested)` — the
  engine ran, judged that many mutants conclusively (always at least one), and the suite (or a
  reasoned exemption) accounted for every one.
* `unit mutation: the engine found no mutants to test` — the changed lines hold source, but no
  mutant sites (a signature move, a `const` value, a tests-only edit, a Python annotation-only
  edit, every mutant a declaration-only module produced): the engine ran and had nothing to judge.
* `unit mutation: no mutatable changed lines — engine not run` — the diff's changed lines hold
  no source files for the language (a docs-only or workflow-only pull request), or for
  TypeScript, only declaration-only modules — filtered out before Stryker ever runs — so the
  engine was skipped.

All three exit `0`; the distinction is reporting, not gating. Each line names its own evidence,
so the log tells a validated pass from a vacuous one — the all-caught line always carries a
non-zero count.

The Rust arm holds the zero-mutant outcome to proof. cargo-mutants applies the changed-line
filter itself, so when it reports no mutants the tool lists the crate's mutants
(`cargo mutants --list`) and checks that none sits on a changed line of the diff it fed the
engine; a filter that matched nothing while mutants exist on those lines is a hard error naming
both counts, never a pass. The error lists each dropped mutant in the survivor line's shape —
`src/lib.rs:7: replace > with == in is_positive` — one location per site. The cross-check is
Rust-only by construction: the TypeScript and Python arms compute the changed-line scope
tool-side and hand their engines explicit ranges, so there is no engine-side filter to verify.

## When it runs

Pull requests only, as its own job per language, diff-scoped to the `<base>...HEAD` changed
lines. It installs and runs from the derived
[package root](/monorepo#source-vs-the-package-root), provisioned like the
[coverage jobs](./unit-coverage#when-it-runs); for TypeScript, your project dependencies must
include `@stryker-mutator/core` and a runner plugin. The
[`gates` input](/reference/workflow#inputs) names it `mutation`.

### TypeScript requires vitest 4 or below

Stryker's vitest runner runs no tests per mutant on **vitest 5 and above**: the dry run and
per-test coverage attribution both succeed, so every mutant scores as `survived` even when the
suite kills it. The adapter refuses to run rather than report that, naming the version it found.
Pin `vitest` and `@vitest/coverage-v8` to `^4`, or leave `mutation` out of `gates` to skip the
job. An exemption cannot lift this failure — exemptions apply to results, and a refused run
returns none.

## Configuration

The check has **no percentage key** — the gate is binary, and config can't loosen it. Its tuning
surface:

* A `mutation` exemption is **line-scoped, never whole-file**: the entry carries a `lines` list
  naming the exact lines whose survivors are explained (confirmed equivalent, deliberately
  defensive, or covered by an integration or e2e test outside the unit tier), with a required
  `reason` that names the relevant test — see
  [the exemption schema](/reference/config#line-scoped-exemptions). The determinism guard rejects
  a listed line whose mutants were all caught.
* [`[rust] features`](/reference/config#rust-features) — cargo features the run enables on every
  cargo invocation, test-target builds included.
* [`build_command`](/reference/config#build-command) — a pre-suite build the manifest can't
  express.

## Learn more

* [Explanation — Mutation](/explanation/mutation): the worked example coverage can't catch,
  equivalent mutants in full, and why it matters for agents.
* [Respond to a red check](/guide/configure#exempt-specific-lines-coverage-mutation): exempting a
  surviving mutant, step by step.