DataHub assertion stress test

Make good data fail on purpose.

Quality Mutant injects six deterministic defects into a passing dataset, then measures whether the existing assertions actually catch them.

Interactive proof

The kill matrix

Select any mutant to inspect its exact outcome. The orange row is the counterexample: every current assertion stays green while referential integrity is broken.

Swipe the matrix sideways to inspect every assertion.

Selected specimen

Choose a mutant

Each mutation executes against the same deterministic ecommerce fixture.

Missing control

A non-null key can still point nowhere.

The orphan mutation keeps customer_id populated, so the non-null check passes. A foreign-key assertion would catch the missing parent row.

SELECT count(*)
FROM orders s
LEFT JOIN customers t
  ON s.customer_id=t.customer_id
WHERE s.customer_id IS NOT NULL
  AND t.customer_id IS NULL

Baseline passes6 executable checks; 0 skipped.

Deterministic evidenceEvery mutant and report has a SHA-256.

Writeback stays gatedDataHub payloads are prepared, not sent.

What this proves

The included recorded fixture executes end to end: generate mutants, run assertions, score coverage, identify the survivor, and prepare a custom DataHub assertion result.

What it does not claim

No live DataHub tenant was used for this public demo. The MCP adapter and writeback payloads are present, but external writes remain deliberately disabled.

Reproduce locally

Inspect the source

Repository: REPOSITORY_URL_PENDING_PUBLICATION

Clone the public Apache-2.0 repository.
Run python -m venv .venv && .venv/bin/pip install -e '.[dev]'.
Run .venv/bin/pytest -q, then ./scripts/demo.sh.
Compare sample-output/mutation-report.json with this matrix.