Improve Test Coverage

codespeak coverage runs your tests, measures coverage, and adds tests to bring it to a target level. This is useful both for improving existing test suites and for ensuring CodeSpeak-managed code stays well-tested.

Prerequisites: Complete the Installation steps first.

Why test coverage matters

AI code generation is as good as the test suite that verifies the changes. Coding agents don't just generate code — they find and correct their own mistakes. The better the test suite, the more bugs it catches, and the better the results agents deliver.

CodeSpeak uses agentic code generation under the hood, and benefits from good tests as much as any other coding tool.

What is code coverage

Code coverage measures what percentage of your code is actually executed by the test suite. For example:

if temperature > ALARM_THRESHOLD:
    indicator_color = RED
else:
    indicator_color = GREEN

A complete test suite tests both paths (above and below the threshold) for 100% line coverage. If the else branch is never tested, coverage drops to 66.6% — revealing a gap.

codespeak coverage finds these gaps and adds missing tests.

Project setup

We'll use CodeSpeak's fork of MarkItDown, which already has a CodeSpeak project initialized.

git clone https://github.com/codespeak-dev/markitdown markitdown-codespeak && cd markitdown-codespeak

Set up the environment:

uv venv --python=3.12 .venv
source .venv/bin/activate
uv pip install hatch

Auto-configure the test runner

CodeSpeak needs to know how to run tests with coverage reporting. Auto-detect the command:

codespeak coverage --auto-configure

CodeSpeak analyzes the project and writes a test runner command to codespeak.json:

Auto-configured test runner for spec 'packages/markitdown/src/markitdown/converters/eml_converter.cs.md':
cd packages/markitdown && hatch run pytest tests/ --json-report
--json-report-file={tests_report_file} --cov=src/markitdown --cov-branch
--cov-report=json:{tests_coverage_report_file} --tb=short

Run coverage improvement

codespeak coverage --target 100 --max-iterations 5

CodeSpeak runs your tests, identifies uncovered code, and iteratively adds tests:

Initial state: ran 227 tests, observed 6 test failures. Coverage: 84%
Iteration 1: ran 231 tests, observed 8 test failures. Coverage: 88%
Iteration 2: ran 231 tests, observed 6 test failures. Coverage: 100%
Reached target coverage.

Pre-existing test failures are ignored during improvement. CodeSpeak added 4 tests and achieved 100% coverage in 2 iterations.

Next steps

TakeoverTesting Workflow