Today we release CodeSpeak 0.3.7. Please find the full release notes at the end of this post.

Automated tests give coding agents feedback before humans do. This is one of the things that makes agents more autonomous and therefore more useful for completing larger coding tasks.

We've all seen agents being "lazy":

• forgetting to run some tests;
• not writing tests for new functionality;
• declaring the job done while tests are still failing...

As CodeSpeak runs agents under the hood, we try to make sure that they are disciplined about tests, and provide some tools you can use for this. In this release, we complete this set of tools with

• codespeak build now checks if the code generator agent has run all the tests and runs them if it didn't
  • (if you don't want to make sure tests are run and fixed, use codespeak impl instead of build);
• codespeak test runs all tests and makes sure that any failed ones are fixed;
• codespeak test --auto-configure discovers the commands that run tests in your project
  • (you can configure these commands manually in codespeak.json, but our agents seem to be pretty good at doing it for you);
• codespeak coverage collects coverage data, identifies what tests are missing and adds them.

This more granular set of commands gives you more control over your workflow. If you want to iterate in a more quick-and-dirty way, you can run codespeak impl and postpone testing the code more thoroughly. If you want to be sure all tests pass, run codespeak build. If you want to invest into a better test suite, run codespeak coverage which make take a bit longer than the other two.

Example

curl -LsSf https://astral.sh/uv/install.sh | sh

uv --version

uv tool install codespeak-cli

codespeak --version

codespeak login

Initialise the project, write a spec, and build

codespeak init --mixed

This creates codespeak.json that contains all the configuration.

Now, write your spec in spec.cs.md, and build it:

codespeak build --spec spec.cs.md

You will get a message like this:

Processing spec 1/1: spec.cs.md
A placeholder test runner configuration was added to codespeak.json for spec 'spec.cs.md'. 
Please fill it in with actual values, or run 'codespeak test --auto-configure --spec spec.cs.md' to auto-detect it.
Built successfully.

Time to auto-configure:

codespeak test --auto-configure --spec spec.cs.md

Output:

Auto-configured test runner for spec 'spec.cs.md'

Now, if we make a change to the spec and build again, we'll see that CodeSpeak keeps track of the tests:

Processing spec 1/1: spec.cs.md
All 9 tests passed.
Built successfully.

If you think some tests may be missing or want to improve the quality of your test suite, refer to this tutorial for details on codespeak coverage.

Conclusion

In the coming weeks, we'll keep improving CodeSpeak and refining the set of tools we provide. Please try it out and let us know what you think on Discord.