Optimizing your NUnit test runs with Launchable

The Launchable CLI provides a simple, uniform interface between your CI process and the Launchable service.

To make integration even easier, we’ve built profiles into the CLI for popular build and testing tools. These profiles abstract away the nuances of specific languages and test runners so you only have to add a few lines to your build script to get started.

One of these profiles is for the NUnit console runner. In this post, we’ll show you how to connect Launchable with NUnit's console runner to train a model and get faster test feedback.

What is Launchable?

Launchable uses machine learning to identify the most important tests to run for a specific code change. This capability lets you run a smaller set of important tests earlier and more frequently in your software development lifecycle, providing faster feedback to developers without compromising quality.

To enable this, you can use the Launchable CLI to connect Launchable to your test runner both to train a machine learning model and to get and run test recommendations supplied by Launchable.

Connecting Launchable and NUnit3-Console using the Launchable CLI

After installing the Launchable CLI package from PyPI and setting your API key, you’ll need to add three Launchable commands to your build/test pipeline.

First, you need to record a build to tell Launchable the code changes that are being tested. Launchable uses this information to train the machine learning model and to recommend tests to run for those specific changes.

To do this, you add the `record build` command to your CI script. You give the build a name and point the CLI to your Git repository on the CI server (`.`):

launchable record build --name <BUILD NAME> --source src=.

Then, you need to record test results after running tests to train the machine learning model:

launchable record tests --build <BUILD NAME> nunit <PATH TO NUNIT3 XML FILE>

You'll notice that this command uses the same `<BUILD NAME>` value; that's how Launchable learns that these test results relate to the new commits in the build.

Once you’ve started sending test results to train a model, you can start subsetting tests. Over time, the test recommendations will improve as the model learns.

Before running tests, you need to request a dynamic subset of tests from Launchable and pass those to the console runner to run them.

You provide the same build name, a target percentage of test duration to run in the subset, and a list of tests (`list.xml`) that you can create using `nunit3-console --explore`:

nunit3-console --explore=list.xml path/to/myassembly.dll
launchable subset 
  --build <BUILD NAME> \
  --target <PERCENTAGE DURATION> \
  nunit list.xml > launchable-subset.txt

The subset command outputs a list of recommended tests specifically for that buildto launchable-subset.txt which you then pass into the console runner:

nunit3-console --testlist=launchable-subset.txt path/to/myassembly.dll

See docs.launchableinc.com for more documentation and examples.

Now you can run a subset of a long running test suite more often. For example, you could run a subset of long running end-to-end UI tests on every git push instead of only after every merge. Or you could subset your pull request tests to get faster feedback, earlier.

If this sounds interesting to you, sign up for free trial of Launchable now! As you can see above, it’s really easy to get started with the Launchable CLI.