Update Cornucopia: Demo Space, Subset Request Summary, Raw CLI Profile

Lots to be thankful for this month, including the latest Launchable product updates. If you want to receive product news including feature and function releases, head over to our new Changelog and subscribe for updates.

November brings the new Demo Workspace where you can see Launchable in action, as well as new dashboard updates including the new Subset Request Summaries and your workspace’s Comprehensive Curve. Also new this month, customers with custom-built test runners can get started on Launchable’s CLI with the new raw profile.

Introducing the Launchable Demo Workspace

You can now view a demo Launchable workspace. Dive into the demo and get a preview of how Launchable could help your test cycles before you sign up and start sending test data. We’ll be keeping the demo environment updated with the latest features so you can see the newest ways Launchable can increase testing velocity.

View Subset Request Summaries in the Launchable Dashboard

View a summary of each launchable subset request in the Launchable dashboard. This new view shows how much testing time the subset saves (when run) and how many tests were cut. Use this view to understand how much time you're saving and if you need to adjust your subset target to save even more time.

Subset at the Testcase Level with pytest

The recently released CLI version 1.30.0 improves the pytest profile to enable subsetting at the testcase level. Subsetting at this level, rather than the file or class level, enables fine grained subsetting and keeps you from running unnecessary tests.

To take advantage of this, update to the latest version of the CLI and update your integration scripts to run pytest --collect-only -q and pipe that into launchable subset.

View Your Workspace's Comprehensiveness Curve in the Launchable Dashboard

You can now view your workspace's comprehensiveness curve in the Launchable dashboard. You can use this in combination with the existing confidence curve to select a subset target. A comprehensiveness curve tells you the percentage of test failures in a failing run you should expect to catch based on how much time has been spent running tests.

"Raw" CLI Profile for Custom Test Runners

The Launchable CLI is the primary interface between your test runner and the Launchable platform. We've built profiles for common test runners and build tools that abstracts away the specific requirements of that tool for listing tests to run, formatting lists of tests for input into filters for subsetting, and handling nuances with test reports.

This approach is great for getting started quickly with common tools, but it also meant that customers who use custom-built test runners were out of luck. To fix that, we've just released a raw profile for the CLI. This profile is a more low-level interface that you can wrap with your own functions.

If your team uses a custom test runner and have been waiting to try Launchable, now's the time! Check out the documentation and contact us to get started.