Shift Left Alternative: In-Place Test Reduction for Faster Releases

What’s the biggest bottleneck in the software development life cycle? If you’re a developer, you know the answer is testing. Bloated test run times are a common obstacle in SDLC for a number of reasons. Pull requests can take a long time to run and often clog the delivery pipeline. Committing a change requires testing, but the results of that test may not be available until 12, 24, or 48 hours later, delaying further work. Or, even worse, test results become available right before a release and as a result, a failure results in a pipeline blow up. 

While most modern organizations have evolved to use the DevOps methodology and CI/CD to automate testing, the software testing life cycle is still often less than efficient, with teams running massive, unnecessary test suites for every change.

STLC inefficiencies drive demand for shifting left.

Shift left testing is the practice of moving testing, quality, and performance evaluation to the early stages of the software development life cycle, in order to reduce testing bottlenecks and speed up release cycles. The idea behind testing earlier and often is to shrink the number of bugs and improve quality and accuracy. Instead of only testing at the end of the software development life cycle, shift left testing encourages consistent, early testing to catch small issues before they become major problems.

Shifting left works to reduce cycle times and is a key tenet of the DevOps testing approach, but it is not always possible or the most efficient way to speed up the velocity of testing cycles.

When there are a multitude of tests to run, it can become overwhelming to know which tests to run and which tests to run first. Many will tag certain tests in a large test suite and create manual, time-consuming processes for running these tests earlier in the development process. Additionally, shifting left almost always results in more testing and often must be performed at night, which is not optimal for resources.

Instead of manual testing or shifting left, another option for devs ready to rev up their testing cycles is in-place software testing runtime reduction, or reducing cycle times without shifting tests. 

No need to shift left when you can have in-place software testing runtime reduction.

What if teams could reduce pre-merge tests in-place for faster feedback against more testing volume? That’s in-place reduction for software testing. 

In-place test reduction allows for your team to reduce cycle times without adding additional pipelines or testing more. There are several benefits to in-place software testing runtime reduction, including faster feedback times, resource savings, and boosted efficiency.

The volume of tests before every pull request gets merged will always be high, because most of the time those tests are running every time they push their branch. But, reducing the amount of pull requests delivers big wins in reducing the number of tests run pre-merge as well as the actual run time of the tests. Overall, this in-place software testing runtime reduction on the pre-merge side gives developers the faster feedback they need to meet speedy release times.

Another big benefit to software testing runtime reduction in-place is the resource savings. Whether your team is running massive test suites at a massive scale or using constrained resources (i.e, physical hardware), in-place software testing runtime reduction can save your team significant time, which can free up devs and machines. 

Let’s look at an example: If you’re doing 20 pull requests a day, if this amount is reduced down by 1/3, the amount and frequency of tests run can subsequently be shrunk down. Or, in physical hardware terms: an entire machine could be freed up. For both boutique and larger organizations, runtime reduction in-place is a great way to conserve both dev expertise and hardware.

Using an intelligence solution can greatly increase the efficiency and workflow of your team. Of course, in-place reduction for software testing is not a foolproof method. Like any testing method, there is risk. In this case, the risk of slippage. Slippage refers to the missteps or failures that leak into the next phase during testing. However, there is a way to circumvent this common problem.

Shift left and in-place software testing runtime reduction with Predictive Test Selection.

Predictive Test Selection uses machine learning to choose the highest-value tests to run for a specific change, in order to slash long-running test suites and reduce testing times. Dynamically reorder your tests and focus on what matters most. Every time your tests run, your changes and test results are passed to Launchable to continuously train a model. The training assesses the changes associated with each build and which tests failed when tests ran. By building associations between changed files and which tests tend to fail, Launchable helps you to prioritize which tests to run, and which tests to run first.

Launchable addresses the risk of slippage, by allowing you to choose between three optimization targets to work against:

• Confidence  - What slippage will happen based on my chosen optimization target? 

• Time percentage - What percentage of time will I save per test run?

• Time duration mode - What fixed duration of time can I save per test run?

Regardless of your chosen optimization target, Launchable gives developers useful free tools to get started toward test time reduction. Test Suite Insights help developers quantify the impact of changes in your test suites while personalized Test Notifications in Slack allow you to get direct notifications about your builds and tests, rather than digging through your inbox or CI. 

Whether you are able to tackle shifting left, or you’re hoping to tackle test time reduction in-place, it’s time to tackle runtime reduction with Launchable.