Breakdown your Monotest

Too many times I am hearing developers and QA complaining about the long feedback from their test. It is true, E2E test do take a long time to execute and give feedback, but that doesn’t have to be always the case. In many cases we don’t see that we can speed up the process and save us a lot of time and headache.
Here are some of my tips that I can give you, from my experience.

Run in parallel

Yes, I know you probably know this, but it is always helpful to say it. Try to run the test in parallel. Each Framework has it’s own ways and methods. Find the one that is used in your tests framework and use it.

More test suites more machines

I tried to parallel Cucumber tests, it was hard…
Also tried to parallel Spock and it had flaky nondeterministic failures due to randomization, I also needed a powerful machine to execute all tests at the same time.
So, I tried another solution. I broke the tests into different suites, I split the MonoTest into smaller test suites. Some were focused on application, other were focused on smoke/regression and some tests were joined into suites that to make execution time faster. This enabled me to:

- have faster feedback time
- control the test suite sequence
- have a flaky suite for those tests that need quarantine

Each suite would get its own runner, meaning that I could use less powerful machines. I broke down the tests to as many machines I needed to get the feedback in a time that we, in the company, agreed on. We got the execution time from 4 hours to 1 and a half hour. We also controlled the tests execution, which resulted in no nondeterministic failures and easy debugging of Flaky tests.

What if I can’t run in parallel due to system limitations

Some application have issues running in parallel, I was working on such a project. In order to run in parallel we wrote a daemon in Python to keep the state of the machines, a Foreman for the tests execution. It helped us where the Framework was lacking in coordinating the state between machines and suites.

I hope this article helps you in the future, to speed up your tests for a faster feedback and a faster deployment cycle.