In a world where technology is evolving rapidly it is important to keep up-to-date with newer softwares and ways of working, moving away from legacy and ultimately preventing being left in the dark. Whether it’s inherited and outdated, or was produced in-house some time ago, at Naimuri we endeavour to keep inline with the latest (and more importantly, the best) tech and tools. However, this isn’t always an easy feat as we learned when faced with the task of transitioning a large suite of automated tests to Cucumber.
Why Fix What’s Not Broken?
When Naimuri was formed, back in 2015, we took on development and support of a complex, monolithic system accompanied by a huge suite of Selenium tests. Though the tests were comprehensive, they weren’t without their faults. Many of them were out-of-date, difficult to understand, and in parts, fragile and flaky. Cue Cucumber…
Cucumber is a framework which allows test scripts to be written (and read) without any coding knowledge. Tests are controlled by a plain English feature file written in Gherkin syntax (Given, When, Then), making it easy to understand what is happening at any given point in the test and to identify failures in test reports. This also makes it possible for any stakeholder or new team member to understand what a test does and what areas it covers. Due to its simple test script architecture, Cucumber provides easy reuse of code, meaning it is efficient to maintain.
The Hard Part
Moving forward to Cucumber wasn’t easy or quick. There are many difficulties to overcome when moving a large test suite to a brand new approach with a different way of working. The first hurdle for us was familiarising the team with the framework. Then the real struggle was to convert the old tests to the new Cucumber format. One way that we made the process easier was to drastically cut down on the amount of tests. We did this by ordering them by priority and coverage of the system. We deemed that having fewer tests covering more areas was better than having a lot of tests covering just a few areas. This would give us a stable base where we could then expand in each area, as and when necessary.
Reaping the Rewards
Since transitioning the test suite over to Cucumber things have been much smoother:
- The tests are simpler to maintain. High coverage areas are where Cucumber works best as a lot of methods are reusable. Therefore, when working in an area where a lot of tests are required, these methods will likely be reused, thus saving time and effort. This has been invaluable to us given the large size of this test suite.
- The feature files are used to communicate the test scenarios across the team. The Gherkin syntax is readable by all team members and so we’re able to use the feature files to confirm a common understanding of how we expect the application to behave.
- We now have a robust test suite that provides much more value. When tests fail it’s now for a genuine reason (a bug in the code or change in code needs reflecting in the test). We’re no longer wasting time re-running tests or trying to investigate intermittent failures.
- We have incorporated the Cucumber tests into overnight Jenkins builds running the automated tests overnight. This enables us to ensure the system’s reliability and to flag-up any areas which have had changes made to them and/or are failing.
Although it certainly wasn’t quick or easy, moving these legacy tests to Cucumber was definitely worthwhile. I can’t imagine where we’d be now if we hadn’t bit the bullet (probably debugging a flaky test that fails once every few runs for no known reason!)
