I’ve been software testing for over twenty five years and have dabbled in automation for most of that time, but in the last ten years working on numerous agile projects, automation has become more and more important, to the extent that pretty well all the testing on the projects I work on is automated.
I’ve created numerous frameworks as I’ve moved from project to project, hopefully improving as I’ve gone along. This blog doesn’t give any solutions to the problems you may encounter, but hopefully gives a few pointers on things to think about before you start.
Remember, it’s a lot easier to get to where you want to go if you start off from the correct place.
Start Off on the Right Foot
You may not think it but your automated tests are basically a software project. If you don’t start off in the correct way it will quickly become a mess, difficult to maintain and almost impossible for anybody else to pick up.
Speak to the developers, this is what they do – if you structure your project properly in the first place, everything down the line will be much more straightforward. You will also get better buy-in later if you have followed their advice.
Have an Automation Strategy and Stick To It
You should basically automate every test you think you will need to run more than once, but how you automate is important.
- Speak to the developers – find out where the risks are and write tests accordingly.
- Don’t double up on tests. If the developers are happy that something is covered by unit tests – e.g. text box validation – don’t write a test for it.
- Test the easy way first. If you can test against a back end service e.g. database queries, enrichment, reference data, do it directly. These are the easiest tests to write and maintain. The most difficult tests to write and maintain are tests which drive the GUI, exhaust all other options (unit tests, direct tests against APIs etc.) before going down this route.
- Think repeatable. Your tests need to be repeatable, i.e. every time you run a test the same input must give you exactly the same output (or at least the bits you care about must be the same). So thinking about test data, environments etc. is more important than when manual testing. It is also important to think about how the tests are structured. For example, can they run in any order, are they independent.
Don’t Do Too Much
There is a temptation that once you have gone to the effort of creating a test it seems a shame not to use it and to create a lot more tests which basically do the same thing. It makes it look like you have loads of tests and keeps everybody happy. Don’t do it. Just because it is easy to do that’s not a good reason for doing it. Spec out your tests as you would for manual testing and create your tests accordingly. The more tests you have; the more you have to maintain, the longer they take to run and the more chance that something will fail incorrectly.
Pick the Correct Technology and Tools
You’re in an agile environment, test is part of the development cycle and should be part of the process you use to manage the lifecycle of the delivery items. Testing shouldn’t have some sort of life of its own, therefore it doesn’t need a test management tool.
Having said that, you need some way of having human readable tests – preferably as part of the automation framework. We use Cucumber. Axe is an option which uses spreadsheets. As well recording test steps, it gives a way for testers with no automation experience a way to become involved in the automation process. The first step in automation (and one of the hardest) is designing repeatable tests so just because you can’t write code doesn’t mean you can’t be involved in the automation process.
There are now open source automation tools for GUIs e.g. Selenium. These can usually be driven by various programming languages. Make a sensible choice – if you are in a team of mainly Java developers it would be perverse to go down the C# or Python route.
Speak to the Developers (Again)
In the the agile world we are all part of the same team. The ratio of developers to testers is likely to be higher than in the old world of separate dev and test teams. In an automated environment testing needs skills which weren’t previously required..
Be Pushy
- If you need help ask. You’ll probably need to learn the basics of writing code yourself, but once you have those basics it is much easier to build on something already there than to start from scratch (see first point).
- One of the biggest challenges of writing ui tests is creating robust selectors for the elements on the ui. Developers should be able to help here; for example by adding unique ids to elements. A small effort on their part could save massive test effort.
- Don’t struggle. It’s good to work things out for yourself, but there comes a point when you’re wasting time or worse, bodging.
- Don’t make major decisions (eg data drivers using xml files) without discussing with a developer (they may have a better solution, like JSON)
- You may be surprised how willing the rest of the team are to help, after all they do an awful lot of testing themselves which used to be invisible to the test team. Automated system testing should build on the existing unit testing.
