Automation drives reduction in software testing errors
Catapult was engaged to help a multinational telecoms firm transform a large-scale telco stack (30-50 systems, 1500 developers, and 300 manual testers) from a traditional waterfall development model – where software testing is done at the end of the process – to a continuous delivery model.
The situation
The company was experiencing heavy pain because insufficient testing was being done during development, leaving the 300 testers with a tough task at the end of the process. We were invited to help ‘move the pain left’ by automating key tests, enabling them to be done as early and often as possible.
The solution
To enable continuous delivery, the client needed integration testing between discrete teams/components earlier, and much more regularly. One of the key issues was that code from the 30-50 component teams was developed and delivered in silos; half of the client’s 6-week software testing window was spent dealing with basic integration defects because each component had been developed in isolation.
Our solution involved:
- Setting the expectation that siloed sprint plans would now interlock, forcing core scenario testing at the end of every 2-week sprint so that basic integration problems were caught and solved early.
- Test planning was managed within the component teams, so that testers could understand and plan meaningful incremental tests in advance.
- Developing automatic tests to check these important functions quickly and easily.
- Automated document control, data-prep, and defect isolation.
The results
From a standing start, and with no material investment, Catapult successfully helped the client reduce critical (P1/2) errors by 80% over 12 months. The number of non-critical errors (P3/4/5) found increased substantially, as the testers were now able to perform their job correctly.
