Automation drives reduction in software testing errors
Catapult was engaged to help a multinational telecoms firm transform a large-scale telco stack (30-50 systems, 1500 developers, and 300 manual testers) from a traditional waterfall development model – where software testing is done at the end of the process – to a continuous delivery model.
The situation
The company was experiencing heavy pain because insufficient testing was being done during development, leaving the 300 testers with a tough task at the end of the process. We were invited to help ‘move the pain left’ by automating key tests, enabling them to be done as early and often as possible.
The solution
To enable continuous delivery, the client needed integration testing between discrete teams/components earlier, and much more regularly. One of the key issues was that code from the 30-50 component teams was developed and delivered in silos; half of the client’s 6-week software testing window was spent dealing with basic integration defects because each component had been developed in isolation.
Our solution involved:
- Setting the expectation that siloed sprint plans would now interlock, forcing core scenario testing at the end of every 2-week sprint so that basic integration problems were caught and solved early.
- Test planning was managed within the component teams, so that testers could understand and plan meaningful incremental tests in advance.
- Developing automatic tests to check these important functions quickly and easily.
- Automated document control, data-prep, and defect isolation.
The results
From a standing start, and with no material investment, Catapult successfully helped the client reduce critical (P1/2) errors by 80% over 12 months. The number of non-critical errors (P3/4/5) found increased substantially, as the testers were now able to perform their job correctly.
Testimonial
Catapult undertook to help the UK Maritime and Coastguard Agency transform the function and management of the UK Shipping Register.
“The effective digitisation of the UK Ship Register is a central element to the creation of a new operating model for the service; fit for the 21st Century. The solution that has been developed is elegant in its design, thorough in its understanding of the customer and simple in its execution – leading users seamlessly through the process. It means that the registration services of the UK flag have the ability to be world leading, supporting the maritime sector and the wider UK economy. The project team has collaborated effectively and engaged the teams across the Maritime Coastguard Agency (MCA) at the appropriate times. The delivery has been tested during the lockdown period at the start of the Covid19 pandemic, with all contact having to be conducted remotely. Both the system and teams stood up to the scrutiny and have delivered an exemplary platform that will be built upon and used by the MCA for many years.”