Regression testing is defined as a type of website and application testing used to confirm that any recent new programming or code changes have not, or will not, negatively affect and impact existing website or app features or functionality. Whether deployed fully or partially on a website or app, testing highlights critical issues with the aim to ensure existing code will still work once the new code is deployed.
There is no getting around it; a proper regression testing process is a must-have for any business that relies on its digital assets for revenue and has produced sizeable results. Figleaves, for example, experienced a 12% increase in mobile conversion rates after they implemented a personalised regression testing program. However, a lot of companies have serious flaws within their regression processes.
We have highlighted some of the most common issues below:
- Cost vs quality
- Reliance on an individual’s domain knowledge
- Time taken to execute
- Out of date test packs
Cost vs quality
The repetitive nature of executing test cases that are not clearly defined and rely heavily on domain knowledge will lead to complacency, as well as fatigue, and have an impact on moral; ultimately leading to human error.
With in-house testers or contractors, you are also paying for their time. If the quality of their work is not up to standard, you must still pay them and accept the unknown opportunity cost of conversions being impacted by low quality testing. Further to that, if you then experience delays in development stages, you will still incur the cost of testers, with no testing activity.
At Digivante, because our clients pay us to execute test cases, not for our time, if you have a delay to development then you will not incur costs unnecessarily. With every step of our test cases being evidenced, quality is never an issue. However, if a client is not happy with the execution, we will re-run it at no extra cost meaning you are only ever paying for a full service and you are always experiencing the best possible quality.
Relying on an individual’s domain knowledge
Most companies that carry out regression testing internally will either use their development teams or internal QA’s. Typically, these test cases rely heavily on domain knowledge resulting in testing at too high a level.
A key issue with this is that business-focused and non-technical colleagues within the business will not be able to easily identify what has been tested. Further to that, if a tester leaves the business, is sick or is on holiday; you would be left in a situation where no one can execute the test cases, and no one knows what has been tested.
This level of risk significantly impacts a business’s ability for effective continuity planning and often notably increases the test window.
Due to the nature of our community, Digivante will always write granular test cases with every step of the journey clearly defined. Our community testers do not have domain knowledge and as such must follow step by step instructions.
How does this benefit you?
- Anyone who can access the environment can read and execute the test cases
- You can leverage internal resource, such as development teams and non-technical staff, to support if necessary
- This will significantly reduce the ramp up time for new starters in your test team
- Business focused and non-technical colleagues can easily identify what has been tested
- Everything is evidenced; literally every step of every journey
- Reduced risk and effective continuity planning and scalability within the team is possible
Time taken to execute
Having built tens of thousands of test cases over the last 12 months, Digivante have identified that on average, a typical front-end ecommerce website will require around 100 test cases for regression.
Because of the need for websites to work properly across a wide range of devices and browsers, the number of individual test cases to be executed per sprint is significant.
Due to the nature of our test cases, we can leverage our substantial community of professional testers to ensure that regression testing can be done in hours rather than days, as well as significantly mitigating the risk of quality being jeopardised.
Out of date test packs
With most companies releasing at least once per month; test pack maintenance is a must. Every time you add anything new or change anything existing, you must create new test cases.
Typically, this needs to be done whilst at the same time starting to execute on the next sprint. The time pressures on QA’s are already significant and expecting them to build new granular and evidenced test cases will further jeopardise quality.
This can result in:
- Having to hire new full-time staff or bring in expensive contractors
- Relying on your development team, at the risk of reducing resource on development and fixing
- Accepting low quality output
Digivante rely on our dedicated in-house team of test analysts and managers who have years of experience in writing granular test cases, meaning we can write new test cases whilst the community execute existing ones.
For more information on our approach to regression testing book a call with a Solutions Consultant.
