Scale your quality assurance process without growing your QA team

It is easy to fall into the habit of hiring full time QA resources to resolve known or assumed QA limitations. It may be that this is a tried and tested approach that has been successful for you in the past, or you just might not know what options are available beyond this. Changing your approach and venturing into the unknown is not easy, but this blog aims to provide some guidance. It will help build your confidence to try new approaches and scale your quality assurance process in new ways.

Demanding digital users

The world is fully digital and the demands of users on software solutions is ever growing. In the past, users were happy just to have the option of a digital solution, even if it was not the most functional, usable or accessible. But now, users are digitally savvy and simply will not accept poor quality.

In fact, according to Verint’s State of Digital Customer Experience research, 64% of consumers have jumped to a competitor following a poor customer experience on a digital channel. End users of internal systems or tools require software solutions to work consistently and efficiently, or it directly impacts their productivity. The more work they are required to do, the more dependent they become on the software performing. If the software lets them down, inevitably they fail to complete a task which either impacts them directly or has a knock effect on other departments or customers.

Improving your QA team productivity and efficiency to meet the challenge – why bother?

Getting more out of what you already have is fundamental to creating a QA team that works best for your business. QA teams are renowned for being squeezed for time at every step of delivery. Adding more resources does not always resolve the problem and could introduce other issues such as additional management, training and cost.

To keep the end user happy, the quality of the software they use is paramount. This means the QA team not only has the responsibility to test that the software works as required, but also does it work for all end users consistently, does it work in different environments, does it perform under stress, is it secure etc?

A good QA team is involved in quality improvements across the delivery lifecycle and across teams. Let’s be clear. QA is not just testing. Testing is very important, but often suffers if it is not underpinned by an effective QA Strategy.

As the workload increases, the QA team becomes more and more stretched, often lowering quality gates due to time and resource limitations. Therefore you need to make the existing team as productive and efficient as possible. New hires are often a consequence of deficiencies in the existing QA team’s processes, so that needs to be looked at first. There is little value in hiring additional resource if you have not thoroughly investigated the current team’s processes. Otherwise, you introduce another person to these inefficient processes which is neither cost nor resource effective.

Challenges of scaling up quickly & stretching too fast

An explosion in digital transformation is leading companies to go digital for the first time, entering new global markets, introducing ecommerce, or creating software solution capabilities.

Many of these changes come with tight deadlines which the company needs to achieve to gain competitive advantage e.g. first to market with a new product, upgrading software to maintain excellent customer support, improved regularity compliance, better alignment to a marketing campaign etc.

These things put pressure on the QA function to continue to perform at the same level as prior to these changes being applied. Given the time pressure, existing staff and recruitment become stress points. Recruiting quickly often leads to the following problems.

Reduced quality of new hires

This decision can be very detrimental to the QA team in the both the short and long term. In the short term the onboarding will be slower, and in the long term the hires may prove not to be up to the job and leave, meaning wasted recruitment costs, training and instability within the team.

New hires deliver poor quality

If the new hires do not have the experience to deliver quality, then a lot of re-work is required which puts further pressure on QA time and reputation within the business.

Existing staff are less productive as they spend a lot of time training new hires

Productivity across the team is reduced as existing resources need to train the new hires. So although there is an increase in resource numbers, the productivity of the team will be reduced in the short term.

Loss of trust in QA

If new hires deliver poor quality then:

• The development team will waste time on poorly created defects that are not reproducible.
• The development team will feel testing is a waste of time.
• Product owners will not trust that new features work as expected.
• The end user experience will be poor.

Planning for software quality assurance at speed and scale

A delivery schedule is often made up of peaks and troughs, although the troughs are becoming less frequent due to increases in business requirements. QA teams need to be able to quickly ramp up coverage to deal with peaks, while also maintaining consistent business-as-usual tasks like regression.

The QA team needs to support both pre- and post-release testing of new features, supporting both test and live environments.

QA teams need skilled technical resources for accessibility, security, performance, integration and collaborative exploratory testing that can augment efforts at critical times in the lifecycle of a digital product.

There is also the need for usability testing to get feedback from real users to ensure the product works as per their expectations. The end user experience is critical; if the product does not do what users expect, underperforms or is missing key features, they simply will not use it. As user experience is becoming central to optimising revenues and conversion, usability tests can deliver extraordinary insights showing how specific changes in customer journeys and design can improve performance and revenue.

These are all QA services that require careful planning and clinical execution.

Is outsourcing the answer?

Using outsourced and crowd-sourced testing agencies has become a vital part of the QA landscape in recent years. But not all agencies are created equal.

The ability to draw down professional manual testing resources on an ongoing basis, to have access to thousands of testers for regression, functionality and usability tests 24/7 can clearly help you scale your QA process and capabilities without increasing the size of your team.

But this scale of testing resource is only useful when it can be translated into prioritised and actionable results. Agile organisations, where the commercial and development function need to collectively make stop/go decisions around development projects, need to see and understand the impact of the issues that are being detected. Aggregated results of mass testing can help with this, as can screenshots and videos of the real-world defects that are being found.

There is a need for large scale, manual repetitive testing tasks, as well as highly focused, ongoing monitoring and usability testing that can feed back into an organisation quickly and efficiently. Some agencies can offer the full range of these services in a bespoke and cost-effective way, and some can’t. Others may offer all QA requirements but not do any of them particularly well. Therefore, it’s important to fully research potential outsourcing options. Options such as unlimited usage, cheap offshore resource and self-service may sound appealing but are often too good to be true.

• Unlimited usage comes with small print that set the limits.
• Cheap offshore resources often do not understand your market or product and become a management overhead as well as an onboarding/offboarding nightmare.
• Self-service is often inflexible in scope and additional support comes at a heavy cost

Pragmatic automation

Test automation can be a great solution to reducing time to market when you release new features and functionality. Indeed, automation is an important tool in the QA workbench, but it cannot do everything. Therefore, taking a pragmatic approach to test automation is very important. That means being objective about where to automate your testing based on where it will save you time and money.

Approaching automation with the expectation that everything can be automated is a wrong and far too simplistic approach, despite what many people may say. Having an automation-first approach works better, as you will consider the ability to automate and make measured decisions on whether you should rather than whether you can. Automating for the sake of automating, with no measurements of cost or benefit is a fast way to fail in automation.

If you automate the wrong tests, it will cost you more – not just in repeat testing and redevelopment later, but potentially in lost revenue and customer approval due to glitchy or non-functioning websites and apps.

A wrong automation test candidate would be:

• A test which is not repeatable
• A test with inconsistent results
• A test that is more effective to run manually

A lot of people think that once something is automated, it is then free testing for ever. But automation scripts need to be updated regularly to ensure they are still valid. If not, the automation team spends half their time figuring out if failures are related to genuine functional issues or if they are due to the script no longer working as expected. This investigation delays the business from making decisions relating to issue priority and understanding the quality of the software.

Ideally you focus automation on regression and set realistic targets for gaining coverage. The coverage should not be set in stone, as every change should be assessed to ensure the test script still meets the right candidate criteria to remain automated, or if it does not, then run it manually.