Test automation can be the solution to releasing new features and functionality with reduced time to market. Automation is one of the tools for QA resources to use but at Digivante we firmly believe full automation isn’t the ultimate end goal. You might be thinking, “but automating any test is going to save me time in the long run.” This isn’t necessarily the case. We take a pragmatic approach to test automation and believe you should be clinical with where you automate your testing, basing your decisions on where it will save you time and money. When inappropriate tests are automated, it costs you.
When does it make sense to automate?
Regression testing is often a prime candidate for automation. Let’s face it, the repeatable nature of regression testing hardly sets a tester’s world alight. Internal or offshore testers repeating the same regression tests will inevitably cut corners when under pressure, and the quality and consistency of results degrade over time. Automation is a route to eliminate that risk.
But automating a regression pack is expensive and time consuming. Let’s say you have a regression pack with one hundred test cases. The path to automating all of those, and ensuring that the automation works as expected, is long and sometimes arduous.
Another area that’s prime for automation is integration testing. You’ve got your main site or app created but you might be integrating with payment systems like PayPal, or stock management systems or content management systems or potentially dozens of other integrations that give your users the experience you want for them. Using test automation, developers can take a “shift left” approach to integration testing using a REST (Representational State Transfer) tool and API (Application Programming Interface) to test integrations early in the SDLC (without target integrations systems being in place). The API requests can be automated to run as part of the deployment of code or in CICD which is quicker than testing via the UI.
Our approach to automation
We take a practical approach to automation. Let’s go back to the regression pack with one hundred test cases.
We start with running every one of the 100 tests manually first to ensure the test case works and we record all of the actions to build the skeleton of the code. By running the test cases manually first we find out which test cases pass and which fail.
Let’s say 90 passed but 10 failed; those 10 are now off the list to automate.
For the 90 that passed, we’ll assess which would make sense to automate. If you run your regression pack regularly, say weekly, it becomes cost effective to automate as much of it as possible.
So next is the process of scripting the tests for automation. We focus on automating stable entities that have repeatable actions and don’t change regularly, otherwise maintenance costs and automation coverage will be impacted.
Keep testing whilst building out your automation
Once you’ve decided what to automate, you need to ensure you have test coverage in the meantime, whilst your automation is being coded. Working with a company like Digivante, you gain access to the crowd testing community that gives you broad testing coverage whilst building your test automation programme .
Even if you have an existing automation pack, or are already in the process of creating one, you might have an area that fails its automated tests. Our recommendation is to then pass the same tests to crowd testers. If the crowd passes the test, then you know there’s an error in your script. But if the crowd fails the test too – you know your automation fail is a real fail too. Instead of just getting an error message, you’ll be able to see – in lots of detail – where the issue is and where your developers need to focus to fix it, allowing for prioritisation of issues.
The Digivante automation process
If starting automation from scratch
When automation is in place
This pragmatic process of using crowd testing whilst building your automation is quicker than trying to unpick your automation script to find the error.
Comparing different test methodologies
| Factor | Manual testing | Automation testing | Crowd testing |
|---|---|---|---|
| Costs | Cheaper in the short run | Cost-efficient in the long run | Cheaper in the short run, cost-efficient in the long run |
| Accuracy | Less accurate and more error-prone due to human factor | Scripts allow you to more accurately detect and track issues | Accurate and detailed issue raising makes it clear to developers where code needs fixing |
| Speed | More time-consuming | Fast and ideal for repeated testing or large-volume testing | Fast and ideal for repeated testing or large-volume testing |
| Area of use | Works for non-trivial issues where human expertise is required | Does not guarantee high-quality user experience | |
| Resources | Human | Automation | Human |
| Flexibility | Very flexible and can be used in a wide variety of cases | Might not be effective when there are changes in the interface (requires changes in the script) | Very flexible and can be used in a wide variety of cases |
Freedom of choice
Working with Digivante gives you freedom of choice: develop and deploy test automation and/or manual testing at scale to maximise the benefits of each.
Whatever your starting point, whether you are adopting test automation for the first time or need broader coverage to meet your QA standards, we focus on your priorities and on employing the right types of automation and testing to suit your needs. We help you apply automated testing as standard, with the option to deploy manual testing by testers who closely match your target demographics and devices without increasing your test team headcount.
Using the Digivante portal, you have a single interface to action both test automation and community testing giving you faster access to data-driven insights for informed decision-making.
- We devise a plan with your team to iteratively grow the test automation coverage as new scenarios are identified. Your automated test suite can be executed manually, via schedule or via a suite of Digivante APIs.
- You can then check test status through the Digivante portal or through the APIs. Results can be similarly obtained, with technical information being provided for truly automated tests, and in-depth results with images and videos for our crowd-sourced testing services.
To find out more about test automation with Digivante, request a free consultation.
