Whether you’re developing an app in-house or outsourcing to an external team, there’s an important question you should ask:
“How will you test my app?”
You can have the world’s best developers at your disposal, but building a good app takes more than their ability to write impeccable code – it’s about delivering a user experience.
Quality issues are a big deal when it comes to apps. They can range from small glitches that annoy users because of how frequently they occur, to less frequent, full-blown-disaster type bugs which have a bigger impact on a few users. These kinds of things happen to apps of all types and no developer is immune.
This is why that testing question is important – at its core, it’s about managing your risk as an app owner. Imagine the disaster, for example, if you have a budgeting app that links to users’ bank accounts, then find there is a security breach. This would be at the extreme end, but you’ve still got to consider that users leave apps because of small annoyances.
From this perspective, testing is about protecting your investment, so let’s take a look at some app quality tests you should expect from a developer:
App testing overview
First of all, it’s important to note that software testing isn’t a singular approach, but a collection of tests and evaluations that may be conducted at different stages. In fact, you should expect testing at all stages of development, including after launch. Things change, platforms get updated and bugs which weren’t visible before can suddenly appear.
At Koombea, we see testing as a cyclical activity through different stages of the development process, as outlined in the diagram below:
Our goal is to be proactive about finding and eliminating potential problems, with particular focus on anything that will be impactful on users in terms of annoyance or the severity of the issue. Any software tester will tell you that it’s impossible to test every possible input or find absolutely all bugs (you’d never actually release the app), however with some focus, you can reduce the risk of any sort of large impact on the user.
This usually means looking at the app as a whole and prioritizing any areas which you deem to hold greater risk. Tests are then run to prove or disprove functionality and any defects found get logged. Those defects can then be prioritized based on their impact and severity.
App quality tests
There are a variety of approaches to app quality testing, each of which are important in their own way. Testing commonly is used to answer questions such as:
- Do all app features function as expected?
- Does the app work as a whole?
- Are the correct outputs given for inputs?
- Are there any vulnerabilities which could put users at risk?
- Is the app easy for people to use?
- Does the app still work well with heavy loading?
- Can users easily pick up where they left off?
The questions above are just a few examples – there are more that could be tested and many which will be specific to the type of app being developed. Here are a few tests which you should expect from your developer:
#1. Requirements testing
Requirements testing is as it sounds – the tester is checking for conditions that are derived from the requirements of the app. These might come from the specifications you have provided or any claims you want to use when you market the app. For example, you might have given a clear design specification or wireframe.
Requirements testing can actually encompass other types of testing (such as for performance), including anything that is implicitly required by users. Tests can be functional or non-functional (again, performance comes under the latter description).
Besides explicit and implicit requirements, there is a third type to test for – any latent requirements. These are the things that people won’t necessarily expect, but will delight them when they use the app.
#2. Smoke test
This is also known as “build verification testing” and aims to ensure that the most important functions of the app work. If you’ve got an accounting software, then your app must (of course) account!
Smoke testing isn’t extensive – it aims to cover all major functions of the app just to ensure that they work at first impression. The term is said to come from hardware testing, where a device passed that initial test if it didn’t catch fire when it was turned on.
The same principle applies to smoke testing an app – if nothing was obviously wrong, it has passed a stability test of the build which determines further testing can proceed (or on the other hand, more work needs to be done on the core functions).
#3. Regression testing
In a nutshell, regression testing is done to ensure that the system still works the way it did before. Where software has been changed or interfaced with a new function, does that older function still work as it is expected to?
This type of testing is especially important in agile development environments. We work in weekly “sprints” where we complete app builds in segments, we need to make sure that this week’s addition hasn’t compromised last week’s build. The same could be said for any time a patch or fix has been made – do all functions still operate as before?
#4. User acceptance testing
User acceptance testing (UAT) is usually the last phase of testing prior to market launch. The keyword is “user” because testing is done on actual users to check that the app is what they expected and that it is usable for their purposes.
A usual method is to simply get people who are from the intended target group to use the app and test out its various functions. Any defects are recorded and corrected, including any issues with usability or clarity of use.
A prerequisite for this type of testing is that the app should have already passed tests such as regression and a UAT environment should have been created. The app should already be considered to be “working”, so criteria are required to define what that means. As the owner of the app, you should have been able to test it yourself prior to this stage.
#5. Performance testing
What happens if your app goes viral and you’ve suddenly got 10,000 new users? What if all of them use your app at the same time? Performance testing covers a range of different tests which are designed to check how well your app performs with a heavy load, with a normal load over an extended period of time and with gradually increasing loads.
Tests might include: load testing, stress testing, spike testing, endurance testing, scalability testing, and volume testing. These tests might pick up problems such as bottlenecking of data, memory leaks, poor network performance and any issues with hardware.
Without these tests, you could find yourself in a position like Whatsapp, where millions of users were left unable to use the app when it went down across the world (although they didn’t confirm why this happened). As happened in this case, you’ll find that people tend to take to Twitter to air their dissatisfaction, something most companies hope to avoid!
Performance testing is one type of test which should be carried out post-market as well.
Final thoughts
Testing is a critical part of app development so you should always expect that it is a priority for your developers. Whether or not testing was done well can have implications for the reputation of your app and its popularity with users.
Look for developers to be conducting a range of different tests and doing so continuously. Testing isn’t something to leave until the final stages of a project, but to implement throughout in an effort to avoid any extensive rework or delays.
In the end, it’s about managing risk as far as possible on behalf of your users.
Koombea builds beautiful apps following robust testing processes. Talk to us about how we can help you today.
