Whether you are a backend developer or a frontend one – you will know the pains of testing, user acceptance testing in particular. On one hand, it means that the bulk of the work is done – a nearly finished product is ready to be presented. On the other hand, that sometimes means that the worst is yet to come. Errors that are found during UAT can be pesky to get rid of if they managed to sneak through previous stages of testing.
It doesn’t matter if the project in question is, for example, a booking app, or a browser-based game, the steps are very much the same. To take a step back, the idea of UAT is to simulate and test as many ‘real’ user scenarios as is possible given the time, staff and resources available for testing. As it is the last stage of testing, it is a very crucial one, since anything that slips past this stage will be included in the final product. This includes bugs as well as problems with usability.
In a perfect universe, there would be plenty of time and resources for this step, and all mistakes are found and rectified before launch. If you’ve done UAT before, you probably just chuckled. The reality is quite different. Usually, a couple deadlines have already been missed, and the lost time needs to be made up for somehow – this time is taken out of UAT. It is easily mistaken as one of the less important steps of testing and development in general.
After all, extensive testing has already taken place, so what is the point of having another testing stage? Investors and clients are often in a hurry and want to wrap things up quickly. Add to that the likely overworked programmers, or, if some were hired, overworked testers, usually also a lack of funds…it can become a recipe for disaster far too easily.
It can be avoided though. How? By eliminating as many potential issues as possible. A fixed budget allocation can take care of most of these issues. It makes sure that enough funds are available to cover all the necessary work – no sloppy mistakes because of hurried work, and no fear of not getting paid for contractors. If the client allows it, it is also a good idea to draw in people unfamiliar with the project – the neighbours, your partner or the janitor. They will give an outside perspective, and their lack of ‘insider-knowledge’ (plus the fact that they don’t need to be paid) will come in handy.
Now, unfortunately, the budget is very often extremely hard to control, and if you are a developer, possibly impossible to influence at all. Instead, there are other elements you can influence. The most efficient way to ensure smooth running is to pre-plan as many elements as possible, while also setting up a detailed testing plan.
‘Elements’ like this could include a staff list – deciding in advance who will work on the User Acceptance Testing can ensure that everybody is prepared, aware of their role, and can effectively utilise their time. While it may be necessary to make some last minute amends or additions to the team in order to cover the workload, having a plan in place will at least give you a steady foundation. And when the storm hits, that is exactly what you will need.
The next step is to set up a detailed list of test cases. Here, the complexity of the project plays a major role – the bigger, the harder it will be to cover every eventuality. The idea here is to come up with as many natural user scenarios as possible. For an e-commerce website, this would involve things like adding items to the cart, checkout, selecting favourites, creating a personal account and so on and so forth. For an app-based game, this could include account creation, in-app purchases, device permissions and of course, gameplay. The earlier in the project this list is made, the easier it will be to include as many possibilities as possible.
What you want to do here is not to come up with fringe cases, but with real-life scenarios. So, while yes, the app may crash if it is run with 11 other apps open in the background, or if it is run in a (badly) emulated environment, but that is not what UAT is testing for. Instead, it is interested in whether or not it runs smoothly if Susan from next door uses it while taking a break from her Candy Crush binge.
A somewhat simplified example, this is still the idea behind this stage of testing. Another aspect that needs to be considered here is the test-environment. For a website or web app, that would be different OS and browsers, for a phone, make and model. You’ll be hard-pressed to find a web project that does not heavily feature on responsiveness and cross-browser availability.
Now, there are thousands, if not tens of thousands of possible combinations of device and browser. It isn’t possible to cover them all, and quite frankly, if in 2017 someone wants to use an old Nokia 2660 (flip)phone to look at a website, that is their choice. It is however not part of the target audience, so fringe cases like this are pretty safe to ignore. In the utopian case that there is time and resources left over after regular testing is completed, this is certainly a good thing to look into, but primarily, the ‘likely’ cases need checking.
That generally includes popular phone models from the past 3-5 years and all the ‘big’ browsers as well as the default ones on the devices that do not already come with Chrome/Safari as a pre-set. There are lots of statistics available on what the most popular combinations are, so try to extrapolate from that, as far as (once again) time and resources allow.
Another thing to keep in mind here is the bigger picture. Always consider why a user will be using the project in question. If it is a game, to pass time. If it is an e-commerce type project, the user will want to (depending on the type), either locate product X and purchase it, or shop around, compare, and spend more time. You should tailor your test cases to the expected behaviour of your typical users.
All of that covers the preparation and planning of the testing – but what about the testing itself? Here, it’s all about the documentation. It can be dangerously tempting to not keep exact tabs on what happens in what case, or to only report encountered issues verbally – especially in smaller sized teams – but do not let yourself be led astray. Good record-keeping can make or break this least popular stage of development.
Now, what is good documentation? Glad you asked. It needs to contain a minimum of information – what happened, what device/OS/browser did it happen on, is it unique or recurring, how serious is it (consider setting up stages like Low, Medium, High, and Show-stopper) and potentially proof/evidence like a screenshot, screengrab or the like. The more info, the easier it will be to resolve, and the less time is lost there.
Also included should be a reference to similar issues. So, say for example, colleague A has an issue displaying a logo on his PC using Chrome and colleague B has an issue with the logo on his Galaxy phone using Chrome – the two are likely related, and should be marked as such. The bigger the team, the harder it is to keep track, so this would be a good task for a project manager. The devs will thank you for it (or if you are the dev, you will pat your own shoulder for it later!).
There is only one more aspect that we have not yet covered: the user ‘acceptance’ part of this testing. In addition to functional issues, UAT is where you find out whether users actually like what you built. What may seem perfectly obvious to a developer, could be extremely complicated to a user. The most flawless piece of code won’t take off if it’s hard to use. So, this is a good time to ask friends, family and the accounting and HR departments to pitch in on the testing. Simple comments like ‘This button should be bigger’ or ‘The checkout process isn’t clear’ can turn your work from good to great. And when it comes down to it, that’s actually the easiest part to fix up. All the hard work is done, the last changes are probably more cosmetic than anything else. At this point you will also find out if users feel that your project takes too long to load or is unpleasant to look at.
You will get some outlying results, but with a big enough group of testers, you will have more than enough to work with – and more than enough input to get a sign-off on your work, and you’re home free!