Game testing, or Quality Assurance (QA), is a role in the industry which seems to bring with it a ton of misconceptions. When I used to work in QA, if I mentioned my job as a “games tester” to someone outside of the games industry, they’d jump to the conclusion that we play our favourite games at home for a few hours in the evening with a beer, and happen to get paid for it…
Ok, so this is a bit of an exaggeration. We know this sounds ridiculous, but in all seriousness people do think that game testing involves leisurely playing through a selection of games. Someone once asked me which of the hundreds of games I’ve played, was my favourite to test? I gave the usual game tester reality speech about working on one project for months on end, repeatedly playing the same tiny section over and over, and reporting bugs.
I’ve noticed that oftentimes, the thought that there has to be a methodical process to finding and fixing bugs within games goes over the heads of most people. The fact that we need to use special problem tracking software can sometimes also come as a surprise. There is a set process to game testing, and writing bug reports. Although there are slight differences from company to company, the general pattern stays the same.
The General Process of Game Testing
Functionality testers will often play a small section of a game repeatedly, basically trying to break that section. Testers have to approach their assigned area of the game from every conceivable angle, thinking outside the box, to try and expose any bugs, no matter how obscure. Once a bug is found, the tester will write and submit a bug report using the particular bug tracking software used by the company (It’s basically a database that stores the bug reports). At Guerilla Tea we use JIRA, but there are many others including Mantis, Bugzilla, and larger companies may have their own proprietary program.
The tester assigns a bug report to a member of the development team, or maybe to their lead tester who will then assign it to the relevant developer. The developer will fix the bug, mark it as resolved, and assign it back to the tester.
Once the next build of the game becomes available, hopefully with the bug now fixed, the tester will try to reproduce the bug in the new build and if it doesn’t occur, he/she will mark it as closed. However if the bug still occurs, the tester will assign the bug back to the relevant member of the development team, and the process repeats for the next build of the game.
This is a brief overview of the process, but what about the individual bug reports? There is a standard format, again with minor differences between companies.
Standard Bug Reports
Bug reports tend to be very similar throughout the games industry and include the following sections:
– Summary Sentence
– Steps to Reproduce
– Rate of Occurrence
– Expected Result
A single, short sentence describing the bug, which will often begin with the type of bug such as art, gameplay or crash. For example.
“Crash – Game will crash at start of Tank boss battle in chapter 2.”
A longer description of the bug, going into far more detail.
Steps to Reproduce
The sequence of actions you did in-game to make the bug happen. It’s deemed one of the most important aspects of a bug report and each step must be very clear, but not exhaustive. You generally shouldn’t have more than about 10 steps. Common sense really; you wouldn’t summarise playing through an entire game if the bug happened on the final boss!
How serious the problem is. We tend to classify bugs as A, B, C or D.
A = Game breaking; crashes, etc. Something that stops player progression.
B = Major problem, but game can still be completed. Serious gameplay or art issues usually.
C = Minor problem. Less noticeable gameplay and art issues.
D = Design bugs. Possible improvements, very minor design suggestions, etc.
The priority is a very important aspect of a bug in terms of the development process and works in tandem with the severity, in a way. Priorities may be marked as “must fix”, “should fix”, “could fix” or “won’t fix”, but it does vary. A bug may be ‘A’ class in severity, but if it only occurs less than 1% of the time under a very obscure sequence of actions, then it would likely be marked as “won’t fix”.
Rate of Occurrence
The rough frequency with which the bug occurs, usually marked as a percentage. The tester must try to reproduce the bug several times, and then judge the frequency from there.
A brief sentence describing what the tester expects to happen if the bug is not present. Could be as simple as “Game should not crash”
I understand that developers will know all the above inside out, but I speak to so many people that have completely the wrong idea about the QA role. It’s an important first rung on the career ladder for many, especially those looking to get into game design and unfortunately a job that doesn’t get the respect it deserves.