What Does A Quality Assurance Tester Do?

Quality Assurance in the context of game development is sometimes seen as a dream job from the outside where gamers get paid to play fun games. Having spent time as a game tester, I thought I’d shed some light on the profession.

Keep in mind that there is no set industry standard, and that specifics will greatly vary depending on the type of game, size of the company and even location. My frame of reference comes from being a QA tester at small to medium sized game developers in Europe.

The Basics of Quality Assurance

A QA is responsible for finding bugs and usability issues during development to improve the quality of a game. It’s usually an entry-level position with appropriately low requirements, which translates into prior work experience or a degree rarely being a must.

In turn, you’ll likely be asked to work flexible hours, meaning working late into the night and lots of unpaid overtime. Coding knowledge, a background in game design and proficiency with the applicable game engine are a plus, as is basic knowledge of the developer’s project management methodology. If there’s are any soft skills I can recommend to you, it’s having patience, keeping your head down and attention to detail.

Since game development is mainly software development, you’ll be using standardized applications for the most part. Here are a few that I’ve repeatedly used on a daily basis:

• Version control such as GitHub, BitBucket or Perforce
• Project Management tool such as Asana or JIRA
• Integrated Dev Environment such as VisualStudio
• Game engine such as Unreal or Unity

The pay won’t be much higher than minimum wage. Head over to glassdoor for a rough idea. Now let’s move on to the day-to-day work, or 10 different ways to spell “testing.”

Feature Testing

You are often going to be testing specific features or fixes that the developers have been working on. When a task is considered done, it is usually submitted for others (meaning you) to review before it is merged into the main branch for a new build – a new version – which is called a pull request. In other cases you may just get a new build with a list of changes which you’re expected to cover.

For the sake of clarity, we’ll use an example PR called “Freighting System” which introduces the ability send resources from one station to another via a new UI panel.

1. Check Requirements

This is usually the first step which has you go over the task description to confirm whether or not the pull request fulfills the requirements. Example requirement:

“Allow players to send resources to other stations they own or to stations belonging to one of their allies with freighters, a new type of unit. Each freighter has a base cargo capacity of 400 resources and a base speed of 4. Resources can be sent by selecting an origin and target station and entering a valid amount of resources, after which the required amount of freighters will depart on a non-combat movement. The freighters sent on the movement will be subtracted from the freighter pool and become available again when they arrive at the target station.”

If a requirement is not sufficiently met, regardless of its implementation, you have to highlight it in your review.

2. Identify Issues

Now it’s time for the actual testing, where you’ll go over the changes that the PR introduces, testing them as-is and in as many different contexts as possible. Here are a few examples:

• Send resources from one station to another
• Try to input a non-numeric character in the input field
• Try to send more resources than you should be able to
• Render the target station invalid during the movement
• Render the origin station invalid during the movement
• Remove an effect modifying the stats of freighters during a movement
• Subtract resources of the origin station while the panel is open

The secret sauce here is to not think like a developer. Developers are logical and predictable, and spend their days building logical and predictable systems expecting them to be used as intended. Actual humans, the creatures that will use those systems, are illogical and unpredictable. Embody chaos, and you will thrive as a QA.

3. Reproduce Issues

With some luck, you will have found a bug. Now you’ll have to retrace your steps and try to make the issue appear again. Once you can reliably reproduce it, you’d write down the instructions. This helps the programmer identify the cause of the issue on their end. Example:

1. Open the freighting panel
2. Send any amount of resources to an allied station
3. Change the allegiance of either the origin station or target station
4. Freighters will now enter an infinite loop entering and exiting the target station

4. Describe Issues

After thorough testing, it’s time to write the report by using accurate language and including screenshots whenever possible to describe what the issue is, and how it can be reproduced. Examples:

• The resource input field in the freighting panel accepts non-numeric characters which generates an error dialog when clicking the send button.
• Freighters are being affected by the Raid Speed empire effect, which should only affect combat movements.
• Resources deposited into the target station will only become visible to its owner after deselecting and selecting that station again.

5. Code Review

If you’re also tasked with technical review, you’ll have to go over the code to point out errors, avenues for optimization and compatibility issues. Most of the time this is left to other programmers though.

	{
 			OnEffectProviderChanged(station);
+
+			var outposts = station.Outposts.OfType<IOwnedLocationEffectProvider>().ToArray();		if (outposts.Length > 0)
+				OnEffectProviderChanged(outposts);
 		}

“We should use OutpostHandler.GetByStationId instead of manager property calls to save us from the refactor in the coming months.”

6. General Feedback

You may also be asked to provide general feedback. Bad user experience (UX) for example is not a bug and more of a usability issue which would be included here.

“It is difficult to recognize your stations in the station list at a glance due to the bright background of the panel. I suggest we add a drop shadow to the station names and increase the font size similarly to the way station names are displayed on the HUD.”

Depending on the severity of the issue and state of the project the PR will now move further down the pipeline or get declined until the outstanding issues have been resolved.

General Testing

Depending on the project, this will be the most or least fun part of your job and involves going over the game in broad strokes to provide feedback on the difficulty, usability, and of course random bugs you encounter. In other words: Test everything.

While rare, relying exclusively on general testing can happen at larger companies where the entire QA strategy is to throw as many bodies as possible at the finished product rather than involving them from the start.

Play Testing

Here you’d play the game under normal circumstances in the eyes of a new player. Is this mechanic insufficiently explained? Is this particular section in the game frustrating? Does this powerup greatly outscale the enemies you encounter during this level?

Afterwards, you might load up an instance with different stats, rules or other modifiers and play the game under those conditions. What are your observations? What breaks? What works surprisingly well?

Stress Testing

This refers to various tests which are supposed to simulate extreme usage or major spikes in concurrent users before it’s time for the real deal. Testing like this is most often automated, but there will be circumstances where a human is expected to see what the results are for the individual player.

Think repeating an action in the game which is known to be laggy in quick succession over and over again.

Smoke Testing

This is what you call doing a general test of a new build after a major change has been introduced. It seems counterproductive to test things that shouldn’t have been affected, but you’d be surprised to see from where bugs sometimes come from.

We had an episode where a profile picture was being loaded endlessly which caused a memory leak when using a dropdown in a panel that didn’t even have images. Go figure.

Compliance Testing

This type of testing is less about the game itself and more more about its compatibility with other platforms. Do all the XBox Live achievements trigger? Has any profanity slipped through the chat filter which would prevent a certification? Are you able to access the Steam overlay at any point in the game?

A game goes through many different channels before getting released, and unless you’re self-publishing on your own storefront, you’ll have to make sure the game complies with various policies and third-party applications.

Recommendations

If you’re serious about getting a job as a QA, here are some of the things I picked up over the years. Your mileage may vary though, so make sure to read up on the experience of others.

• Seek a QA position at a developer, not publisher. This way you’ll have access to the people making the game, not the ones selling it.
• Dodge companies with a sketchy reputation. If a company treats its employees poorly, it will treat its QAs even worse.
• Steer clear of any bullshit “quotas” – you have no control over how many bugs you find.
• If possible, try to take on a project in the early stages of development. Being brought in at the last minute relegates you to finding a specific subset of issues, and you’ll probably be laid off the moment the game ships.
• Consider working at a smaller company. Your pay can’t be much worse anyways, and being considered part of the dev team means your feedback will be taken seriously.

Lastly, if you have ambitions that lie beyond quality assurance, such as becoming a game designer, make them abundantly clear. Being a QA for a while will not automatically get you transferred to the department of your choosing. So get involved with the dev team and spend time off work pitching a sensible design, fixing a bug using your own code or suggesting ways to make your workflow more efficient.

Closing Thoughts

Quality Assurance is hardly a dream job. The pay isn’t good. The hours aren’t good. You will start associating games with work instead of fun. Your work is criminally underappreciated, and you’re one of the most easily replaceable employees at the company.

Sometimes you’ll spend hours testing a feature just to turn out empty handed. When you do find something, you’ll face the wrath of a frustrated programmer whose ego you bruised or a nonchalant “it works on my machine”. Sometimes an obscure bug which was literally impossible for a single person to find slips through the cracks and you’ll receive the full blame.

Coupled with the relatively low requirements, QA is often considered a stepping stone or entry point into the games industry rather than a viable long-term career path. That said, some people go on to become QA Leads, and they’re some of the most hard-working and valuable people you can have in a team.

You’re also going to learn a lot about game development just from being exposed to so many aspects of the game during different stages of their production, and communication is a key skill you’ll likely brush up on – after all, most of your time is spent telling people above your pay grade that they screwed up.

And even if it’s not glamorous, you’re working on a video game. If you can weather this, you can weather anything – especially in this industry. Good luck!