Beta Testers: Getting Them. Keeping Them.

I've been running Spiderweb Software for about fifteen years now. I owe my ability to make a living writing my cute, little Indie RPGs to a lot of things. Tenacity. A moderate amount of designing skill. Dumb luck.

But, as much as anything else, I owe my ability to stay in business to a group of skilled, dedicated, hard-working volunteer beta testers.

There is a huge chasm between the rough, quivering lump of code you just wrote and a solid product you can distribute to actual people. To make your game/mod/level/adventure publicly available without melting someone's computer, you will need testers. And, if you are a lone individual or running a little company like mine, you won't be able to afford to pay QA people. Thus, you will need to go out and find testers.

I've been doing this for a long time. Here is some advice for how to find the people you need to test your game (or other sort of application) and how to deal with them when you have them.

1. How To Get Testing Volunteers

I am established and have a fan base, so this part is easy for me. I write that I need testers on my web site, take the beta testing application page live, and winnow through a huge pile of applications. If you're just getting started, you will have a harder time.

Make a web page and put up some info and screenshots, so that people can drop by and see what you are making. Then go to online forums where gamers hang out. Write a short, respectful post saying "Hey, I'm making this and I need testers. Drop by my web site and take a look. Send such and such information here." While forums justifiably hate advertisements, offering an opportunity like this is generally tolerated. (And even sometimes appreciated.)

Ask potential testers to send you information about themselves. What sort of computer do you have? What sort of hardware? What sort of games do you like? What have you tested before?

Hopefully, you will get applicants. However, I'll warn you now. You will get very few decent testers. People who can play a game, analyze it, spot bugs, report them in a coherent manner, and come back for more are very rare. They are precious jewels. You'll have to sort through a lot of, well, people who aren't jewels, to find them.

Also, don't be afraid to use your friends, but don't get your hopes up. Very few people have the mental skills and dedication it takes to be a good tester. The odds of any given friend having those characteristics are low. But give it a shot.

2. How To Screen Your Testers

When you get a pile of testers, you'll need to pick out the ones who are potentially useful. One advantage of asking them to write about themselves is that you will be able to scratch out a bunch of incoherent, confused, and crazy folks right away. If they can't answer a few simple questions, they will not be able to report bugs well. We all know how dangerous the incoherent can be in positions of importance.

Once I've picked out a few applications, I make them provide a signed Non-Disclosure Agreement (I let them fax the form, mail it, or e-mail me a .pdf signature). Google "non-disclosure agreement template" to find a few boilerplate versions.

Now, to be honest, if someone leaks your beta, there isn't much you can do about it. But make people sign an NDA anyway. It's legal protection. It stresses to them how important secrecy is. And making them send an NDA in is great for weeding out flakes. If they aren't together enough to send a signature, they probably won't be able to test well.

OK. Hopefully you have a few testers. Some of them might even be decent. If not, you'll be back to begging on the forums again.

3. How To Instruct Your Testers

You will need to give your testers clear instructions. How should they report bugs? Where should they report bugs to? Any particular format you want bug reports in? The exact details will vary depending on the sort of product you are trying to test. However, you will need to tell the testers something.

This is also a good chance to explain to them important basics. Things like how to take a screenshot or how to compress large files before sending them. And make sure to stress that, in any given report, they need to say the exact version of the beta they found the problem in. (So you don't go chasing down bugs you have already fixed.)

4. How To Interact With Your Testers

First off, always be nice. Always take constructive criticism well. These people are volunteers. They are spending their priceless and irreplaceable time helping you for free. Be polite. Be professional. Never lose your temper. If you find yourself writing a snide e-mail, get up. Take a walk. Count to ten. Then delete it. I have lost my temper with a tester only two or three times, and I always felt terrible about it afterwards.

This is a good chance to work on developing a thick skin. Nothing the testers subject you to will compare to the abuse you'll take online after your project has been released.

Don't hesitate to ask for more information about a report when you need it. Ask for exact steps to reproduce the problem. Screenshots. Saved games. Stress to testers that they need to provide lots of details. You never know what tiny scrap of information will help you to find a bug. Vague reports are useless reports. And, if a tester never answers your follow-up questions, that tester should probably be fired.

Yes, you will sometimes need to fire your volunteers. If they don't use the product. If they don't send helpful reports. If they don't treat you in a polite and professional manner. (The way you are treating them. Right?) Just because they aren't being paid doesn't mean that they should always be kept around. They are taking up your time and they are eating up a precious beta tester slot. Also, being able to beta test is valuable, even if no cash is paid. It is professional experience (more on that later) and a chance to actually take part in the creative process. These are valuable things.

5. How To Reward Your Testers

You must treat your testers with the respect and gratitude they deserve. Thank them. List them in the credits. Give them a free copy of your game.

Some testers will help you in order to get experience, in the hopes of getting a real QA job. If a helpful tester asks you for a reference, give one, and make it good. I have had people ask me for references before, and my success rate for getting people their desired positions is 100%.

And Good Luck.

Don't be ashamed to ask for help. For a certain, rare, marvelous breed of person, the ability to take part in the creative process is a great opportunity. Find those people and let them help you. Be nice. Be professional. Figure out what works for your particular product and go with it. After all, without testers, all software would suck.