Showing posts with label testing. Show all posts
Showing posts with label testing. Show all posts

Thursday, February 10, 2011

Three Rules For Difficulty In RPGs


As I put the final touches on Avadon: The Black Fortress, I am finally wrapping up the most touchy, painful, contentious part of the game: play balance. Should it be harder or easier? Is this particular fight fair? How many players will wash out halfway through the game and be forever angry at me for my suckitude?

The fans of role-playing games are a pretty diverse bunch. Some prefer stories. Some fixate on stat-building. Some want to beat everything easily, and others are irritated if there are no challenges. It's a pretty amorphous blob of interests, and nobody can satisfy them all. I just try to appeal to as much of the blob as possible.

Over the years, I have made three observations regarding game difficulty. These are the ideas I always come back to as I decide how tough a given section or encounter will be. I think these guidelines are, to the extent anything can be in a creative endeavor, The Truth.

Observation One: There are two sorts of fights in an RPG: Fights that are supposed to be easy and fights that are supposed to provide a challenge.


In other words, first, there are fights that will almost never ever kill a player, also known as trash, or trash mobs. If your trash mobs are frequently killing the character, your balance is messed up. (Early versions of Avernum 6 had a big problem with this.) Most of the time, the vast majority of the fights in a game will be this sort.

Then there are fights that the player can possibly lose (mini bosses, bosses). And, of course, for fights that can kill the player, there is a spectrum of how likely that end result is. Some bosses will only kill you if you really aren't paying attention. Others require actual skill and strategy, and maybe a few tries to get your tactics down.

Observation Two: In an RPG, you have to have some of both sorts of fight.

RPG fans expect a lot of trash to slaughter, so they can be a badass and Conan-like and so they can collect experience to get strong and get new spells and swords and stuff.

The trickier part is understanding the need for tough fights. Very often, players don't like to be seriously challenged. They hate to lose. They hate to lose repeatedly. Sometimes, the temptation to just give up and have every fight on the default difficulty be easy peasy is overwhelming.

But you still need to have tough fights, for several reasons. A game full of only easy fights against trash is monotonous and dull. The suspension of disbelief in a role-playing game is delicate, and, if a dragon is only as tough as Goblin #0145, it just feels wrong. And because the adrenaline rush of achieving something difficult (be it slaying a demon lord of winning a game of solitaire) is one of the great pleasures of computer games, and you just can't lose that.

So, you might ask, why don't I just put in tough fights and really carefully balance them so that everyone can beat them in just a few tries? That brings us to the third observation, which is both subtle and vitally important.

Observation Three: If a fight has any chance of beating the player, there is a percentage of users who will NEVER be able to beat it.

It took me a long, long time to realize this. Too long. But it is vitally important to understand the difficulty of doing game balance.

Everyone has bad days. Everyone has blind spots. Some people who reach your tough fight will have used up all their healing potions, or refuse to use the healing potions, or forget that they have healing potions, or never have realized that healing potions are potions you can drink that heal you. Because of this, whenever people reach a tough fight, there will be a few of them who just can't beat it. They just can't. You can adjust the percentage of people who lose, but it will never be zero.

(By the way, along these lines, if you put any puzzle or riddle in a game, there is a percentage of users who will never figure it out. This is why I've drastically reduced the number of puzzles in my games. Arguably, I have reduced it too much. It bears thinking about. However, this is my rationale for doing it.)

When someone loses to a fight more than three or four times, they will almost always be angry, and they will always blame you. Some of them will temporarily lower the difficulty, get past the fight, and move on. Some will gut it out and prevail. And some will ragequit and you will never sell a game to them again.

I try to appeal to a wide group of customers, but there is one customer I can never appeal to: The gamer who can't beat a fight and refuses to lower the difficulty. I get e-mails from this person all the time, expressions of hurt and betrayal and rage, accompanied with the promise that they will never buy another one of my games. A promise I believe, by the way. I hate getting these e-mails. Everyone does. It's like a punch in the stomach. But I suck it up, send a nice message back, and move on. It's like death and taxes. It's part of the biz.

But, with every difficult encounter, I eventually have to plant my flag and say, "You must be at least this badass to go on this ride." If you aren't sure why I do this, please consult with the reasons accompanying Observation Two.

Of Course, These Aren't Universal Laws

I expect that the standard swarms of hair-trigger internet nitpickers haven't read this far. They went to the Comments to excoriate me based on some minor point about Observations One or Two, or they want me to explain how Shadow of the Colossus can be a good game with only boss fights. (Answer: Not an RPG.)

But role-playing games are a genre. Genres have conventions. That's what makes them Genres! I can go along with the conventions or fight against them, but, either way, they are part of the DNA of the games I make.

Role-playing games need trash and they need bosses. Put both in, and never lose sight of what makes trash trash and what makes bosses bosses.

Tuesday, January 25, 2011

Avadon Developer Diary #5 - Getting It Done.


This will probably be the last developer diary for Avadon: The Black Fortress. We hope to release it for the Macintosh at the end of February. Then the process of porting it to Windows and the iPad shall commence. Of course, I could get the flu or be hit by a truck, which will alter the release schedule slightly.

The first preview of Avadon recently appeared on Inside Mac Games. There's a lot of good information there.

I've already written diaries about the origin of the game, the tone, the sorts of decisions that need to be made, and the character classes. For me, the longest, most grueling part of writing the game is making the scenario. Dungeons have to be designed. Dialogue needs to be written. Gold coins, clay pots, and spoons need to be placed. In other words, the game itself needs to be made.

That process finally ended last week, meaning that we're in the endgame. Time to wrap up all the odd details and kick the game out the door! This is what I generally call the OHGODOHGODITBURNSGETITOFFMEGETITOFFMEEEEEEEEE phase.

This process, during which everyone involved is pretty much running off fumes, has several parts: Odds and Ends, Endgame Balance, Bug Fixing, and Getting the Game Stable.

Odds and Ends

There are always lots of aggravating parts of the job of writing a game that I put off and put off. Now they can't be put off anymore. Every job I hate. Doing the sound effects. And writing the documentation (and hint book). And making the game icon, and the installer. Nothing can be delayed anymore.

A lot of this work is very important, as they determine a player's first impression. Things like the icon and the starting music and art are the first things a player will experience. These details require a lot of attention, so I work on them at a time when I can focus on them as much as possible.

For Avadon, I've put a lot of work into directing the art at the beginning of the game so that they help the player understand a complicated world, with a lot of intricate politics. The early moments of the game can't be wasted. I want to intrigue the player with the story at every possible moment.

Endgame Balance

Balancing the end of the game is particularly difficult. There are several reasons for that. First, there is a wide spread in power levels among players. The dedicated grinders and min-maxers have made characters that are about as powerful as the game allows. More casual players have characters who have fallen farther and farther behind. An encounter that is challenging to a hardcore player will be impossible for a casual player, and an encounter that is tough for a casual player is probably painfully easy for the serious player.

It's a very difficult target to hit, complicated by the fact that at least some of the epic battles at the end of the game should have some pushback to them. They should feel a little tough.

So I try to make the late game fights hard but not impossible for the more casual players. I also try to put in some optional endgame fights that are a real challenge for the toughest players. These two targets are actually tricky to hit precisely, and it requires a lot of feedback and rebalancing.

In Avadon, I want the player to be able to challenge Redbeard for control of the Black Fortress. I want this fight to be soul-crushing, but, with great skill, preparation, and luck, winnable. This is an extra-difficult target to hit. Making something almost impossible is hard. Still, I think I'm getting close.

Bug Fixing and Getting the Game Stable

Of course, this should happen. Now that the game is mostly done, I am getting to the point where everything has been tested and is basically playable. Now that I am trying to actually release the game, I am trying to get it to the most stable, functional point I possibly can.

It's tricky, because Avadon has a lot of player decisions that can make big changes in the ending. Fortunately, my industrious beta testers are doing a great job of trying out all the different possibilities.

This is actually a much more complicated process than it sounds. You see, whenever I make a change, even the most seemingly tiny, innocuous change, there is a chance that I just broke everything.

Games Are Like Giant Cubes of Jello

One of my favorite books on software development describes an unshipped piece of software as a ten by ten by ten foot cube of jello. When you finish it, it is wobbling and shaking. Then, slowly, the vibrations stop and it becomes stable. However, whenever you poke the jello, it starts to wobble again and it takes a long time to become still.

Avadon is a huge cube of jello that is wobbling like mad. As testers play it and don't find serious problems, it stops wobbling. When I make a change, any change at all, I poke it. When the jello is almost still, I go, "OK, I will release the game ... NOW!" and hope it isn't broken. This is how the process works at its best.

So fixing bugs now is a process of triage. When I get a bug report, I think, "Is this serious enough to risk fixing it, bearing in mind that my fix might completely mess up the game?" As we get closer and closer to the ship date, more and more minor issues get kicked off to the v1.0.1 release.

If you've ever wondered why games ship with bugs, this is part of the answer. There is no excuse for releasing a broken game. However, small flaws are always tolerated in order to avoid disaster. Perfection is for v1.0.4.

So Back To Work

If you've been reading these diaries, thank you! I hope I made Avadon sound interesting, and I hope the game is to your liking. It's been a very long road. I've put a lot of myself into this game, and it really is the sort of game I would want to play. Thus, if it turns out to be a failure, I'll have a lot of thinking to do.

Thanks for reading, and I'll see you on the flip side!

Wednesday, August 12, 2009

Beta Testers: Getting Them. Keeping Them.

I've been running Spiderweb Software for about fifteen years now. I owe my ability to make a living writing my cute, little Indie RPGs to a lot of things. Tenacity. A moderate amount of designing skill. Dumb luck.

But, as much as anything else, I owe my ability to stay in business to a group of skilled, dedicated, hard-working volunteer beta testers.

There is a huge chasm between the rough, quivering lump of code you just wrote and a solid product you can distribute to actual people. To make your game/mod/level/adventure publicly available without melting someone's computer, you will need testers. And, if you are a lone individual or running a little company like mine, you won't be able to afford to pay QA people. Thus, you will need to go out and find testers.

I've been doing this for a long time. Here is some advice for how to find the people you need to test your game (or other sort of application) and how to deal with them when you have them.

1. How To Get Testing Volunteers

I am established and have a fan base, so this part is easy for me. I write that I need testers on my web site, take the beta testing application page live, and winnow through a huge pile of applications. If you're just getting started, you will have a harder time.

Make a web page and put up some info and screenshots, so that people can drop by and see what you are making. Then go to online forums where gamers hang out. Write a short, respectful post saying "Hey, I'm making this and I need testers. Drop by my web site and take a look. Send such and such information here." While forums justifiably hate advertisements, offering an opportunity like this is generally tolerated. (And even sometimes appreciated.)

Ask potential testers to send you information about themselves. What sort of computer do you have? What sort of hardware? What sort of games do you like? What have you tested before?

Hopefully, you will get applicants. However, I'll warn you now. You will get very few decent testers. People who can play a game, analyze it, spot bugs, report them in a coherent manner, and come back for more are very rare. They are precious jewels. You'll have to sort through a lot of, well, people who aren't jewels, to find them.

Also, don't be afraid to use your friends, but don't get your hopes up. Very few people have the mental skills and dedication it takes to be a good tester. The odds of any given friend having those characteristics are low. But give it a shot.

2. How To Screen Your Testers

When you get a pile of testers, you'll need to pick out the ones who are potentially useful. One advantage of asking them to write about themselves is that you will be able to scratch out a bunch of incoherent, confused, and crazy folks right away. If they can't answer a few simple questions, they will not be able to report bugs well. We all know how dangerous the incoherent can be in positions of importance.

Once I've picked out a few applications, I make them provide a signed Non-Disclosure Agreement (I let them fax the form, mail it, or e-mail me a .pdf signature). Google "non-disclosure agreement template" to find a few boilerplate versions.

Now, to be honest, if someone leaks your beta, there isn't much you can do about it. But make people sign an NDA anyway. It's legal protection. It stresses to them how important secrecy is. And making them send an NDA in is great for weeding out flakes. If they aren't together enough to send a signature, they probably won't be able to test well.

OK. Hopefully you have a few testers. Some of them might even be decent. If not, you'll be back to begging on the forums again.

3. How To Instruct Your Testers

You will need to give your testers clear instructions. How should they report bugs? Where should they report bugs to? Any particular format you want bug reports in? The exact details will vary depending on the sort of product you are trying to test. However, you will need to tell the testers something.

This is also a good chance to explain to them important basics. Things like how to take a screenshot or how to compress large files before sending them. And make sure to stress that, in any given report, they need to say the exact version of the beta they found the problem in. (So you don't go chasing down bugs you have already fixed.)

4. How To Interact With Your Testers

First off, always be nice. Always take constructive criticism well. These people are volunteers. They are spending their priceless and irreplaceable time helping you for free. Be polite. Be professional. Never lose your temper. If you find yourself writing a snide e-mail, get up. Take a walk. Count to ten. Then delete it. I have lost my temper with a tester only two or three times, and I always felt terrible about it afterwards.

This is a good chance to work on developing a thick skin. Nothing the testers subject you to will compare to the abuse you'll take online after your project has been released.

Don't hesitate to ask for more information about a report when you need it. Ask for exact steps to reproduce the problem. Screenshots. Saved games. Stress to testers that they need to provide lots of details. You never know what tiny scrap of information will help you to find a bug. Vague reports are useless reports. And, if a tester never answers your follow-up questions, that tester should probably be fired.

Yes, you will sometimes need to fire your volunteers. If they don't use the product. If they don't send helpful reports. If they don't treat you in a polite and professional manner. (The way you are treating them. Right?) Just because they aren't being paid doesn't mean that they should always be kept around. They are taking up your time and they are eating up a precious beta tester slot. Also, being able to beta test is valuable, even if no cash is paid. It is professional experience (more on that later) and a chance to actually take part in the creative process. These are valuable things.

5. How To Reward Your Testers

You must treat your testers with the respect and gratitude they deserve. Thank them. List them in the credits. Give them a free copy of your game.

Some testers will help you in order to get experience, in the hopes of getting a real QA job. If a helpful tester asks you for a reference, give one, and make it good. I have had people ask me for references before, and my success rate for getting people their desired positions is 100%.

And Good Luck.

Don't be ashamed to ask for help. For a certain, rare, marvelous breed of person, the ability to take part in the creative process is a great opportunity. Find those people and let them help you. Be nice. Be professional. Figure out what works for your particular product and go with it. After all, without testers, all software would suck.