This is a critical area of the game design process. It is very important to listen to what your playtesters mention. There’s usually a reason they say the things they do. But understanding what they actually mean can be tricky. Let’s start with some basics.
Matt Loomis, fellow game designer and blogger who I had the privilege of meeting at Protospiel-Milwaukee back in March, recently wrote an article all about this topic. You can find it on his blog – TheMetaGame.blogspot.com.
If you are a game designer I recommend reading his article and following his advice. One piece of advice from the article that I have also mentioned is this:
If players start the game taking the same actions every game, start the game from there.
The point of that is that if everyone makes the same decision on the first or second turns, make that decision part of the game setup. Players should start the game making unique decisions. I totally agree with Matt!
Here are a few pieces of common feedback that should be understood and interpreted:
- The game takes too long. (They are bored)
- The game isn’t fun. (They are bored)
- The game felt like work. (The game isn’t fun – see above)
- The game was really slow. (Too much downtime, or not enough to do on their turn)
- I couldn’t do much on my turn. (Not enough unique options or too limited per turn)
- My decisions felt scripted. (Not enough unique options)
- I don’t like the art. (Get over it… it’s a prototype!)
- You should use such-and-such component. (Thanks. Hey, did you realize this is a prototype?)
Obviously some of these are just not helpful. But players are saying them for some reason or another. So let’s try to examine why playtesters say the things they say…
Advice from Chevee Dodd:
I pinged Chevee about advice for interpreting and understanding feedback. For those who don’t know him, please visit his website (cheveedodd.com). He is the designer of Scallywags and a few other upcoming games that are awesome. Here’s what he had to say:
Understanding feedback is a vital part of game design that many people don’t understand. When someone tells you that you should dramatically change your game because they don’t like an aspect of it, that is not bad feedback… they may have just not worded their concern properly. Take this feedback and look at it objectively. If they want you to toss out a portion of your game, or add in a whole new layer, ask yourself “Why?” It is often because they felt that their options were limited or their actions weren’t meaningful enough. So, maybe their suggestion is to add atomic cannons that shoot rainbow fireballs, blinding your opponents and making them discard their hands. It’s very likely that this is completely wrong for your game… but at the core, maybe there IS a need for you to be able to mess with your opponents hands and not having that is making the game frustrating. So, you add a little ability here that lets you discard a card at random from their hand and suddenly you’ve added a bit of control (through chaos) to the hand-management portion of your game.
That is some excellent advice! I think the atomic cannons shooting rainbow fireballs sound particularly awesome!
I think Chevee’s point is that sometimes playtesters will mention something while meaning something else. When players mention things like adding awesome cannons that let you do great things, they are probably suggesting it because they don’t feel like they are able to do great things. And that is an important point of applying feedback. Before we get into interpreting feedback I want to make a more global point about all of this…
Don’t Take It Personally:
You’ve invited people over to try out your latest and greatest game design. This is the one you think will make it big and revolutionize the board game industry. You can’t wait to get it on the table because you just know people are going to laud and magnify you for it.
And then they play it. And then they hate it. Uh Oh.
They slander it. They posterize it. They straight up say it was the worst thing ever. You start to feel angry and think to yourself, “These idiots… they have no idea how awesome this is.” You get upset and ask them to leave. You feel dejected. You feel like you’ve been rocked by a rainbow fireball. You start to wonder where things went wrong and why they don’t like you. Boom… that’s the problem – It’s not about YOU!
Here is my favorite advice for applying feedback:
They are not rejecting or criticizing YOU. They are rejecting or criticizing your GAME!
The same goes for praise. Just because someone might love your game doesn’t necessarily mean they like you. The bottom line here is that we are applying feedback to the game and not to the designer. The game is what matters at this point. Don’t let your emotions cloud your opinion of your playtester’s feedback. And don’t take it personally if they think the design seems broken. Often if players feel a design is broken, it’s because it is broken. Get over it and move on!
Okay. So you did not take it personally and you are ready to interpret what your playtesters were really meaning when they said they hate your game.
Often playtesters do not think like a game designer. That’s why I recommend attending things like UnPub or Protospiel where you can get feedback from other game designers. You can often find other designers at your friendly local game store. But how can you interpret feedback from non-designers who may not know the best way to state what they actually mean? Let’s examine a few scenarios of a hypothetical game about boll weevils.
Your game design has players developing a colony of boll weevils. Players will build a structure for the weevils, feed the weevils, and try to claim the best territory for the boll weevils.
During your playtest someone mentions that colonization seems too valuable. Another player mentions that they think you should have different abilities for the boll weevils. A third player mentions that they got to a point where no matter what they did it didn’t seem to help. Let’s examine these three pieces of feedback and interpret what they may actually mean.
#1: Colonization seems too valuable. (Balance is the issue)
In this case I could imagine the player falling behind another player who had chosen the colonize action. The best way to interpret this feedback is to understand that the player felt like the actions they chose were not as good as those of the colonizing player. In this case, as the designer, you may need to work on balancing the short-term/long-term benefits of taking one action over the other. Balance in a game is often a very difficult thing to get correct. The result is that you will often receive feedback that is directly related to balance without the playtesters mentioning the word balance.
#2: Add Different Abilities. (Lack of interesting choices is the issue)
The playtester that wants different abilities for their boll weevils is likely feeling that they cannot do enough on their turn, or that they cannot do anything interesting on their turn. They recommend adding abilities as a way to specialize their game and give them something more interesting to do. The key to understand here is that the player doesn’t feel like they can do enough interesting things. They want abilities so that they can turn dull decisions into decisions that require more strategy. When playtesters suggest adding things to a game it is often because they feel the game is lacking interesting choices.
#3: Nothing They Do Can Help. (Results of decisions are not apparent)
When players begin to complain that there is nothing they can do that will help them there could be several reasons. The first reason is that the game is about to end and they are so far behind that it does not matter what they do because they just cannot win. The second is that the game limits how much a player can do on a turn that each decision is so insignificant that it won’t matter. And a third reason is that players may not be able to understand how their decisions are effecting the game. Keep an ear open for any time players feel like their decisions don’t matter. This is a big deal. Games are supposed to be fun and if players decisions do not matter, how much fun could they be having? Players may say things like this when they are not having fun, or when they have lost interest. This will require taking a deep look at your game design and perhaps reworking a few things. As a designer it is critical to understand that you want every single decision that a player makes to have a significant effect on their opportunity to win the game.
Those are just a few scenarios out of an infinite playbook of playtester feedback. What I hope you have understood from this is that often playtesters will say one thing and mean something else. They may be very specific about what they think should change in a game while actually meaning something very vague.
So when playtesters offer feedback I recommend asking them why they suggested the change. Asking them why can open the conversation up to the real feedback that they are really trying to tell you. Ask them what is at the heart of their comment. This can be really useful.
We’ve discussed interpreting feedback. Now we dive into applying that feedback. This can be difficult and tedious, but now that you understand what your playtesters were actually saying, you are ready to improve your game.
As shown in the graphic at the top of this article I prefer having ten playtests between changes in my games. Ten seems to be a good enough number where you can likely hear the same feedback from several different players. If, over the course of those ten playtests, you do hear the same feedback, then you likely know this is something that should be changed or adjusted. If you make sweeping changes between each playtest you’ll have no idea what part may or may not have improved the game.
The other thing I like to do with feedback is apply one change at a time. With Scoville, for example, several different people had mentioned that the brown cubes felt a little weird in the game. But it took several playtests to hear it from those several people. Later on other players said they loved that brown cubes introduce a way to mess with your opponents. Here’s the key:
Don’t let one person’s opinion of your game dominate how you change the design.
When multiple people mention the same thing then you know that something’s up. Then you can feel free to address that feedback.
So my two guidelines are these:
- Only apply one major change to the design at a time.
- Try to playtest each change at least 10 times before adding the next change.
Using these two guidelines should help you to understand which parts of your game design are awesome and which need some work.
So now you have a way to interpret and apply feedback. Get to the root of what your playtesters are saying. Apply the appropriate changes one at a time. You will be well on your way to creating a well polished gem of a game! Thanks for reading. Feel free to leave your feedback of this article below. I will try to interpret it correctly!
I had the privilege of attending my first Protospiel this past weekend in Milwaukee. Protospiel is a convention for game designers to bring prototypes and get feedback from other designers. So I took my game Scoville along and got some awesome feedback! I think that I’ll focus this recap on my game rather than provide opinions of the games I played that are unpublished. That would not be fair to the designers even if I really enjoyed their games since all the games I played are still in progress. So rather than posting a drawn out chronological recap of the weekend I will just post the drawn out highlights for the play tests of Scoville.
I was fortunate to have Scoville played five times and was pleased to play 8 other games by other designers. Protospiel is an awesome thing for a designer to attend!
Here’s a little background about my Protospiel expectations and goals…
Protospiel: First Contact
Coming to Protospiel I had two goals: 1) validate whether or not Scoville is any good and 2) connect with people who know what they’re talking about. A secondary goal was to leave a copy of the game with Grant Rodiek for inclusion in the Prototype Penpal Program. That was something I could always do later on, but I thought it could be cool to send a copy off with him.
I also had some expectations about the feedback I might receive. I knew that I wanted to adjust the auction phase of the game. So I to see the same feedback about it that I had seen from my prior play tests. I was also a little uncertain about the quality of my prototype (that thought was quickly vanquished!). Thanks to everyone for the kind words about the quality of my prototype. I’ll post an article sometime about how I make prototypes.
So if I received validation and made some connections then I would have considered this weekend a success. Let’s see how it went.
Scoville Play Test #1
Getting to the convention at 8:15am on Saturday allowed me to get my game set up right away since few people were there. I got four people to give it a go and they seemed to really enjoy it. I won’t explain the game much here since I’ll be writing a post all about the game itself. Here are the suggestions that I received after the game:
- Beware of color blindness (Cool apps: Color Blind Vision (Android: FREE) and Colorblind Vision (iOS: $2.99)).
- Stage II orders seem to provide too many points.
- If everyone bids zero in the auction, flop the player order.
- Put endgame trigger scenario onto the guidesheet.
- Tiebreaker should go to the player with the most coins.
- The game was described as a “Euro with luck but no dice.”
- There should be no randomly chosen player order at the start of the game.
- During fulfillment there should be the option to pay for becoming the first player.
That’s a lot of great feedback. The game uses 10 differently colored cubes so I have been aware of the color blindness issue. There are several solutions for this. The biggest takeaway from play test #1 was that I received the auction feedback I was expecting. My plan would be to test a new auction mechanic on Sunday.
One player, who happened to be the winner by a lot, wanted to try a strategy that I am aware of but have not yet seen attempted. Since peppers can be sold for coins based on how many of that color are planted in the fields there is a strategy that you can plant a pepper of a certain color in each round and harvest that same color each round without doing anything else. I have done the math in my head and I do not believe that this would be a winning strategy (at least I hoped not because that would make the game broken). More on this below.
Scoville Play Test #2
After working on Protospiel goal #2 of making connections and meeting some awesome people, they were willing to give Scoville a try. During this second play test there was more bidding and jostling of player order. I think that was the reason that the auction was not mentioned in the post-game discussion. This play also resulted in much closer scores than the first play. Here are the suggestions I received:
- Peppers should be worth something at the end (that are currently worth nothing in the endgame: Use Them or Lose Them!)
- The artwork on the fields should somehow better illustrate where the player pawns can be placed.
- The game was described as a “medium to heavy Euro.”
So I received quite a bit less feedback from play #2. But the fact that I still didn’t receive any feedback about how anything seemed broken meant that perhaps Protospiel goal #1 (validation) was starting to become apparent.
Scoville Play Test #3
Later Saturday night a prominent figure in the board game reviewing business was able to play Scoville. So with three other players I got play test #3 going. In terms of rounds this was the shortest game I have seen. The game lasted 6 rounds. The players again seemed to enjoy the game and nothing seemed broken to them. They did mention the auction as the weak point of the game, so I received good feedback about that that I could implement on Sunday. Here’s the suggestions:
- Possible Trademark issue with the names of peppers used on the recipe tiles.
- Turn order needs adjusting. Option 1: Flop the order. Option 2: Purchase your spot.
- Perhaps just get rid of the reverse order for the harvest action.
- Brown peppers seem too valuable.
I want to point out that the brown peppers are somewhat of an enigma in the game. They don’t breed with anything except the best peppers. They take up space on the map. But they are used quite a bit in the recipes. I had not received feedback that browns were too valuable before this. The normal feedback on the brown peppers is that they seem pointless. So this was interesting feedback from a fresh perspective.
I was also pleased, in a bittersweet way, to hear the same feedback on the auction mechanic. I now knew that I could incorporate a revised auction mechanic on Sunday and expect good things.
I was intrigued by the suggestion to remove the reverse player order for the harvest. My first thought was “absolutely not.” What that would lead to is either huge bids during the auction or rounds of the game where one player can make a huge jump in points. I’ll have to examine this further.
Scoville Play Test #4
Sunday morning I was able to play Scoville for the first time during the weekend. I had not played in the previous play tests. And this time it was just a two player game. I have tried to design the game such that it scales well from 2 to 6 players. There are no AI players necessary and the game feels nearly exactly the same with 6 players as it does with 2.
Since it was now Sunday I was going to implement the new auction mechanic: Bid for Player Order. Now during the auction phase players would be bidding for turn order. Whoever bids the most gets to choose their spot in the turn order. The next highest bidder gets to choose the next spot, and so on. This way, if a player wanted to become the first harvester they could bid high and then choose the last spot, which would allow them to harvest first.
The new auction in the two player game seemed to work, but I suppose that this new auction mechanic would work even better with more players. What the new auction mechanic provided was a way to earn the first harvester spot. That is critical to strategy in the game.
Here are the suggestions I received:
- Are points balanced on the Order tiles?
- Change the artwork on the Cross-Breeding table for the cross-breeds that result in two peppers.
The points on the Order tiles may be slightly unbalanced, but not to the point of brokenness. These can be easily revised, which I may do depending on analysis of the scoring for the first 25 play tests. The artwork suggestion is an excellent one that I will definitely change.
Scoville Play Test #5
The final play of Scoville included the big winner from play #1. He wanted to test the coin building theory and see if it could potentially provide a winning strategy. I welcomed him to try it but made sure that the other players were initially unaware of his proposed gameplay. It was a great final play and I was happy to see that the new auction mechanic really worked well with four players. Here are the suggestions:
- Don’t call it “harvesting, call it “breed-vesting.”
- Check out the game Santiago since there is a similar “fields” mechanic (uh oh… worried about this!)
- The different parts of the game were described by one player as Resources (Auction), Tactics (Orders), and Strategy (Recipes).
The first thing to discuss was the auction. Of note is that this game had the highest average bidding per round of all 5 play tests during the weekend. I think this is due to players now having two things to bid for (first player spot or last player spot) rather than for just moving up in player order. The thing of note was the compliment someone gave to the auction saying that the auction was a good mechanic for the game. This brought the game full circle over the weekend. Previously the auction was described as the weak point of the game. Now it was “good.” I’ll take that!
The other thing that was validated from this final play test was that the game was not broken in that attempting to get coins by planting and harvesting the same color did not result in a winning strategy. The player was going full steam ahead from the get-go with that strategy and came in last place (though could have finished in third place). I was pleased that the game wasn’t close to being won by that strategy. Overall it was a great play test.
Overall Scoville Analysis
Perhaps the best part of the analysis is that people really seemed to enjoy the game. While my goal was to validate whether or not it was any good, I came away from Protospiel very humbled by all the kind words people had for the game. Let’s dig in a little bit and check out the scoring breakdown:
Some further analysis revealed that the number of coins bid during the game varied quite a bit. In terms of coins bid per round the numbers were 2, 6.14, 7.66, 1.38 (2-player), and 7.85 per game. The highest average was the 4-player game with the new auction, though this wasn’t unexpected.
Overall it was apparent that people had fun when playing the game. That’s the most important thing to me as a designer. There are some things that I would like to continue to develop leading up to Gen Con that I mentioned to the players. But I want to avoid the situation where I am needlessly adding complexity. That would steal from the simple elegance of the mechanics currently in the game.
Thank you to all 16 players who play tested my game. I really appreciate the feedback. It was an awesome weekend! And special thanks to Grant Rodiek for humbly accepting a copy for the Prototype Penpal Program. I know that I can expect some awesome, honest feedback!