Paul Buchheit on listening to users:
This is a companion discussion topic for the original blog entry at: http://www.codinghorror.com/blog/2008/02/i-repeat-do-not-listen-to-your-users.html
This is a companion discussion topic for the original blog entry at: http://www.codinghorror.com/blog/2008/02/i-repeat-do-not-listen-to-your-users.html
Jeff, no new achievements (that I know of) have actually been released for the medic. All that’s happened so far is that someone found a bunch of logos names for medic centric achievements.
Evan, but they have announced that achievements will be released. Also, IIRC the article I read stated the medic will be the first class to receive new weapons, which are unlocked when you reach a certain level of medic specific achievements.
I suppose this is because Medics don’t engage in much direct combat, so they’re not as exciting to play as, say, a Demoman or Soldier.
The other factor amongst gamers is that Medics/Healers are the first to be targeted and killed making them more frustrating to play.
The other factor amongst gamers is that Medics/Healers are the first to be targeted and killed making them more frustrating to play.
Oh I’m not so sure… often they are hard to get if they are behind a spamming Heavy. And players are often targeting the Heavy, because they don’t know better.
That was an awesome post! You put into words what I have been thinking/doing (and not necessarily succeeding at) over the past year.
Henry Ford said something like: “customer’s can’t envision the future, but they inform the present.”
Nobody plays medic because the class got butchered in transition from TFC to TF2.
The other factor amongst gamers is that Medics/Healers are the first to be targeted and killed making them more frustrating to play.
Heck, there’s even a T-Shirt on this…
one of the things I use to help improve the software i write for my school is an error email account. i wrote an error class to send the details of errors to that account. i and the support staff can check that account and use the data to be proactive about fixing bugs and also as objective data to accompany help tickets that may stem from the bugs. we usually find that the source of the problem is users doing things we never expected them to do. it’s an invaluable tool.
Hey Jeff. Cheers for the link back to ubercharged.net. Been following this blog for a while, but mainly from the day job coding side of things. Saw the link coming in when I was checking stats, and did a double take when I saw it was linking to my TF2 blog (my other blog is about coding at www.rawblock.com , but is in a state of a little disrepair at the moment), weird how the interweb works sometimes.
The new medic achievements have been due to be released in the “coming weeks” for a few months now. I guess that’s running on valve time ( http://developer.valvesoftware.com/wiki/Valve_Time ), so it means we’ll see a release any time in the next year or so 
Take privacy into account of course.
Great post, I agree with a good deal of it. After working in sales for a while, I have learned that people say what they think that they want, but not what they need. You need to qualify responses that you get from users so that you truly understand the needs, instead of just blindly running off to meet their wants.
A great example of where “listening to the users” leads to very poor usability is the one and only: Microsoft Live.
I think you overstate the point that listening to users is a bad idea. In particular, the Team Fortress 2 argument doesn’t really support your point of view. I actually play Team Fortress 2, as do some of my friends, and ask any of us why we don’t play medic and you’ll get the same answer without even having to gather detailed statistics.
Medics are the bitches of the game. They don’t get to do much combat, and they get yelled at if they don’t heal everyone who needs it, even though the best personal strategy for them to get more points is to stick like glue to one player. They spend all this time charging up their uber, which seems like it should be cool, and then what do they get to do with it? More holding the button down on the medigun, only this time instead of healing, it makes you invulnerable. Because actually letting them participate in combat with a super attack would’ve been too cool.
Your example on the Halo maps does support your argument. But Team Fortress 2 doesn’t.
Although I agree with your post, I want to point out that adding 1920 x 1200 or 2560 x 1600 resolutions is usually nothing more than a menu item, an extra enumeration member, and two extra lines of code.
A comment on your example of Valve’s hardware survey - it’s also important to know how the data was gathered. In this case I believe the Steam application took the specs outside of game time, that is, in the background while you’re doing your everyday tasks. In my case my computer (a laptop) is usually connected to a 1900x1200 screen which is fine for coding and desktop use, but for the newest games I usually drop it down to 1024x768. However, this means that I’d be reported as having 1900x1200, which wouldn’t be so useful if they’re deciding what gaming resolutions to support.
This medic argument brings up an important point. When you have data like this, how do you decide between multiple hypotheses of what the user actually needs?
Clearly, there are at least two competing arguments as to why the medic class is the least played. (This is just my interpretation from the comments here. I don’t play TF2.)
-
Balance. The Medic class has fundamental weaknesses that contribute to a low desire for players to play this class that are best addressed by changes to the classes abilities.
-
Learning curve. The Medic is balanced, but is simply difficult to play. The best way to address this is to add additional achievements to reward skilled and dedicated Medic players to help them get through the learning curve.
3?) Fun factor. The Medic is balanced, but game play elements are not as fun as other classes. People like achievements though, give them those, because re balancing is hard, and costly to get right.
That 5.5% number simply doesn’t provide enough information about what is going on. It would be interesting to see what the real process was to arrive at the answer they did. The canonical way to do this would be to gather some new data, designed to test the competing hypotheses, but in my experience this tends to be an “expert’s” call. Time is a finite resource, so people tend to make do with the data they have.
I suspect part of the lack of medics is the results of the third and sixth charts, “Average kills per class per hour” and “Average lifespan per class in minutes” where medics are also at the bottom. I wouldn’t play the thing where I get the least kills and die the most, either.
Your users are experts in what problems they have. But not necessarily experts in solving the problems.
“Listen to your users” is still a different concept to “outsource your entire product design department to the most vocal disgruntled user you can find”.