Slaying Mighty Dragons: Competitive Ranking and Matching Systems

“Conan, what is best in life?”

“To crush your enemies, to see them driven before you, and to hear the lamentations of the women.”

Bob you are probably right but I was trying not to be cynical :stuck_out_tongue:

If we imagine an ideal ranking system though I still think it would incorporate some method to stop players “maintaining through stagnation”.

Ideally it would also address the issue of “smurfing”, possibly by analysing what differentiates these players from others with similar rankings and matching accordingly - for instance in an MMO they might have much greater gear (a href=“http://en.wikipedia.org/wiki/Twinking”“twinking”/a), perhaps higher percentages of headshots, or just a faster rate of change of ranking (i.e. climbing the ladder faster).

I’m curious to see what others consider to be fundamental elements of a “perfect” ranking system.

To me, an enjoyable game (online or off) is one where I am nearly beaten, but manage to pull through. Of course, this kind of game is impossible to get every time with a ranking system, but a good one should make these kinds of games a more common occurance.

I agree. I know people I play with, myself included, don’t actually mind losing a close game because we can often see where we could have pulled it off and learn from our mistakes which I think is something Jeff highlighted.

I’ve always wanted some sort of ranking for games when finding servers. Something like the range of their skill. That way you can find a server that’s most appropriate for you. As this post outlines, there are some difficulties though.

I just wanted to add that all the math stuff seems to be pretty basic probability theory. I also imagine the rankings are also based on the Bayes theorem, and derivatives of it.

So if anyone is interested in understanding more than they should look around the net for some basic explanation of probability theory. The basics will also help you understand other topics such as spam filtering.

I often play go online (an asian game equivalent to chess for us) on the KGS server.

You can find an explanation about their ranking system here :
http://www.gokgs.com/help/rank.html
http://www.gokgs.com/help/rmath.html

Basically, they use a system similar to the ELO system, but they improved it by taking previous games into account when calculating your rank. This way, if you play against a strong player who just created a new account, this player will rapidly climb the ladder, and your loss against him will be considered as a loss against a strong player.

It also avoids loosing all your rank when you are in a bad day, since you previous games are here to couterbalance your bad performances.

I agree completely with the ideal of a 50% win/loss ratio. Stomping all over people in an unfair match is about as much fun as being stomped on in an unfair match.

Any matching system also needs to take environmental factors into consideration - there’s no point matching equally skilled players if the network introduces a massive advantage to one side. I found Gears of War particularly bad for this (I’m in the UK).

Of course, Gears online is horribly broken anyway as someone usually quits early (often within a few seconds of starting) which unbalances the game entirely.

Also, I’m mainly interested in the team-play aspects of online games. I like a good blast, but it’s nice to meet people too. Anonymous matching rips the soul right out of Xbox online gaming as you rarely meet the same player twice. I agree that there should be a separate ‘team’ ranking system, but would like to see equal effort spent on ensuring it works. Gears ignored it completely :frowning:

Halo 3 out today over here (yay!). I’m hoping it still has the ‘party’ online mode that Halo 2 had, and a ranking system to go with it.

I dislike those rating systems. I am a casual player, don’t ever play MMO’s or RTS’s or FPS’s. I like simple browser-based flash/java games that can be played in 10 minutes.

I loved “Dice Wars” since the first moment I tried it. It just matches my gamer profile. And, like almost everyone else, I thought it would be a great idea to turn that game into multiplayer. And then “K-Dice” appeared. I celebrated it and started playing. After less than a week, I was fed up with that game and returned to dice wars.

The difference was: I could play dice wars for fun.

K-dice has an ELO-based ranking system. Which at first would sound as a good idea. The problem was, it changed the game into a completely different one. I was not playing a simple strategy game of stacking dice and conquering territories (which was the goal of Dice Wars). I was playing a game of scoring points in some ranking system, and the actual dice games were the means to achieve those points. I wouldn’t just click “start” and play any more, because every game counted. I started considering the chances to win a given game before clicking “accept”. I was really pissed off when I started a game in a bad position which would mean loosing like 500 points in a row.

I was not having fun any more, not the sort of fun I have when I solve a game of minesweeper.

The point I am trying to make is that including a ranking system transforms the game into a different one. Not necessarily a bad one, but not the game I was looking for at the first moment. I am a casual player, so I want games that I can play in 5 minutes, not during a month of effort and time investment.

Reminds me of what Joel Spolsky says about rewarding developers with monetary prizes, paying them for amount of lines written or bugs solved. They change their focus from “developing good software” to “maximizing monetary input”.

I used to love Gears, before Christmas 2006 when every 12 year old on the planet got a copy. The game went from a tactical squad game where team members spoke to each other and coordinated their actions to a curse filled screeching baby voiced cacophony.

Your next blog should be on how children are ruining gaming…

the bayesian approach seems to be one that will
converge quite quickly to optimal values for
everyone playing. team play could be handled
as multivariate gaussians, with the covariance
built up out of the individual player gaussians
on the fly. if teams wanted to be ranked and
play together frequently, they could have their
own 1-d gaussian that followed them around.

on decay: decay is critical. one way that decay
works well is along with a gaussian description
of rank – the uncertainty just creeps up over
time, although the center doesn’t need to move.

one thing that rating systems really ought to do
better: display. either decide that people don’t
want to see the tedious details of how their ranking
is calculated, or lay it all out for them.

as the dice guy mentioned earlier, knowing your own
ranking can make you consistently worried about losing
ranking points. i think that a good ranking system
might just pair players together according to how much
risk they wanted to take (i.e. how closely they’d like
the expected result to be a coinflip), but not give them
any (displayed) number value whatsoever.

s.

"I used to love Gears, before Christmas 2006 when every 12 year old on the planet got a copy. The game went from a tactical squad game where team members spoke to each other and coordinated their actions to a curse filled screeching baby voiced cacophony.

Your next blog should be on how children are ruining gaming…"
Kai Tain on September 26, 2007 03:05 AM

Preach on, Brother! It’s because of those obnoxious 12 year olds, that my friends and I don’t play most games online anymore. They’ve completely ruined the online gaming experience. So to deal with it, I simply don’t go there anymore. Now, we play system linked games or play in LAN parties where we control who can play. Don’t invite the 12 year olds, and the game is still fun.

I appreciate having my rank slowly decay with inactivity for my own personal well being. If I jumped back into Halo 2 online (not like anybody will be playing it in a month) I would get creamed. Having stopped playing just under two years ago people who are “evenly matched” based on where I was ranked when I left are no longer even matches. My skill level is probably half (or less) of the peak ranking I achieved.

Yes, John C above just voiced what I was going to say. Rank decay is not necessarily about keeping players playing – it can also reflect a decay in actual skill. In skill level estimation ranking systems, that is fundamental – one presumes that lacking practice, a player will tipically see his performance decay. I’d like to see both uncertainty increase and the center slowly drop, however.

I’m a big fan of ranking systems and looking forward to see improvements in this area. The configurable difficulty option mentioned above also seems like a great idea – I like playing games where the odds aren’t in my favor.

Cheers!

a href="http://fibs.org/"FIBS/a uses something like this.

One problem with your DD analogy is that you don’t lose XP in DD unless something powerful drains some from you. AFAIK there’s no way to lose XP from a botched encounter with a lesser creature like a kobold.

To me, there should be a difference between “Difficulty Ranking” and “Player Ranking.” Yes, I like being able to play people who are approximately equal in skill (where the difficulty ranking comes into play), but I ultimately don’t care how I rank as a player (i.e. How many games I’ve played, the number of kills/wins/losses, etc).

Does anyone here actually remember playing Diablo II online? After raising your character to Nightmare difficulty, going back through and playing on easy didn’t lower your character’s level…you just didn’t get any experience for whooping up on sewer rats. Which is where the distinction between your difficulty ranking and your (publicly viewable) player ranking would come in handy…

"Wow. You just totally pwned me. You’re a level 65 Pally? Sweet. And you’ve been at that level for…(checks stats)…three years. Although you’ve played over 5000 matches. On easy.

Get a real hobby, bro."

“In order to increase his rank, he must defeat other higher ranked players. If a high ranked player loses to a low ranked player, he loses much more of his rating THAN he’d gain if he won the match.”

If you don’t know when to use them: a href="http://www.sparknotes.com/writing/style/topic_172.html"http://www.sparknotes.com/writing/style/topic_172.html/a

Grammar Police, that section is quoted from someone else. That’s why it’s in a blockquote tag.

http://jobemakar.blogspot.com/2007/05/why-elo-rating-system-rocks.html

So you might want to post a comment on that blog… then.

AFAIK there’s no way to lose XP from a botched encounter with a lesser creature like a kobold

I was thinking that myself while driving in to work this morning. It’s close, but not exactly the same. The monsters aren’t your peers, they are disposable non-player characters. They can’t gain or lose rank because they have no persistence.

The point I am trying to make is that including a ranking system transforms the game into a different one.

Sure. All this ranking and matching talk assumes that players want to play competitively against each other. There are plenty of other valid forms of play-- cooperative, solo, or unranked.

I think implementing rating decay to match a user’s decay for not playing is a flawed reason. A player who doesn’t play ranked matches may be playing unranked matches, or playing ranked matches on another account. Effects like getting out of practice should be naturally reflected in the resulting wins and losses, not anticipated by the scoring system.