I’m surprised no one’s mentioned the automatic part of the BCS (ranking of college American football teams) and Google’s web page ranking (how Google [and everyone else] orders search results).
Google rank is an example of a random walk probability model, and you can do either point or full Bayesian estimates (with or without priors, which amount to damping in the graph algorithms). Bayesian algorithms (typically sampling) tend to be more robust than point estimates (typically EM or gradient descent), but can be more expensive to compute. The general class of algorithms were developed in social network analysis.
To reduce game rankings to Google rank, represent a player A as a web page that has one link to every player B beaten by player A (count multiple defeats as multiple links). This produces a graph where the link weight from player A to B is the number of times A was beaten by B. Then the Google rank of a player is their rank.
There’s a good explanation of both the probability model and algorithms behind these kinds of rankings on the Wikipedia:
I’ve started playing chess on FICS this month and I’ve read the Wikipedia entry on Elo ratings a couple of weeks ago. Funny you mentioned it.
The problem with the rating system is of course that people tend to take these ratings seriously. As if they have some intrinsic meaning. So, they choose players to improve their rating.
This morning, I was musing over your blogpost, and thought; Why not do a binary search for your opponent? Suppose the lowest level player has 800 points, and the highest has 2200 points. Now, an entry level player should play against a player close to (2200+800)/2 = 1500 points. If he wins, he should play against (2200+1500)/2 = 1850. If he loses, he should play against (800+1500)/2 = 1150. The lower and upper bounds should be determined in subsequent rounds, by the highest he loses from and the lowest he wins from.
I used to play a tactical realism shooter and some of the most fun I had (and the times I learned the most) was when I was effectively cannon fodder for the best players. (No, I’m not a masochist, I actually became one of the best players after a time – because I kept playing with, and learning from, the best).
It seems to me that a lot of this ranking crap would specifically prevent a motivated newbie from learning from the best, and thereby short-circuiting the tedious learning process.
And sometimes it’s fun to steamroll a bunch of lesser players, especially when they loudly extol their skills. :^)
I’m with Sean Patterson – allow for the choice of skill matching, but don’t require it.
Of course, my favourite game of all time was a community mod by and for the lunatic fringe and I can’t stand most commercial games, so I’m sure the unwashed masses actually LIKE this ranking…
Wow. I wish that I knew how to play this game - it’s so cool! My older sister doesn’t like it, though. She calls it the “game of marital discord” because her husband absolutely LOVES the game and goes out to play it with some friends on Fridays. He doesn’t get back until really late, or so I’m told. Ta Ta!
I think that it is inevitable that a ranking system will have bugs and flaws, people are too complex to completely predict with a standard system.
Yipyip has a point, if you don’t play with the good players you’ll never learn from them. There is nothing that can replace that.
The ranking decay can’t just be ignored and the fall of skill level worked out in the wins and losses. the reason for this is because the ranking is meant to prevent unequal matches. The ranking decay would also discourage smurf accounts. Also it prevents someone from attaining a high state and keeping it by not fighting anyone.
I agree with the idea that the ranking should be hidden to prevent the game becoming rank-oriented. To the players it should only seem that they become better but are continually challenged.
Try to get your friends invited to a TrueSkill game. Not gonna happen. They have to be randomized. Anyways Trueskill isn’t the end all. In Gears I was always thoroughly owned in probably 9/10 of the games I tried online. Shouldn’t that never happen? Even if I suck I shouldn’t do that bad should I? That and listening to people talk about how drunk/stoned they were drove me away.
As Alan mentions above (and is also mentioned in the cited article “Collective Choice: Competitive Ranking Systems”), the TrueSkill rating system is based on the Glicko rating system. The innovation of incorporating uncertainty into the rating system did not come from Microsoft with TrueSkill. It was already present in the Glicko system and has been used for years on the Free Internet Chess Server.
Back in my Unreal Tournament days, there wasn’t a system or anything on the servers, but most of the good server farms had Beginner/Intermediate/Advanced next to game room for you to hop into. That way the newbies or extremely rusty could work their way into the system, and when you felt bold enough, you walked into a “gladiator” arena with the experts.
I don’t have XBox Live, but does anybody know if they have an “open” room as well. There are definitely two areas to play to. The non-competitive gamers that just want a good match, and the competitive folks that want a legitimate way to say that they’re the best.
Shouldn’t a player’s skill level be tested to determine a level of expertise? It would be natural for me that any type of game would have certain skills that the player would have to master to gain a rating.
This is similiar to fitness test. How many pushups can you do? How fast can you cover a mile. Could games build in tests for players to determine a skill level? Probably. These could be administered to any player to determine a skill level for the player. Similiar thing with golf for handicapping.
I think you could do the same for any game if some thought went into what kind(s) of tests you would put the players through.
Analogy to Dungeons and Dragons has also the role playing side:
I agree on the idea that its fun to beat monsters when you are only nearly beaten. So I would learn to advance faster when I could choose the monsters I want to take out. I like the learning and freedom of choice. If I was tired, I could match against easier monsters but make still some progress. But if the game chose the monsters for me, I would be forced to take out those, even if I could do better or even loose.
In the real world there are no match making systems. For a role playing game a match system would be ok only for arranged battle arenas. Though that way a skillful player could slay all the others in a wilderness. That would require that skills don’t affect too much compared to player’s character level.
And what is wrong with unbalance? Certain creatures are more powerful than others, but by teaming up you can beat stronger characters. Why not let the player be a human, hobit, or a dragon?.