bias, n. [bahy-uhs]
1. a personal and sometimes unreasoned judgment
2. Statistics – a systemic, as opposed to a random, distortion of a statistic as a result of sampling procedure
There is a natural human tendency for individuals in a hobby to rate the objects of their affection, whether it be movies, comics, or whatever. Gamers are by no means immune to this practice. There just seems to be some primal need to establish what your favorites are and to notify the rest of the world. And once that information is out there for a group of people, clearly someone is going to consolidate it and come up with a list of the best whatevers of all time. In the gaming world, the most recognized of these lists is probably the one compiled by BGG, in which games are ranked by their Geek Rating. Being one of the top 100 games on the Geek is a feather in any game’s cap.
But the thing with lists like these is that there’s always some bias present. I’m not talking about deliberate bias, as is cited in the first definition of the word I displayed at the beginning of this article. And I’m certainly not referring to tires! No, I’m talking about statistical bias, where, due to the makeup of the voting group, the voting procedures, or any of a host of other factors, there is a tendency toward unbalanced results in the lists.
For example, about a dozen years ago, the Geek ratings were biased in favor of what would eventually become known as Eurogames. Thematic titles tended to get lower ratings then they probably deserved, which led to flame wars, a mass exodus of gamers from the Geek, the launching of the Fortress: Ameritrash website, and all sorts of exciting stuff. Still, the bias was unfortunate and led to many gamers feeling underrepresented.
Ironically, thematic games are very highly rated these days, so this bias is probably no longer present. But there remain quirks in the ratings. The most prevalent, in my humble opinion, is the bias in favor of newer games and against older designs.
Now, I could roll out all kinds of statistical flap-trap, filled with Greek symbols and such, to try to prove this point. But sometimes, a simple glance suffices. Here are the facts, with all statements being based on the Geek ratings of January 1, 2019:
- 8 of the top 9 rated games on the Geek were published between 2015 and 2017.
- 14 of the top 20 games came from this period as well.
- These trends continue, as over half of the top 50 and over 40% of the top 100 games were released between ’15 and ’17.
Now companies have been publishing games for a very long time. It’s certainly fair to say that most of the titles that came from the 20th century are of little interest to the vast majority of people who rate games on the Geek. Still, starting in 1990 or so, or 1995 at the latest (the year when Catan was published), we started to see some very interesting and excellent games released. So let’s say we’re talking about 25 years of notable titles. If the Geek ratings are to be trusted, 8 of the best 9 games from that 25 year period came out since 2015. That’s only the last four years (and really, it’s only the last three years, since there hasn’t been enough time for the 2018 games to get enough votes to break into the upper echelons of the ratings). Three years out of 25, and yet it’s represented by almost 90% of the top 9 and 70% of the top 20 games. That, my friends, is bias.
So is it possible that game design has improved so much in the last few years that these rankings are pretty accurate? I guess it isn’t inconceivable, but I don’t come close to buying it. For one thing, we’re talking about a massive representation from a very short period of time, much more than could be explained by advances in game design. For another, a bunch of great games came out during the late nineties, the aughts, and the first part of the 2010’s, but very few of them are highly ranked (and, in fact, a lot of them are far down in the rating charts; the older the games are, the more pronounced this effect is). And finally, this tendency to favor recent titles has always been present in the Geek rankings, although probably not as dramatically as we see today. It’s just a weird consequence of the way that people rate games, a…well, a bias.
Let me be clear: I’m not saying there’s any sort of conspiracy to rob El Grande of its god-given right to be a top 50 game. No, I don’t think there’s anything sinister going on here. There’s actually a bunch of factors that lead to this bias and detailing them all would take more space than I care to devote to the issue. I just want to point out that older games tend to have depressed ratings on the Geek and, consequently, it’s difficult to use those ratings to judge the quality of these games.
So, assuming that we want our game ratings to actually be somewhat accurate, what can we do about this discrepancy? A lot of arcane, mathematically-based methods have been proposed in the past. But once again, I’d like to keep things simple. What if we look at the peak position of these games? For example, Puerto Rico was the top-rated game on the Geek once (in fact, it held that position for over 6 consecutive years). During that period, it was considered, at least by the members of the Geek, to be the best game in the world. Currently, PR is ranked 17th on the Geek. Now, the game hasn’t gotten any worse during the past 10 years, so it isn’t unreasonable to say that the fact that it was once a #1 game is more meaningful than where it currently resides in today’s rankings. That doesn’t make it the best game ever, but there’s reason to say that it’s one of the best.
Peak position has a lot of attractive features. For one, it obviously eliminates the bias against older games, because games are being judged at the height of their popularity, against their own peers. Another really good aspect is that it reflects how the games were viewed during their first few years. There are so many good games that were widely played when they were first released that, for one reason or another, have completely fallen out of favor. In a few cases, this represents a lack of staying power, but far more often, it stems from a series of capricious events that have nothing to do with the quality of the game. Maybe another game with similar features came along a year later and overshadowed the earlier title. Maybe the publisher had financial problems, or the game went OOP and it’s hard to find copies to play. All sorts of stuff can lead to a game being forgotten before its time; it’s really quite common. But these are still great games, far better than their ratings would lead you to believe. And their peak position would reflect this forever, the knowledge that at some time, this game was considered one of the best designs in the world.
The problem is, the Geek doesn’t include historical rankings. So unless you were part of the hobby and closely following the Geek a dozen years ago, there’s no way for you to find out that Caylus (currently ranked 47th; it’s days in the top 50 are definitely numbered) was once the second ranked game on the site. Well, at least there wasn’t until recently.
JonMichael Rasmus is one of the more prolific statistical contributors to the Geek, a fellow who regularly posts gaming analyses on numerous subjects. In May of 2017, he posted this Geeklist—https://boardgamegeek.com/geeklist/224892/every-top-100-game-close-complete-i-get—in which, based on a great deal of independent research, he provides the peak position for every game that was listed in the Geek 100 since August 14, 2001. Included in this analysis is data from before May of 2005, which had been missing from earlier lists, since it was hard to find and based on charts with fewer than 100 positions. Rasmus updated the Geeklist on February 23, 2018, and the referenced list is current up to that date.
This list represents an enormous amount of research and completely delighted me when I saw it, as I had been seeking historical peak position data for some time. In fact, I was so taken with the list that I decided to keep it current, so I have been updating my own copy of it every week since its original appearance. With a huge tip of the hat to JonMichael, I’d like to share some details of that list with you.
According to Rasmus’ research, since 2001, 338 games have attained a position in the Geek top 100 for at least one week. 7 games were ranked #1, 62 games got as high as the top 10, 144 made the top 25, and 227 landed in the top 50. Just as an example of the wide variety of games that reached the top strata at some point in time, here are all the games that made the top 10, listed in order of the number of weeks they stayed at that position (and with the year of publication given in parentheses), as of January 1, 2019:
Puerto Rico (2002)
Twilight Struggle (2005)
Pandemic Legacy: Season 1 (2015)
Paths of Glory (1999)
Tigris & Euphrates (1997)
Through the Ages (2006)
Through the Ages: A New Story (2015)
Terra Mystica (2012)
Power Grid (2004)
Die Macher (1997)
The Settlers of Catan (1995)
The Princes of Florence (2000)
War of the Ring (1st edition) (2004)
Memoir ‘44 (2004)
Terraforming Mars (2016)
El Grande (1995)
Android: Netrunner (2012)
Command & Colors: Ancients (2006)
Star Wars: Rebellion (2016)
Up Front (1983)
Twilight Imperium (3rd edition) (2005)
Le Havre (2008)
Wallenstein (1st edition) (2002)
Dominion: Intrigue (2009)
Europe Engulfed (2003)
Age of Steam (2002)
7 Wonders: Duel (2015)
Brass: Lancashire (2007)
Hammer of the Scots (2002)
Mage Knight Board Game (2011)
Space Hulk (3rd edition) (2009)
Modern Art (1992)
The Castles of Burgundy (2011)
Gaia Project (2017)
Hannibal: Rome vs. Carthage (1996)
Race for the Galaxy (2007)
Great Western Trail (2016)
Go (~2200 B.C.)
Carcassonne: Hunters and Gatherers (2002)
Star Wars: Imperial Assault (2014)
Full Metal Planete (1988)
The 7th Continent (2017)
Puerto Rico: Limited Anniversary Edition (2011)
Ticket to Ride (2004)
That’s quite an interesting list! If you’re new to the hobby, I’m sure there’s a ton of games there that you’ve never heard of (including a reasonable number of classic wargames, a category of designs that have literally no representation in the current Geek 100). But if you’ve been gaming for 10 or 20 years, there’s undoubtedly a lot of titles there that will bring back some fond memories, including, I hope, a few that remain personal favorites, but which are rarely mentioned these days. That, to me, is the real value of JonMichael’s research.
Well, with all this data, the temptation to analyze it is almost irresistible. And far be it for me to resist temptation! So in tomorrow’s article, I’ll look at the trends in the list, come up with a (hopefully) more refined way of interpreting the data, and then use it to create a new Top 100 games list. Until then, let us know what you think of this approach to rating games in the comments section.