So in yesterday’s article, I introduced the concept of using a game’s Peak Position (the highest rank it has attained at any point in time in the Geek’s list of its top 100 games) as a better measure of its “quality” than its current ranking (particularly for older designs). The Peak Positions I’m using are Sbased on research carried out by JonMichael Rasmus, who summarized his results in the following Geeklist: https://boardgamegeek.com/geeklist/224892/every-top-100-game-close-complete-i-get. I’ve been updating these results for the last 20 months to keep them current.
Hopefully, I’ve made a good enough case for Peak Position that you feel it has some value in evaluating games over time. So let’s start delving into this data. As I mentioned yesterday, there are 338 games which have been ranked in the Geek’s top 100 since August of 2001. We’ll start by breaking down those games by the year of their publication and which quartile (ranks 1-25, 26-50, 51-75, and 76-100) their Peak Position falls into. I’ve decided to use the following publication time periods for the purposes of analysis: pre-1990, 1990-1999, 2000-2004, 2005-2009, 2010-2014, and 2015-1019. So five year slices for the 21st century games, and longer periods for the older ones. First, as a means of comparison, let’s look at the games in the current top 100. Next to each time slice, I’ve listed the total number of games from that time period which are in the top 100, followed (in parentheses) by the breakdown of games appearing in first, second, third, and fourth quartile:
Pre-90: 2 (0, 0, 1, 1)
’90-’99: 2 (0, 0, 2, 0)
’00-’04: 3 (1, 1, 0, 1)
’05-’09: 15 (3, 4, 5, 3)
’10-’14: 37 (6, 9, 11, 11)
’15-’19: 41 (15, 11, 6, 9)
You can see how much more recent games are favored, particularly for the highest ranked designs. Almost 80% of the games come from the past decade and titles released prior to 2005 are practically invisible.
Now, here is the same breakdown for games ranked by Peak Position:
Pre-90: 19 (9, 6, 4, 0)
’90-’99: 47 (29, 8, 7, 3)
’00-’04: 75 (36, 21, 12, 6)
’05-’09: 80 (25, 20, 19, 16)
’10-’14: 72 (25, 20, 12, 15)
’15-’19: 45 (20, 8, 5, 12)
Pretty different, huh? The five-year periods from 2000 to 2014 are almost equally represented. The most recent games seem to be getting short shrift, which looks problematic, but keep in mind that there’s only a little bit more than three years reflected in these numbers (since there’s only been time for a few 2018 titles to get enough rankings to make the top 100 list and since 2019 has just begun). If the same proportions hold a few years from now, the odds are pretty strong that there will be at least 70 games from the ’15-’19 time period with Peak Positions of 100 or less, which will put that time slice right in line with the three previous ones.
One other thing worth noting is that considerably more games have their peaks in the 1-25 quartile than in the other ones (and the percentages go down with each quartile). Here are the totals for each quartile, followed by the percentage of all 338 games which have their peaks in that quartile:
1-25: 144 (42.6%)
26-50: 83 (24.6%)
51-75: 59 (17.5%)
76-100: 52 (15.4%)
Part of this is the natural result of a statistic that’s based on the highest ranking that a game has achieved over its lifetime. But it’s also a consequence of the fact that, according to JonMichael, the first four years’ worth of data were based on lists of less than 100 games each. I don’t know how much less than 100 we’re talking about, but it makes it easier to understand why the games from 2004 and earlier have such a large proportion of their peaks falling in the 1-25 quartile. Naturally, the breakdowns from those earlier games make the overall percentages more dramatic than they would otherwise be.
So is Peak Position the absolute best way to rate games? Are there no further refinements that can be made? Of course not—I still have half an article that needs to be filled! While it’s really nice that the method puts older games on a level playing field with newer designs, many of you may have realized there’s a possible inequity with this. Namely, back in 2001, there were a whole lot fewer games, and, specifically, a whole lot fewer good games than there are today. The same thing is true, to a lesser extent, when you compare, say, 2010 with 2019. Basically, to make the top 100 today, you have to compete with all the great games that have come down the pike, extending over a period of 30 or more years; back in 2001, you were vying with, for the most part, only 10 years or so of games (and much fewer games were published every year back then). Because there was so much less competition, it was considerably easier for an excellent game to be highly ranked back then than it is today, so the playing field really shouldn’t be completely level.
Look at it this way: when Puerto Rico became the top rated game on the Geek, it didn’t have to compete with Gloomhaven, the current #1 game, not to mention another subsequent #1, Pandemic Legacy. Would it still have grabbed the top spot if those two games were around at the time? Who knows! But it certainly would have been harder to do so.
Here’s another data point. Forum Romanum is a 1988 Kramer multi-player abstract that reached a peak of #11 late in 2001. There’s a good chance that most of you have never heard of it. I’ve actually played it quite a bit, at least a dozen times during the early 00’s. I’m quite fond of it, but at no time did I ever think it was one of the ten best games ever designed. And yet, 17 years ago, enough people thought sufficiently highly of it that, given the lack of competition, it almost cracked the top 10. There’s no way that something like that could happen today.
So I think it’s fair to say that the Peak Position method slightly overcompensates and gives older games a bit too much credit. The older the title, the more its value is probably overstated. So if we want to truly rate games from any time period fairly, we need to rein back the value of older designs a bit. How do we do that?
Well, to start with, let’s come up with a base system for evaluating the value of a specific game, so that we have a way of comparing groups of games (such as games of a particular genre, or from a specific designer). Once we have that base method, we’ll have something we can modify based on the age of the game.
There’s no right or wrong way to do this, so, as usual, I’ll try to come up with something simple that behaves the way I think it should. Since we’re looking at games with a peak position that ranges from 1 to 100, one approach might be to give 100 points to a #1 game, and then give each position one fewer point, until the #100 game gets 1 point. I don’t think that works. It means a game with a peak position of 51 gets 50 points, half as much as a #1 game—that seems way too strong. And, that #51 game gets 50 times as many points as a #100 game, which also feels out of whack—after all, it’s a pretty great achievement to be considered the 100th greatest game of all time. No, we can do better. I started by asking myself how many more points I wanted a #1 game to be assigned than a #100 game. A factor of 10 times as many points felt about right. Moreover, I needed those #100 games to be given more than just a few points, to keep them from being overwhelmed by games ranked only slightly higher. Finally, it seems as if the differential between ranks should increase the higher up the scale you get, since it feels as if the difference between the #1 and #2 games is greater than the difference between #100 and #99. Once I established those rules, it didn’t take long to come up with a scoring method. Here’s what I’m using:
|Add. Pts. per Rank||Point Range|
So the points assigned to a game range from 50 for a game with a peak position of 100 to 500 for a game ranked #1.
Now that we have a point scale, let’s modify it for age. Once again, simple is best, as long as it seems to do the job. I’ll consider a game’s age to be the last date that it reached its peak in the Geek rankings. Unfortunately, Rasmus’ Geeklist doesn’t record that information, but we can come up with a good approximation by taking the date each game first peaked and adding the number of weeks it stayed at that position. This isn’t always exact, because those weeks at the position might not have been consecutive. For example, after Agricola wrested the #1 spot from Puerto Rico back in 2008, there was a period of a couple of months when the top spot flip-flopped between the two designs. So the last date of PR’s peak was actually a bit later than we’re giving it credit for. But that’s the best data we have and the difference in evaluating each game figures to be miniscule at worst.
The method I’ve decided on for reducing the point values for older games is simply to subtract 1 percentage point for each 6 months from the current date. So, since I’m using January 1, 2019 as my analysis date, games from the last 6 months of 2018 get full credit, games from the first 6 months of last year have their point totals multiplied by 0.99, then 0.98, 0.97, and so on for each 6 month period. The largest modifier I’m applying is a reduction of 34%, or about one third of the point value, for games which attained their peak in 2001. Again, there’s no real science to this—it just feels appropriate to me.
So all of this gives us a way of rating games, or at least ones good enough to have made the Geek 100 at some point. I don’t want to take it too seriously, since I’ve taken hard data from the Geek and laden two somewhat arbitrary modifications to it. But it’s still a reasonable method, so let’s have some fun with it. The obvious way of applying it is the gamer geek’s favorite pastime, ranking game designers. I’ll do that, in some detail, in tomorrow’s article. But for now, let’s use the method to come up with its top 100 games. Here are the games with the 100 highest point totals, starting, naturally, with the current #1 game on the Geek:
|2||Pandemic Legacy: Season 1 (2015)||490.00|
|3||Through the Ages: A New Story (2015)||470.40|
|4||Twilight Struggle (2005)||470.00|
|5||Terra Mystica (2012)||451.20|
|6||Terraforming Mars (2016)||440.00|
|8||Through the Ages (2006)||427.20|
|10||Star Wars: Rebellion (2016)||411.60|
|11||Puerto Rico (2002)||405.00|
|12||Android: Netrunner (2012)||396.00|
|14||Power Grid (2004)||384.00|
|16||7 Wonders: Duel (2015)||368.60|
|17||Gaia Project (2017)||360.00|
|19||Mage Knight Board Game (2011)||357.20|
|20||Le Havre (2008)||344.00|
|21||The Castles of Burgundy (2011)||342.00|
|22||Great Western Trail (2016)||340.00|
|23||Dominion: Intrigue (2009)||336.00|
|24||Tigris & Euphrates (1997)||335.00|
|25||Memoir ’44 (2004)||331.20|
|25||War of the Ring (1st edition) (2004)||331.20|
|27||Brass: Lancashire (2007)||330.60|
|28||Commands & Colors: Ancients (2006)||330.00|
|28||Paths of Glory (1999)||330.00|
|31||The Settlers of Catan (1995)||326.60|
|33||Die Macher (1997)||321.60|
|34||The Princes of Florence (2000)||317.40|
|35||El Grande (1995)||316.80|
|35||The 7th Continent (2017)||316.80|
|37||Space Hulk (3rd edition) (2009)||311.60|
|38||Twilight Imperium (3rd edition) (2005)||306.60|
|39||Star Wars: Imperial Assault (2014)||304.00|
|40||Wallenstein (1st edition) (2002)||296.00|
|41||Europe Engulfed (2003)||292.00|
|42||War of the Ring (2nd edition) (2012)||288.80|
|44||Race for the Galaxy (2007)||288.00|
|44||Twilight Imperium (4th edition) (2017)||288.00|
|46||Robinson Crusoe (2012)||283.92|
|47||Puerto Rico: Limited Anniversary Ed. (2011)||281.60|
|51||Up Front (1983)||277.20|
|53||Blood Rage (2015)||271.60|
|54||7 Wonders (2010)||271.44|
|55||Age of Steam (2002)||269.80|
|58||Hannibal: Rome vs. Carthage (1996)||266.40|
|59||Hammer of the Scots (2002)||266.00|
|60||Spirit Island (2017)||264.00|
|60||Viticulture Essential Edition (2015)||264.00|
|62||Arkham Horror: The Card Game (2016)||261.36|
|64||Dead of Winter (2014)||255.68|
|69||1960: The Making of the President (2007)||246.48|
|70||Mansions of Madness (2nd edition) (2016)||245.52|
|71||Modern Art (1992)||241.20|
|73||Dominant Species (2010)||236.64|
|74||Carcassonne: Hunters and Gatherers (2002)||234.60|
|75||Go (~2200 B.C.)||231.20|
|77||Ora et Labora (2011)||229.68|
|78||Mechs vs. Minions (2016)||227.36|
|79||Ticket to Ride (2004)||227.20|
|80||Battlestar Galactica (2008)||224.40|
|81||Star Wars: X-Wing Miniatures Game (2012)||223.20|
|83||Full Metal Planete (1988)||214.40|
|84||Taj Mahal (2000)||209.76|
|85||Forum Romanum (1988)||209.04|
|86||7 Ages (2004)||207.36|
|87||War of the Ring Collector’s Edition (2010)||206.40|
|88||Das Motorsportspiel (1995)||205.92|
|88||Railroad Tycoon (2005)||205.92|
|91||A Game of Thrones (first edition) (2003)||204.48|
|92||Ticket to Ride: Europe (2005)||204.40|
|93||Lost Cities (1999)||204.24|
|94||Eldritch Horror (2013)||200.88|
|95||Age of Empires III (2007)||199.68|
|96||Lord of the Rings: The Confrontation (2002)||198.80|
|97||Food Chain Magnate (2015)||196.00|
|98||Mu & More (1995)||195.36|
|99||Mage Wars Arena (2012)||194.40|
That’s a really interesting mix of old and new titles. Here’s the breakdown, by quartile and time slice:
Pre-90: 6 (0, 0, 4, 2)
’90-’99: 12 (1, 6, 2, 3)
’00-’04: 20 (4, 4, 6, 6)
’05-’09: 21 (6, 8, 2, 5)
’10-’14: 23 (7, 4, 5, 7)
’15-’19: 18 (8, 2, 6, 2)
In the first quartile, the most recent games still have the edge, although not nearly by as much as with the Geek ratings. But for the entire 100 games, all the time periods from this century are pretty evenly represented and even the pre-2000 games have an appreciable number of titles in the list. So overall, this seems like a more balanced ranking system than the Geek 100.
Anyway, tomorrow, we’ll end the series by using Peak Position to rank the hobby’s greatest designers. See you all then!
Thank you for this work Larry.
Echoing my comments from yesterday’s post, this is a valiant but flawed construct. At issue is the assumption that a newer game had more games to beat out and should therefore be afforded higher marks, but this discounts the role that the huge influx of gamers has had. Gloomhaven doesn’t have to compete against every other game, it just has to compete against other games that are in print and are actually on those new gamers’ radar screen. In a churn era that may not actually be that many games, in a hype era it’s not hard to believe that those new gamers’ rating means something different (*), and in a “too many games” era it’s easy for a game to be rated primarily by those already predisposed to like it.
(*) Case in point, Wingspan has been out for a couple of weeks and just cracked the top 500, and doubtless will be flirting with the top 100 in a month or two. Is this really one of the 100 best games of all time, and more importantly do we really trust ourselves to know this to be the case in so short a time?
Jeff, it’s true that new games aren’t competing against *every* other game in the Geek database in their struggle to reach the Geek 100, but the effect still isn’t insignificant. After all, almost 60% of the games currently in the 100 are over 4 years old and over 20% were released prior to 2010. Plus, many more games are published each year than there used to be. So some sort of handicapping seems to be necessary.
Analyzing ratings is a complex thing and you’ve obviously applied a lot of thought to it. My position is that there’s information that can be gleaned from earlier rankings, which is not necessarily reflected in the current ratings. Peak position shows how games performed when competing against their peers, which removes at least some of the issues you raise.
Well, sort of, but again I think any manipulation of the numbers that doesn’t take into account the sources of those numbers, and “correct” for that (if we could agree on how that should be done), can only tell us so much.
It’s as if we asked “what are the best albums of all time?” and the majority of voters had never even heard of Love Supreme, Blue, Exile on Main Street, and had heard of but never listened to Nevermind, Blonde on Blonde or In the Aeroplane Over the Sea, but man, that new Panic! at the Disco album is lit, y’all! There’s just a limit to the information content that such a group of respondents can give us.
Agreed, the ratings are flawed and very non-scientific. But I’m not trying to determine national policy with them; I just want to have some fun and figure out the relative popularity of some games. So instead of delving deeply into the ratings, I’m trying to sidestep the issue entirely by going into the past and taking previous results into account. If we were talking about music, I’d be inclined to do something similar and look at the old Billboard charts, to see how popular Glenn Miller and Frank Sinatra were during the 40’s, and so on. The Truth may not be out there, but the data certainly is, so why not use it?
But your example is not accurate. In BGG, old ratings are still counted, and that is why Puerto Rico or Carcassonne have more votes than Terraforming Mars or Scythe, for example. So, in our case, the majority of voters has heard, played and rated the classics. But not as highly as new games have been rated. Nothing wrong with that.
Also, here’s a funny thing. I had the thought to see how the games fall out by “number of owners” but I don’t see how to do that at BGG. But it IS possible to sort by number of voters. Using that as the metric, here is your BGG top 20:
4. 7 Wonders
6. Ticket to Ride
8. Puerto Rico
9. Small World
10. Power Grid
12. King of Tokyo
13. TTR: Europe
14. Love Letter
18. Race for the Galaxy
19. 7 Wonders Duel
20. Lords of Waterdeep
Gloomhaven is down in the 70s according to this way of listing; but Scythe and Terraforming, both 2016, are 22 and 25, so it’s not as biased for old games as you might think.
I could quibble about the order and some good games are missing but I think I’d be more comfortable saying “these are the 20 greatest games of all time” than saying “the current BGG list is the 20 greatest games of all time”, if I had to pick one or the other!
Three quick things, in addition to more praise for your thoroughness and insight:
1. The ‘Top 100’ lists from prior to 2004 tended to have somewhere between 30 to 40 games in it (The list had 50 games and also included expansions which I removed for internal consistency to the current list.) From 04 and early 05 there would be around 70-80 games.
2. I can provide actual last date at peaks if that is useful… I don’t know if more precision on that metric is useful.
3. There are some other metrics that may be useful to use to add or qualify – most specifically # of weeks in the top 100. Firstly, it would lower the value of some of the weird outliers from early in the chart’s history and second it would cut, slightly, against newer games that haven’t had the time yet. I didn’t include it in the geek list, but it also can be provided… but these things are far more art than science and I just want to provide anything that you would believe to be helpful. As it is, I think it looks good.
Thanks, JonMichael. Wow, less than 40 games in the list prior to ’04! That’s less than I thought. It also probably explains why some earlier games didn’t achieve their peak until 2004, well after they had been released.
I can’t imagine that a game’s actual last peak date would be more than 6 months away from it’s calculated one. So the inaccuracy is 1 percentage point at worst. So thanks for the offer, but it’s probably not necessary.
Total weeks in the top 100 is an intriguing stat. I’m not quite sure what could be done with it, particularly compared to the simplicity of Peak Position, but I’m sure it has some value. Right now, like you, I’m pretty happy with the method.
Me again, JonMichael. So as I mentioned in my third article in the series, the data I’d most like to see is Peak Positions for games that only got to the top 200, or maybe top 150. I realize that that limits how far back we can go, but there’s at least some justification for saying that a game ranked #150 today is of comparable “quality” to one ranked #100 ten years ago. Do you know of any statistics that go beyond the top 100 for analysis? Thanks, and thanks again for all the work you put in for this.
I apologize, but I don’t have any of that data… it may be in the Wayback machine, but I’d be hard pressed to figure out how to find it there. Sorry.
Thanks for the response. It was a long shot, but you never know unless you ask.
Thanks for this article.
The problem I have is that unless I’m missing something, whatever game is currently #1 on BGG is always going to be ranked #1 on this list due to your penalty for a game being older.
Yup, that’s true. The rationale being that the current #1 achieved that position while facing the greatest amount of competition.
There’s lots of other methods that could be used. The number of weeks at the peak could be used as a tiebreaker–that’s what JonMichael does. Number of weeks in the top 100, another stat which he mentioned he tracks, is also a possibility. So there’s more than one way to skin this cat,
Great work and fun to read analysis. I’m not sure how I view some of the newer games that have such a high rating, but I do like being aware of what’s out there when I plan my next purchase. If you follow the NHL (I’m a big hockey fan) the debate that often comes up – and probably does in every sport – is who’s the greatest of all time? And then people will compare, not so much Gretzky/Crosby/McDavid, etc, but what the appropriate eras were like in the game (i.e. smaller goaltenders vs better training) so it becomes downright impossible to adequately compare players from so long ago to the players today. You just can’t do it without a bias or a flawed measuring stick. I think we may have a similar problem comparing games. But I still view El Grande as the Wayne Gretzky of board games.
Pingback: BoardGameGeek Top 100 - Wie wichtig ist diese für Brettspieler?