Steam Holiday Sale generated over $340M in game sales--it’s a SWAG, baby!
Posted on Wednesday, February 5 2014 @ 02:15:39 PST
This member blog post was promoted to the GameRevolution homepage.
What was PC gaming worth in 2013? It depends on who you ask.
Ask DFC Intelligence, and they’ll estimate the 2013 PC gaming market at $21.4 billion (http://www.bloomberg.com/news/2013-11-04/valve-lines-up-console-partners-in-challenge-to-microsoft-sony.html). Ask Gartner, and you’ll be shaving a whopping $4 billion from that initial estimate—they peg 2013 PC gaming at $17.7 billion, crossing the $20 billion mark in 2014 and finally reaching parity with DFC’s 2013 estimate with a 2015 target of $21.6 billion.
The PC Gaming Alliance thinks Gartner is full of ****. According to their research, PC gaming already broke $20 billion back in 2012 (http://www.gamesindustry.biz/articles/2013-03-28-pc-software-hit-record-USD20b-revenue-in-2012), when it made up 38% of the gaming market. Gartner has PC gaming at $14.4 billion during the same timeframe with only half the market share.
So how can these research firms, with all of their proprietary information and rock-solid sales figures, be so far apart on historical estimates of the same market? Simple—they’re SWAGing.
A SWAG is a Scientific Wild-Ass Guess--a combination of rough partial data, approximated calculations, and general historical experience. Exact definitions vary, but a SWAG can generally be differentiated from a forecast or a financial report by the amount of assumptions and rules of thumb that are applied to patch up holes in the available data. Almost all proprietary third-party estimates are, at least in part, SWAGs.
Now, there’s nothing inherently wrong with a SWAG; a good one will get you within an order of magnitude of the truth and will be used for business decisions like market entry. The problem arises when firm XYZ spits out a number with three trailing decimal places and zero error, and then that amorphous specter, ‘the press’, picks it up as THE number without questioning methodology or motivation. The PC Gaming Alliance commissioned a gaming study and found out that all evidence pointed towards PC Gaming as the single most dominant market force? Gee.
When Gabe Newell says that Steam has over 75 million active users (http://www.joystiq.com/2014/01/15/steam-has-75-million-active-users-valve-announces-at-dev-days/), that’s a fact—he’s got one hundred percent of the data, and even then we could still argue over the definition of ‘active user’. But when IHS Screen Digest says Valve made $1.1 billion in revenue during 2012? They can call it “a deep understanding of market dynamics and technological developments [with] support from the most comprehensive, accurate and detailed database of global media and technology statistics available anywhere in the world”, but, at the heart of it… they’re SWAGing.
So, to understand just how much uncertainty can sit behind a seemingly solid number, let’s go through a simple SWAG together by estimating the total size of the 2013 Steam Winter Sale.
The 2013 Steam Winter Sale ran from December 19th to January 3rd and featured over 150 games deeply discounted, sometimes up to 80% off. For the first time, Valve also introduced a system to craft in-game items by participating in the sale and published the total count of items created. This public data will be the cornerstone of our SWAG.
As of about one hour prior to the close of the sale, 1,681,565 items had been crafted by Steam users. These items were crafted by creating Snow Globe badges, which were created by combining Snow Globe trading cards, which were earned for every $10 spent during the sale (see where this is going?). To make things even more complicated, there were both normal and foil trading cards whose badges created different amount of items, and there were trading cards that were created--but not combined into badges--hosted for sale on the Community Market.
So, first problem: How many total trading cards are there?
16,632,476 trading cards. If we felt like being simple, we could multiply by $10 and come to a sum of around $166 million dollars, which is, like, damn. But Valve, benevolent benefactor that it is, added multiple ways for a user to have created a trading card:
Somehow, we need to sample the population of Steam users and figure out how frequently members both crafted non-Snow Globe badges during the valid period and voted in the Community Choice sales.
A few fun facts about Steam:
- Steam Community can be accessed from a web browser, which means public user data can be web scraped.
- The majority of user profiles are public
- There is an official Steam Holiday Sale group with over one million members that we can sample from.
So, all we need to do is program a web scraper in Python, grab a random sample of 100,000 profiles from the Steam Holiday Sale group, attempt to scrape user profile data from each of them, and then aggregate the results. We can then generalize the data across the entire Steam population by weighting up by the number of Snow Globe badges earned.
So with all our assumptions laid out, we have 16,632,476 total trading cards, 3,014,667 of which came from badge crafting and 5,551,586 of which came from voting for the community choice sales. This leaves us with a grand total of 8,066,223 cards coming from game purchases, for a total of $80,662,228 spent.
But… If we ‘1 to 43’ the 75,000 users we scraped all that data from, we end up with just shy of 3.2 million users. Just after the 2013 Holiday Sale ended, Gabe Newell announced that Steam had broken 75 million users. If we’re attributing $80 million in sales to just 3.2 million users, how do we account for the behavior of the other 71.8 million?
One way would be to true-up with straight line multiplication, making Valve’s 16 day sale worth approximately $80 * 75 / 3.2 = ~$1.9 billion dollars. That estimate seems… ambitious.
Here’s where we can apply some industry experience. In most mass consumer businesses, customer contribution follows a negative exponential function, meaning that the ‘best’ customers contribute a disproportionately large amount of value relative to the majority. The general form of a negative exponential function is as follows:
Don’t believe that happens in real life? Take a look at the game ownership data we scraped from those 75,000 Steam users:
$340 million. Woo! Done! Or not.
Now, we have to digest all the gristle we ground into that sausage of a number. Think of this as a sanity check: What does $340 million actually imply?
So, what do you think—is $340 million a reasonable revenue number to attribute to the 2013 Steam Winter Sale period? Even when the entire analysis hinges on a single public statistic, 76.3% of the revenue is based on one assumption, and the methodology we used would produce a lower estimate if we scraped a larger user population? How far off are we? One million dollars? Ten million? One hundred?
In fairness to professional analysts, most published estimates aren’t quite so haphazard. Sizing a market generally starts with a summation of all available public revenue, which grounds the final number in more or less reality, depending on the industry. For example, Sony, Microsoft, and Nintendo are all public companies, so sizing the total value of the console market for hardware producers should be relatively consistent outside of future growth assumptions.
Even for our analysis of Steam, there were more accurate methods we could have used to estimate total Winter Sale revenue—imagine if we had scraped all the games owned by our 100k users at the very start of the sale, scraped the same users again at the end, and then looked at what new games those users had gained multiplied out by the sale prices and total active users. We’d probably get a bit closer to the truth. Maybe.
But at the end of the day, we’d still be SWAGing.