How gamification ruined your Next Fest stats (and why it doesn’t matter)

[The GameDiscoverCo game discovery newsletter is written by ‘how people find your game’ expert & company founder Simon Carless, and is a regular look at how people discover and buy video games in the 2020s.]

Good morning, evening, or whatever the time zone happens to be, wherever you are in the world. It’s time for us to return for another week of handy game discovery news and analysis – and thanks for coming along for the ride.

This time out, we start with an expansion on last Wednesday’s newsletter. Are players really hitting your Steam Next Fest demos to the tune of tens of thousands of CCUs? Are your wishlists numbers similarly affected? There’s bad news, and good news…

Gamification, Next Fest stats & signal vs. noise…

So we mentioned last Wednesday that “if you’re seeing [Steam] game demos get unlikely CCU numbers – such as Terra Nil, which briefly hit 95,000 CCUs, this is likely due to ‘badge farming’.” After that, we ended up being contacted by a few other regular Steam devs who were confused about their Next Fest demo stats.

We thought it was worth a quick ‘explainer’, then. How’s the ‘farming’ taking place, what effects might it have had on your Next Fest player stats, and why is this more of a temporary confusion factor rather than a long-term problem? Here’s what we know:

• Valve helped to incentivize Next Fest this time out with some badges: “Play any demo from Next Fest to unlock a badge you can feature on your Steam Profile.
  Level up your badge as you try out more demos.” Although the XP maxes out after you play 10 demos, you can see total demos played on your Steam profile – see this example with 1056 demos played (!). So some people are grinding, for fun.
• Rather than playing, Steam users are using third party apps like Archi’s Steam Farm to level up automatically. As the FAQ notes: “It doesn’t have to download any of your game files, it can play games right away. Secondly, it’s entirely independent of your normal Steam client – you don’t need to have Steam client running or even installed at all.” So it’s passing ‘I’m playing this game!’ variables to Steam…
• Most of the time, you won’t obviously encounter ASF-related traffic on your regularly released, paid game. (It checks games that you’ve already bought for Steam trading card ‘drops’, and then idles until all the cards have dropped. So it might increase average playing time for your games, if you have trading cards – but that’s it.) However, in this case, individual Next Fest demos may see stats distortion.

Anyhow, here’s a useful real-world example, from Pavonis/Hooded Horse’s much awaited 4X title Terra Invicta. The game has 34,000 Steam followers – so hundreds of thousands of legit wishlists. Its Next Fest performance was top echelon, since it was #3 in the ‘Most Wishlisted’ chart. But its back-end demo stats are a little, uhh, screwy:

Do we really believe that nearly 250,000 of the 489,000 unique Next Fest demo players for Terra Invicta decided to play the game for less than 10 minutes? No, we don’t – that’s probably due to ASF & similar apps swiftly telling Steam your demo was loaded.

Hooded Horse’s CEO Tim Bender and myself were tipped that the odd spike at 3 hours is related to Steam ‘idling’ apps at work. Sometimes these apps are set as default to show you playing the game for longer – even if, as in this case, nothing happens.

But there was definitely an enthusiastic player base checking out the Terra Invicta demo, as can be seen from YouTube results. So we’re inclined to believe the 58,000 players who tried the game for more than 200 minutes – and all the in-between durational stats, actually. But this is data cherry-picking of a subjective kind.

Another example comes from Hellscreen’s Jamie D, who told us that 43% of his ‘downloads’ – and 80% of his ‘players’ for his game – were from the Netherlands. These are clearly not real players at that volume. (It’s perhaps due to a popular free VPN being there.) Unfortunately, not all ‘farmed’ plays of demos can be tied to a particular country and excluded, if we look at country data across many demos.

Anyway, the important message here is: don’t worry if your average Next Fest demo play time seems low, or suddenly dived. But also, definitely don’t get excited that you have lots more demo players than expected. Steam’s gamification has inadvertently led to stat-hungry players ‘playing’ every Fest demo – without actually loading them.

The good news here is that your game’s Steam wishlists are unaffected by these idling programs. Some Next Fest wishlists may be marginally more speculative than ‘organic’ ones. But there are no idling programs that automatically wishlist games as a matter of course. So believe your wishlists, and keep on keeping on!

[We’re GameDiscoverCo, an agency based around one simple issue: how do players find, buy and enjoy your premium PC or console game? We run the newsletter you’re reading, and provide consulting services for publishers, funds, and other smart game industry folks.]