Let’s Talk About: The GDC Platform Development Survey

Brothers and Sisters, welcome back to Let Us Play!

If you’ve been watching the news within the gaming sphere today, you may have already encountered this story, but I think it’s important to talk about it and attempt to put a cap on this sort of sensationalist reporting, and set the record straight before the rumours surrounding it become pernicious. This morning (Australian time), some information, seemingly devastating statistics for Nintendo, was posted to GameRant. This supposedly new information, was fresh out of Games Developers Convention 2017, which ran from Februaury 27th to March 3rd. The chart that was released that had been drawn up out of a survey of over 4500 attendees of GDC 2017, which showed what percentage of participants were developing games for which consoles, with only 3% ( around 135 out of 4500) saying that they were developing for the Big N.

Rory Young of GameRant said:

10 different platforms were included on the survey list, with mobile platforms like iOS and Android being lumped into one Smartphone/Tablet category and VR headsets being lumped together similarly. Out of those 10 platforms or categories, the Nintendo Switch was ranked very last.

This chart is the one that they are referring to and the one that GameRant have in their article. However this isn’t the whole story.

Through a lack of investigation, GameRant has reposted a biased chart, and it has subsequently exploded, being shared around by Nintendo enthusiast sites and neysayers alike, some of which acting as though it’s some kind of death knell for Nintendo’s new baby, and the company as a whole.

Word of advice; as two million sales will tell you, this tired meme couldn’t be further from the truth.

Yes, the chart does indeed show only 3% of the 4500+ developers, working on games for the Nintendo Switch, dead last in the runnings. However, you can’t treat a chart like this, as though it’s some statistical island. When it comes to interpreting the data, there is no given context on which to base ones understanding of what is being seen, and without context even seemingly cut and dry statistics are worthless.

All we know that 3% of 4500+ devs are working on a game for the Nintendo Switch; the three big contextual questions that have to be answered are:

  1. When was this survey taken?
  2. Who was taking/administering the survey?
  3. Is there possibility of bias and/or false data?

Firstly, I think that it’s important to point out this chart is far from new. Ok, this particular chart above is pretty new, it was released on March 15th, through the statistical journalism websites Statista, not from GDC themselves. There was also a small accompanying article written by staff writer Felix Richter. This is important because it shows that Richter and Statista are a node along the informational highway, a point in which information can be manipulated, and biases and agenda inserted, and the biases are pretty clear.

Here is the original chart, as posted by GDC themselves, back on January 15th 2017:

As you can plainly see, this chart is titled “Which platform(s) are you *currently developing* games for? (Choose all that apply)”

It is Richter who has renamed his chart to imply “importance” of a console something that the original chart was not trying to establish, but will have an subconscious effect on how people view the Nintendo Switch, after reading the article.

It is also Richter and his own personal biases which has resulted in his chart stopping with the Nintendo Switch in last place, when the original chart included twice the number of entries, half of which are below the Switch. Sure it doesn’t look good on the original chart for the Switch either, being down in tenth place, but it’s not like the platforms on par, or below the Switch are new or unknown devices, including current generation handhelds like the 3DS and Playstation Vita.

Richter also writes in the accompanying article:

Even though Nintendo’s new gaming console Switch appears to be off to a great start, it still lacks where it matters most: in terms of game selection. At launch, Nintendo fans had just five games to choose from, including one blockbuster title in the new Zelda game.

If he is only referring to physical releases, and only in the North American region, then sure. However that’s not the whole story. Digital release games such as The Shovel Knight: Treasure Trove with timed exclusive access to the Spectre of Torment chapter, I Am Setsuna, FAST RMX, and unexpected gem Snipperclips were also available on the eShop from day one in the North American region. In the PAL regions we also received the NeoGeo ports (virtual console?) of King of Fighters ’98, Metal Slug and Shock Troopers, amongst others. Finally, in Japan another six games were released including PuyoPuyo Tetris, Disgaea 5 Complete, Dragon Quest Heroes I & II and Blaster Master Zero.

Across all regions, digital and physical releases, over twenty games were released on launch with the Nintendo Switch. Some people my say this is me nit-picking and fair enough, I can understand why you might think that, but the fact remains that North America isn’t the be all and end all of games releases, and in this day and age, neither are physical games. These are things to be considered when posting to the WORLD WIDE web. On day one, I personally purchased more games, than Richter claims were even available.

Richter’s statement is just wrong, and suggests a conscious bias or an ignorance of the subject, neither of which are desirable traits in a journalist. Taking into consideration the decision to cut off his chart to portray the Switch as being the least “important” platform, I would personally argue the former, but that’s just my opinion. Take from it what you will.

Next up, when was this survey even taken?

When searching for hits on the GDC Survey for the 2017 event, I first started seeing people talking on Twitter about the survey back in November 2016, with people commenting on the survey, their answers and early impressions from the incomplete results. This is one of the most important pieces of context for why Nintendo Switch may be ranked so low.

This survey was taken between the First Look trailer from October 20th, 2016, and the Official Switch Presentation on January 13th, 2017. This was an uncomfortable period for developers working on the Nintendo Switch, to say the least. Even big name companies like Bethesda and 2K were not allowed to confirm their involvement with the Switch, despite their games Skyrim and NBA2k17 respectively, being featured in the First Look trailer. Square Enix were asked to retract a statement concerning Dragon Quest XI earlier in the year, and lots of other games have been kept quiet for a very long time. Even the number of development kits that have been sent out, has been a secret. Nintendo did however announce that there were over eighty games coming to the Nintendo Switch this year, and on February 28th, Nintendo revealed that sixty of those were indie titles.

Assuming that Nintendo didn’t outright tell developers not to answer this question, an unlikely scenario to be sure, it’s more than possible that due to the strict non-disclosure agreements that these studios have with Nintendo, that these developers didn’t know whether or not it would be a good idea to answer this question, anonymous or not. Simply, they didn’t want to incur the ire of Nintendo for saying the wrong thing.

If this survey were taken at the end of 2017, with the same results then we’d have a problem, but knowing the sort of nonsense that Nintendo were pulling with their borderline abusive silent treatment, I’m really not surprised that the number is so low. In fact the GDC’s write up concerning their original chart may reinforce this hypothesis:

Nintendo’s new Switch console is also expected to debut this year, so it’s interesting that 3 percent of respondents said they’re currently working on a game that will release on Switch and 5 percent said they expect their next game will be released on Switch.

Clearly there is a disconnect between those two questions, and it begs the question, how far does that actually go?

Furthermore, the above tweet brings up the question of bias again, with the claims of a heavy emphasis on VR. Looking at the write up, I would agree that a lot of the content is referring to VR, and a good number of survey questions focus exclusively on VR and AR development and devices. In a pamphlet that is only fourteen pages long, three of which are covers and introduction, a few pages are taken up by charts, a whopping four and a half pages are taken up by talk of VR/AR, exclusively. That’s a lot, considering the remaining pages are a mish-mash of everything else.

It seems quite clear what GDC were focusing on in their survey this year.

While it won’t account for all the percentage points on on that chart it makes me wonder, if this bias towards VR/AR developers affected the results. Interestingly, PC, PS4/Pro, Xbox One/Scorpio and VR devices are all at the top of the list, with the inclusion of mobile app developers which I would be surprised to find any lower. All of the top platforms are VR capable, including some of the smartphones. Could a high rate of VR developing respondents, contribute to the low score of the Switch considering it’s not VR capable?

There’s no way to know for sure, but it’s a possibility. One of many I’m sure.

And Finally…

I want to end this whole schpiel by saying, way before we even knew for certain that another console was in development, people have been declaring the Switch, or the NX as it was then called, the last console of Nintendo. We only got information about the hardware of the Switch in the past few months, but even so, well over a year ago, people were posting articles about how the Switch IS, not could be, IS underpowered, and by that virtue alone, will cause the end of Nintendo. There has always been, and always will be some Nintendo naysayers, who hate the company for their own reasons, and want to see them fail. It’s these people who are willing to believe and regurgitate any information that seems to support their negative opinions, without stopping to rationalise what they are seeing.

With that in mind the chart doesn’t even need to be true at all, because we, as humans, have a negativity bias. This applies to our personal negative experiences outweighing our positive ones in general, as well as the ability to retain negative information better that positive. We are more likely to step up and complain, or give scathing reviews if we think something is bad, and find it difficult to explain why things are good when they work exactly as they are supposed to.

On top of that, it seems to be the way of things that with enough repetition anything can become a “fact”, people will believe this chart 100% true and accurate, because lots of websites are repeating it. In actuality, it’s always a good idea to take stuff like this with a huge spoonful of salt, and then check the sources. Especially all you “journalists” who have repeated this story verbatim, without looking deeper into it. Shame on you!

Be smart consumers of not just your products and purchases, but information too. Trust, but verify!

Peace and High Scores!

  • Kyle Yager

    Spiel with an ch? When did that become a thi- I mean, yeah, I’ve been seeing a lot of the whole “latching onto anything that furthers the narrative you believe” business lately, in gaming and other situations too.

    • Absolutely, it’s everywhere. Confirmation bias is one of the most intellectually dishonest ways to share news and information, but probably one of the most prevalent.