A recent article (opens new window) from Radio NZ did a great job of pointing out just how useless online polls are, and raising concerns about how often New Zealand media outlets, including Newshub, the AM Show and the Herald, rely on them as source material for news articles.

A Newshub poll before the election had National pegged to win, and a Herald poll said kiwis backed the Queen in her spat with Meghan and Harry.

One egregious example was from the Wairarapa Times-Age, which recently wrote an article claiming that locals weren’t keen to receive the COVID vaccine, based on 1,500 votes submitted to a Facebook poll. It’s not hard to spot why this is a problem. Although 1,500 people sounds like a good number, it’s only around 3% of the 50,000 people who live in the region. And, of course, there’s no guarantee that any of the people who responded to the poll actually live in the Wairarapa. Given how often issues based groups "game" these kinds of polls, asking all of their members from around the country, or even around the world, to vote to skew polls like this, it’s unbelievable that any journalist would then choose to use the numbers from a poll like this to write an article. And this example is even worse, given the damage that this particular article is likely to cause, scaring people into not being vaccinated.

Thomas Lumley from StatsChat said in the article that, even if people are aware that informal polls should be taken with a large grain of salt, they’re still likely to be left with some kind of misinformed impression from the numbers they see. And when it comes to these straw polls, the results can vary wildly. He gives the example of three polls asking whether John Key or David Cunliffe won a debate back in 2014. The three different news outlets’ polls showed that John Key won in the minds of 36, 48 and 61% of people - a variance that’s so large as to be useless, and obviously hugely biased due to the different political make-up of the readers of news outlets with different political leanings.

Henry Cooke from Stuff, and Radio NZ’s Mediawatch, both created tongue in cheek online polls asking whether the Media Council should ban journalists from ever reporting anything based on non-scientific polls, and the results didn’t disappoint. In both of them, over 80% of respondents agreed with the idea - which is not to say that the Media Council should, or could, create a rule like this. But it does show how, when you have some control over what and who you’re asking poll questions to, it’s fairly easy to get any answer you want.