Political bias in digital media – findings from The Election Worm

A few months ago I set up thenewsaesthetic.com, a site to build small experiments in digital media. The first of these experiments was The Election Worm – a platform which captured all election stories from major digital media publishers in Australia, and then asked visitors to rate any bias in the headlines. The experiment got a bit of attention, and I thought it was worth noting down some of the learnings.

1. People have strong opinions about politics.

This one is a bit obvious, but still surprised me. I’m not a political person, and I was careful not to present this as an endorsement of any one party (or publisher) in particular. From the outset I had people claiming I was supporting one party of the other, including a few colourful emails. The next experiment on the site won’t have anything to do with politics.

2. There’s a lot of content created around the election.

Across the four weeks, 4,716 unique headlines were collected from News, Fairfax, ABC and The Guardian. If it takes three minutes to read each article, it would take you 10 days without a break to read them all. The cost of putting a new article online is essentially zero – there’s no page limit online and every new piece of content is another opportunity to monetise through advertising (unless you’re the ABC). This would be an interesting metric to track across future elections.

3. People struggle to differentiate digital publishing with newspapers.

Despite making it pretty clear this was about articles published online, the most consistent feedback I received was along the lines of “look at this Telegraph front page! How can you say they’re not biased?!?!” It seems that the cultural impact of physical newspapers is still considered to be greater than the impact from digital news sources, despite the online audience being higher than Monday-Friday readership for every publication except The Age.

4. Spammers come from both sides (and some just seem confused).

Spam votes were automatically removed. A spammer was defined as a single person voting for an improbably high percentage of biased articles vs. neutral articles. Spammers fell in to two groups – the first were people who just voted consistently for bias to one party (presumably to prove that the entire Australian digital media is biased); the second group voted for bias in both directions with almost no neutral votes (presumably these people didn’t understand the concept of bias given the high percentage of unquestionably neutral headlines). There were 43 total spammers, who took time out of their lives to vote 1190 times (only to have those votes removed). 220 more spam votes were cast for Liberal party bias, however it’s not worth reading too much into this given the top ALP bias spammer voted 93 times and the top Liberal bias spammer voted 221 times.

5. The reality is in the details.

Across more than 4,000 articles, all publishers were judged to have a left bias. It will take another election to understand whether bias in digital media always favours the incumbent, but the headlines that were most consistently voted left or right tell the story most people expected to see. Looking at headlines that were voted 100% left or right (and had at least 4 votes) – Fairfax and The Guardian made up 80% of the ALP bias, with News responsible for just two articles. When it came to the top 10 articles showing bias to the right, News was responsible for eight, and Fairfax only two articles.

Top 10 ALP Bias

Top 10 Liberal Bias

(note that the headlines were presented with their subhead – which I haven’t included above due to space, but also affected the ranking)

6. All publishers flock.

Everybody moved together – when Abbot made a gaffe, everyone notched a bit to the left. When Rudd stuffed up, everyone jumped to the right. Over the final weeks of the campaign, everyone trended back to the right – seemingly not wanting to be seen to be backing a loser.

7. Believe none of what you hear, and half of what you see. (And not a lot of what you read)

After the site got a bit of attention, the media jumped on it. The Australian, The Daily Telegraph, and Andrew Bolt all wrote about it. I was asked for interviews with John Laws and Adam Spencer. The Australian’s Nick Taberkoff was nice enough to point out this was an experiment, and certainly not academic. The Daily Telegraph and Andrew Bolt didn’t add in that caveat. The number of votes, voting mechanic, and fact this was an experiment, were all made clear. This isn’t academic research – it’s some code that was written on a Sunday afternoon, and to present it as canonical evidence of anything is questionable.

- September 2013