Numbers can lie: the science behind opinion polls

by Michael Tanner | Apr 30, 2021 | Business

Results from opinion polls are regularly accorded more weight than they deserve, particularly given that their methodology is not transparent; their language can shape answers; and only a small percentage of those contacted respond. Yet polls may also independently shape voting choices. Michael Tanner reports.

Recent months have seen a peculiar, yet recurring, trend. Revelations of a scandal or an out-of-touch comment from the federal government causes significant outrage among much of the population, particularly on social media. The government seems on the ropes. Then the polling numbers come out, showing the opposite. Then out come the commentators, proclaiming that people on social media – particularly the “Twitteratti” – are out of touch.

And so it is with the latest Newspoll, which shows Morrison still the preferred Prime Minister, and more “likeable” and “caring” than Anthony Albanese; meanwhile, the two-party preferred numbers remain neck and neck.

These events might seem to occur independently. They don’t. Polling – far from being an independent snapshot of the public’s mood at a given time – has far-reaching consequences that are seldom acknowledged.

Recent elections have highlighted the limitations of polling. Pollsters got it wrong in the 2016 elections in Britain and the United States and Australia’s 2019 federal election. Even though Joe Biden came out on top in the 2020 US election, there was a significant discrepancy between the final outcomes and the polls.

Tightly guarded secret

To view polls as scientific is unwise. The biggest influence in any scientific experiment is the make-up of the study groups, but few polling companies reveal how they find their target audience.

Another influence is the phrasing of questions and answers. According to Pew Research, a US-based polling company that researches polling methodology, “the choice of options provided, how each option is described, the number of response options offers and the order in which options are read” all have an effect.

In telephone surveys, respondents more commonly choose answers that are heard later in a list. More people are typically in favour of government support when it is phrased as “assistance to the poor” rather than for “welfare”, owing to the different connotations attaching to the language.

Few people stay on the phone when a pollster calls. Pew Research estimates that only between 5-15% of  those contacted respond to the poll. This means polls are not representative of the general population; those who do stay on the line are unlikely to reflect the wider electorate. They may have a bone to pick with the government or be a passionate supporter, for example.

And even among those who do, the number of people sampled is too small to make too many reliable conclusions. ResolveStrategic, for example, regularly includes 2,000 people in its sample polls for The Sydney Morning Herald and The Age; this number reflects 0.01% of the electorate.

The bandwagon effect

The real power of polling comes when the results are disseminated. While just 0.01% of voters may be polled, the results will be shared with hundreds of thousands or millions of Australians – a significant proportion of the electorate.

Elections are won and lost by undecided voters, and polling results can have a marked influence on future voting intentions, thanks to human psychology.

At play is the “bandwagon effect”, which is the “phenomenon of public opinion impinging upon itself”, where those who believe a party will win are more likely to vote for that party in order to be on the “winning” side.

The bandwagon effects can be substantial. An experiment to simulate the influence of polling on voter behaviour suggested that seeing pre-election polls could increase by seven percentage points the number of people voting for the leading group.

The authors also note that they studied highly charged issues – immigration, families, the environment, guns – that typically reflect tightly held beliefs, and that are less likely to change in response to a poll result. For less emotive policies, such as the economy or health, the magnitude of the effect would likely increase.

According to polls, 24% of voters went into the 2019 federal election undecided as to who they would vote for. Some 7% of these voters is a substantial number.

Media coverage affects moderate voters

How do we square up the effect of polling with the fact that Labor, in 2019, was polling better than the Coalition yet still lost the election?

Another major driver of voting behaviour for those who are undecided is media coverage.

International research highlights the influence of media coverage on voting intentions. Unsurprisingly, positive coverage increases electoral support for a political party. Media endorsements of a candidate is particularly effective on moderate voters – those most likely to enter an election campaign undecided. The discrepancy between polling and outcomes in the 2019 election can be explained by the influence of the media.

Both media coverage and the coverage of polling affect voter decisions. Things may get murky when the media and the pollsters are both aligned.

My weekly Tamed Estate column reveals numerous examples of where media outlets appear to have toed the government line and that of business interests. Less explored is the links between the polling companies, the Liberal party, and the media companies.

Tamed Estate: media kicks off 2021 copying and pasting government “drops”

The Age and The Sydney Morning Herald recently published a series of articles detailing voter attitudes that resulted from a poll conducted by ResolveStrategic. While the articles mentioned the head of ResolveStrategic was once the director of research and strategy at CT Group (formerly Crosby Textor) and that his work had included contracts with the federal government, the links go much deeper.

CT group is a lobbying firm and political consultancy with deep links to the Liberal party, being involved in the Liberals’ 2010 and 2013 federal election campaigns, as well as the 2012 Liberal National Party state election campaign in Queensland. Lynton Crosby, co-founder of CT, was the federal director of the LNP in 1997, and oversaw the LNP campaigns in the 1996, 1998, 2001 and 2004 federal elections. In 2002 he was described as “one of the most powerful and influential figures in the nation”.

The Australian relies heavily on NewsPoll results, which it exclusively publishes. Newspoll was established in 1985 in part by News Limited, the holding company of News Corp Australia. In 2015, administration of Newspoll was transferred to Galaxy Research, which was later acquired by YouGov.

Poll results are given more weight than they deserve, particularly given the opacity of the methodology. Yet they may independently act as a tool to shape voting preferences.

A Pushy Number: Libs’ pollster Crosby Textor granted access to 27 million unlisted mobiles

Michael Tanner is completing a Doctor of Medicine/Doctor of Philosophy. His writing explores the intersection of economics, the media and public health. His writing has also been published in The Age.

Don't pay so you can read it.

Pay so everyone can.

Pin It on Pinterest

Share This