Political polls turn statistical uncertainty into media drama, forensic scientist warns

Summarised by Centrist

A poll of 1,000 voters can trigger headlines about political earthquakes, surging parties and collapsing leaders. Yet when more than 100,000 actual votes had been counted on election night, commentators warned it was too early to draw conclusions. 

Former forensic scientist Ruben Miller says the contradiction exposes a media industry turning statistical uncertainty into political drama.

Miller argues that political polling is often presented with far more certainty than the evidence justifies.

He points to the final Curia and Verian polls, which placed National slightly ahead of Labour. Both came close to the election-night results, but Labour ultimately led National by just 3,650 votes, or 0.15 percentage points.

Miller explains that the margin of error between two parties is considerably larger than the margin for either party individually. A half-point lead therefore cannot reliably establish which party is ahead.

He also questions headlines declaring political momentum following small movements between polls.

One August headline described Labour as slumping after a 3.7-point fall, even though Miller says such a change between surveys of around 1,000 people could arise through chance alone.

The contradiction became particularly apparent on election night.

Commentators cautioned against drawing conclusions from early results involving more than 100,000 votes. But Miller explains that representativeness, not simply sample size, determines reliability.

Early votes were not a random sample of the electorate, making them potentially misleading despite their numbers.

With nearly 400,000 special votes still outstanding when he wrote, Miller also cautioned against premature conclusions about the final composition of Parliament.

He does not dismiss polling itself, arguing that properly conducted surveys provide valuable estimates of public opinion.

Instead, he challenges journalists and commentators who transform those estimates into confident political narratives.

His central message is that polls measure uncertainty; they do not eliminate it.

As Miller puts it, “a measurement is not a prophecy”.

Read more at Stuff 

Receive our free newsletter here