In brief

• Auckland Transport (AT) reduced speed limits in 2020, claiming a 39% reduction in road deaths.
• AT’s claim was based on a study by Abley.com.
• But there’s questions about AT’s decision to conduct a public advertising campaign based on unreliable data.

AT ads claim “smashing” success

In July 2020, to huge fanfare, speed limits on hundreds of km of Auckland roads were reduced to 80 km/h or lower. This past year, Auckland Transport has been crowing about the success in cutting road deaths and fatal crashes, inspiring the last government to roll out the programme nationwide. But just how good is the data? The answer may shock you.

AT used public money on a string of advertisements this year claiming a near 39% reduction in “deaths” on roads “where speed limits were changed”, otherwise referred to as “Phase 1” roads. Clicking the ads took viewers to some of the research behind the claim, and that’s where things get interesting.

Enter Abley.com

One study, by Christchurch-based consultancy Abley.com, used NZTA/Waka Kotahi data to extrapolate out some pretty impressive figures according to the executive summary:

“In the 24 months following the June 2020 Auckland speed limit reduction, Phase 1 roads have seen a 30% reduction in fatalities. In comparison, over this same period, the rest of the network has seen a 9% increase in fatalities…Overall, the difference between the control sites and the Phase 1 sites showed:

“[A] 38.8% reduction in fatalities compared to what would have been expected if no changes to speed limits was made.”

The study looked at crashes (more than one fatality may have happened in each crash) on the Phase 1 roads for the five years leading up to the June 2020 speed reductions, and reported an average of 6.2 fatal crashes a year prior to the change (implying 31 crashes overall), and an average of 4.5 fatal crashes per year in the two years after limits dropped (implying 9 fatal crashes in that time). That drop represents a 27% fall in fatal crashes.

A fatally flawed study on fatal crashes collides head-on with facts?

What the study didn’t do, however, was provide a breakdown of exactly how many fatal crashes had occurred each year of the study. When Abley.com kindly made that breakdown available, it raised big questions about the accuracy of AT’s publicly-funded advertising campaign.

The newly-released figures show huge variance in fatal crash numbers from year to year, eclipsing the statistical significance of the 27% drop in fatal crashes on the Phase 1 roads after lower speed limits came in.

For example, fatal crashes increased 125% between 2018 and 2019, fell 45% between 2019 and 2020, and (this is important) increased 100% between 2021 and 2022, AFTER speed limits were cut. 

The study claims to have factored in traffic volume during covid, but the doubling from three crashes in lockdown year 2021 to six fatals in 2022 suggests it’s too early to tell on such a tiny dataset.

AT tries to present it as careful analysis but it smells like politics to us

Just one glance at the Auckland Transport website and it is immediately clear that their focus on efficient traffic flow has expanded to include equity and the climate. It has all the hallmarks of using whatever bogus data you can find to justify a political leaning. 

To be fair to Abley.com, it did warn AT that the small sample size (fatal crashes don’t happen often enough to be statistically significant on the Phase 1 roads) meant that little reliance should be placed on the conclusions: “The number of fatal crashes in the before and after periods at the Phase 1 sites is small so limited confidence can be placed on the actual change in crash risk at these locations.”

Abley also confirmed that its study had no “statistical confidence interval”, which would be a guide to whether its findings were valid, because AT did not ask for that confirmation work to be carried out.

“All the analysis and the associated conclusions are included in the original report, that is the full extent of the work we were commissioned to undertake,” a spokesman said.

All of which begs the question: who at Auckland Transport ignored this glaring not-so-fine fine print and embarked on a public advertising campaign effectively creating fake news?

We are asking Auckland Transport for comment.