In our development of our Western Australian forecast, one thing I noticed was that state voting-intention polls (state polls from here on out) of WA tend to be quite favourable for the party in power. On a two-party-preferred (2pp) basis, the incumbent government in WA has tended to perform 1.8% worse in their election-day result as compared to their final pre-election polling:
Western Australian state governments have tended to underperform their polling
(if you’re on a mobile device, scroll right for full data or turn your device landscape)
| Election | Incumbent | Polling avg. | Result | Error |
|---|---|---|---|---|
| 1986 | Labor | 56% | 54.1% | -1.9% |
| 1989 | Labor | 50.5% | 47.6% | -2.9% |
| 1993 | Labor | 44.4% | 44.5% | 0.1% |
| 1996 | Coalition | 56.5% | 55.2% | -1.3% |
| 2001 | Coalition | 50% | 47.1% | -2.9% |
| 2005 | Labor | 54% | 52.3% | -1.7% |
| 2008 | Labor | 50% | 48.1% | -1.9% |
| 2013 | Liberal | 59.5% | 57.3% | -2.2% |
| 2017 | Liberal | 46% | 44.5% | -1.5% |
| Average | -1.8% |
Polling averages and results refer to the incumbent government’s 2-party-preferred. Italicised polling averages refer to those where the pollsters did not produce their own 2pp estimate; the estimate listed here was calculated using the pollsters’ published first-preference vote and last-election preference flows. Credits to Dr Kevin Bonham for providing us with his historical polling data.
Going back to 1986 (the first election we can find state voting-intention polls for WA), the incumbent has only outperformed their polling average on one election (1993). With just 9 state elections in our sample, might that be an artifact of the small sample size and noisy data?
Most state governments underperform their polling
(if you’re on a mobile device, scroll right for full data or turn your device landscape)
| Election | State | Incumbent | Polling avg. | Result | Error |
|---|---|---|---|---|---|
| 1986 | WA | Labor | 56% | 54.1% | -1.9% |
| 1989 | WA | Labor | 50.5% | 47.6% | -2.9% |
| 1992 | QLD | Labor | 55% | 53.7% | -1.3% |
| 1993 | WA | Labor | 44.4% | 44.5% | 0.1% |
| 1995 | QLD | Labor | 51.5% | 46.7% | -4.8% |
| 1996 | WA | Coalition | 56.5% | 55.2% | -1.3% |
| 2001 | WA | Coalition | 50% | 47.1% | -2.9% |
| 2004 | QLD | Labor | 59% | 55.5% | -3.5% |
| 2005 | WA | Labor | 54% | 52.3% | -1.7% |
| 2006 | VIC | Labor | 54.7% | 54.4% | -0.3% |
| 2006 | QLD | Labor | 54% | 55% | 1% |
| 2006 | SA | Labor | 59% | 56.8% | -2.2% |
| 2007 | NSW | Labor | 55.2% | 52.3% | -2.9% |
| 2008 | WA | Labor | 50% | 48.2% | -1.8% |
| 2009 | QLD | Labor | 49.9% | 50.9% | 1% |
| 2010 | SA | Labor | 48% | 48.4% | 0.4% |
| 2010 | VIC | Labor | 48.9% | 48.4% | -0.5% |
| 2011 | NSW | Labor | 35.9% | 35.8% | -0.1% |
| 2012 | QLD | Labor | 39.1% | 37.2% | -1.9% |
| 2013 | WA | Liberal | 59.5% | 57.3% | -2.2% |
| 2014 | SA | Labor | 47.7% | 47% | -0.7% |
| 2014 | VIC | Coalition | 48.4% | 48% | -0.4% |
| 2015 | NSW | Coalition | 55.1% | 54.3% | -0.8% |
| 2015 | QLD | LNP | 50.1% | 48.9% | -1.2% |
| 2017 | QLD | Labor | 51.8% | 51.3% | -0.5% |
| 2017 | WA | Liberal | 46% | 44.5% | -1.5% |
| 2018 | SA | Labor | 50.8% | 48.1% | -2.7% |
| 2018 | VIC | Labor | 54.1% | 57.3% | 3.2% |
| 2019 | NSW | Coalition | 50.5% | 52% | 1.5% |
| 2020 | QLD | Labor | 51.5% | 53.2% | 1.7% |
| Average | -1% |
Polling averages and results refer to the incumbent government’s 2-party-preferred. Italicised polling averages refer to those where the pollsters did not produce their own 2pp estimate; the estimate listed here was calculated using the pollsters’ published first-preference vote and last-election preference flows. Tasmania was excluded as they use a proportional system which doesn’t produce 2pp figures. Credits to Dr Kevin Bonham for providing us with his historical polling data.
On average, state governments in states with single-winner lower houses have underperformed their polling by about 1%, doing so in just over two-thirds of state elections (23/30). This pattern is something which is definitely statistically significant (p < 0.005, using a one-sample t-test); meaning that it’s very unlikely we could have gotten results like these if polls truly did not over/under-estimate the incumbent government’s 2pp.
So, case closed, right? We should adjust the polls we get for this tendency in our state election forecast, and reduce Labor’s chance of winning in WA by some fraction of a percentage point? Labor is currently polling at 60+% of the 2pp at time of writing. Losing a percentage point off that isn’t going to hurt their chances of winning all that much.
Well, looking back at the table, it seems that polling has actually underestimated the incumbent in the three most recent state elections (2018 VIC, 2019 NSW and 2020 QLD). In fact, it does seem like the skew to incumbents has been gradually decreasing over time, which is borne out in a simple graph:
While some of the correlation seems to be driven by the VIC 2018 outlier, it seems relatively clear that the polling skew to incumbents has been declining over time. In particular, I would note that the most recent pandemic general election in Australia (QLD 2020) saw the state’s Labor government overperform its polled 2pp by 1.5%. Furthermore, across the Tasman Sea, New Zealand’s Labor government (which went to re-election in a very similar position to where WA Labor is in today) A centre-left government which is credited with having successfully contained the coronavirus and kept their state in fairly good shape throughout the pandemic and ensuing recession.
As a result, they go to re-election with some of the largest leads ever seen in opinion polling, causing concern among the opposition that they might win a majority in a proportionally-represented legislature for the first time since the introduction of proportional representation. outperformed their final polls by about 4%.
A further pattern in the polling errors is that the skew to incumbents tends to be slightly larger when an incumbent is ahead in the polling average:
However, the relationship here is not statistically significant (p = 0.314), meaning that this might well be an artifact of how few elections we have where the incumbent was behind in the final pre-election polling average (9). In particular, we have no precedent for the polling lead which WA Labor has amassed (> 10%) at time of writing, meaning that we have no idea if there will be any relationship between the size of their polling lead and possible polling skew to/against the incumbent.
I can think of a few hypotheses for a skew in either direction (warning: just hypotheses, not necessarily true):
- Some voters like having checks and balances on the government of the day, and will vote for the other side if they sense that one party is going to win big. This isn’t impossible; note how Labor’s 2pp declined in the final weeks of the 2007 campaign (even though not much had changed), or how (at time of writing) the most Labor-leaning and 3rd most Labor-leaning states (Tas and SA) both have Liberal state governments,
(I bring up Tas and SA because both seem to fly in the face of the federal drag theory)
If this is the case, WA Labor will almost certainly lose a few percentage points off its vote on election day. However, this point doesn’t explain why recent state polling has become less likely to favour the incumbent; not to mention the polling error with the biggest skew against the incumbent is in VIC 2018, where the state Labor government was tipped for a hefty win anyway. - Some issue with the representativeness of polling samples has been fixed/resolved/reversed itself. There’s really not much evidence to prove or disprove this, as we don’t have enough online vs robocall vs in-person phone polls to be certain of how the shifts in methodology have affected incumbent skew (especially given how little transparency there is in Australian polling). If it is correct, WA Labor might not see any drop-off on election day, and might even make a few gains (similar to how the LNP overperformed their polling against QLD Labor in 2012 by about 1 ~ 2% even though QLD Labor was already down by > 10%)
- The pandemic might make voters less willing to take a chance on a new government. This is a very specific explanation that only explains very recent incumbency skews (QLD 2020, NZ 2020); however, it does seem like incumbent governments are doing fairly well for themselves across the globe (full analysis coming later as part of our Pandemic Politics series) even on top of the polling surges they’ve gotten. If this is the case, WA Labor is likely to do about as well as their polling suggests, or perhaps even better.
Bringing this back to our WA state election forecast:
Given the fact that we don’t really know which of these hypotheses is most applicable to the 2021 WA state election, in developing the 2pp vote model, I opted to use a slightly different prior in our Bayesian updating process.
In our updating process, we use data collected from as wide a range as possible (e.g. all state polling) to form what is known as the prior, or the “default” belief before new evidence is added into the mix. We then update this prior based on more evidence (for example, updating a prior about how skewed all state polling is to incumbents with polls specifically from WA) to get a posterior, or what our expectations should be with the new evidence included in.
Normally, our prior for how skewed state polls are would be calculated as a simple average of all state poll skews in our database. However, to reflect the uncertainty
No two polls are ever exactly the same; many pollsters use different methods of sampling voters or of aggregating the results. Usually, when polls stuff up in a really big way, pollsters will change their methods to try and avoid a similar error; these changes can sometimes fail to correct the issue (polls underestimating Democrats again in 2020), or sometimes they overcorrect and stuff up in the opposite direction (polls overestimating the Conservatives in the UK 2017 election after they corrected for underestimating them in the last election).
Pollsters aren’t static – usually when they realise their polls are skewed to one side or another they will apply corrections to try and fix the issue. Given how little information comes out about their methods, we don’t know if they will try to avoid overestimating the incumbent again this time and hence we should be cautious about building in house effects in our model.
as well as the statistically significant decline in incumbent skew (p = 0.01) I have opted to use a weighted average of state poll skews to calculate the prior instead, in which more weight is given to the polling skews of more recent elections.
This produces a prior of +0.3% skew to incumbents instead of +1.0%; which means that after the Bayesian updating process we expect polls to only be skewed to the incumbent by +0.6% instead of +1.4%.
Obviously, if Labor massively underperforms its polling on election day, this decision will look rather silly, but I do think that it’s the right decision when considering current trends and the uncertainty in the electoral environment. Only time (and the voters of WA) will tell.