Local elections Projected National Share (PNS) of the vote 2022

By Stephen Fisher, 6th May 2022.

The BBC Projected National Share (PNS) of the local election vote 2022 is Con 30, Lab 35, LD 19, Others 16.

There is an explainer of the methodology at https://electionsetc.com/2022/05/04/understanding-the-local-elections-projected-national-share-pns-in-2022/.

Historically, party performance in local elections has followed a similar pattern of change over time to the general-election vote-intention opinion polls, as shown the graph below.

Changes in the PNS this year are broadly in line with changes in the polls relative to both 2018 and 2021. The Conservatives are down 7 in the polls since both 2018 and 2021, and down 5 and 6 points respectively in the PNS. Labour are at the same level in the polls and PNS as they were in 2018, but up 4 points in the polls and 6 points in the PNS since 2021. 

In both the polls and the PNS the two parties were tied in 2018. After Boris Johnson became PM the Conservatives achieved a lead that won them the 2019 general election and lasted through to 2021. Following partygate and various other controversies, that lead has been reversed. This week’s local elections essentially confirmed the message from the polls that Labour are now ahead.

In recent years the Liberal Democrats have revived their tendency to do much better in local elections than they do in general election vote intention polls. That pattern was established in the 1980s with the Liberal Alliance, but ended after the Lib Dems joined the coalition in 2010. This year the party continued its post-coalition revival. They are up 2 or 3 points relative to both 2018 and 2021, in both the PNS and the polls.

Its not such a consistent pattern with respect to other baselines.

Indications from the local elections for the next general election?

The Projected House of Commons’ seats from the PNS (with changes from the 2019 general election) is

Con 253 (-112)

Lab 291 (+88)

LD 31 (+20)

Others 75 (+4)

The Projected House of Commons takes into account differences in local and general election voting on recent occasions when the two kinds of election have been on the same day.

Continue reading Local elections Projected National Share (PNS) of the vote 2022

Understanding the Local Elections Projected National Share (PNS) in 2022

by John Curtice and Stephen Fisher, 4th May 2022.

Much of the speculation about what might happen in the English local elections tomorrow has focused on how many seats each party could or should gain or lose. Indeed, we can expect many a judgement to be cast on Friday on the basis of this evidence. However, given that these elections are held under first past the post (and in London multi-member plurality), seats won and lost can be a poor guide to how well a party has done in the ballot box. A party whose vote has fallen less than that of their principal rivals may gain seats even though it has lost votes. A third party whose vote is geographically spread may make a substantial advance in votes yet reap little reward in terms of seats. Meanwhile, even if these issues do not arise, seats won and lost only provide an indication of whether a party has lost or gained ground as compared with when the seats up for grabs were last contested – which this year, as is usually the case, was four years ago.

Yet we cannot simply add up votes cast either (even if we had the resource to collect them all on election night). In England (unlike Scotland and Wales) it is never the case that the whole country votes in local elections at the same time. The places that vote one year are politically different from those that vote in another. Given all these limitations, a key indicator of party performance that has come to be part of the ritual of local election night is the calculation of a ‘projected national share’ (PNS). This is an estimate of the share of the vote that the principal parties would have won in a GB-wide general election if voters across the country as a whole had behaved in the same way as those who actually voted in the local elections that year in England. It provides a single, seemingly straightforward measure of party performance that can tell us not only how well or badly a party has done as compared with four years ago, but also as compared with any previous local elections for which a PNS is available – even though the places in which local elections are held varies considerably from one year to the next.

Yet almost inevitably answering such a ‘what if’ question is not as straightforward as it might seem. Given the large number of wards being contested, the calculation of the PNS has to be made on the basis of a sub-sample of the local contests. None of the parties fight all of the wards being contested, and some may well fight fewer wards than others. At the same time, local election results in England tell us nothing about the performance of the nationalist parties in Scotland and in Wales. Any estimate of the PNS is affected by the decisions that are made about how best to address these issues. It is thus not surprising that there have often been some differences between the PNS that we have calculated for the BBC at previous local elections and that calculated by the local election experts, Colin Rallings and Michael Thrasher of Plymouth University, whose National Equivalent Vote (NEV) appears each year in The Sunday Times. 

Those differences are typically limited with neither the PNS or NEV consistently better for any party. But since 2015 there has been a tendency for the PNS to be higher for Liberal Democrats and Others, and correspondingly lower for the Conservative and Labour than the NEV series. This can be seen in the table below. 

 NEV:   PNS:   Diff:   
 ConLabLDOtherConLabLDOtherConLabLDOth
20132629143125291432-100+1
20143031112829311327 -10+2-1
20153631102335291125-1-2+1+2
20163233142130311524-2-2+1+3
20173928181537281817-200+2
20183736141335351614-2-1+2+1
20193131172128281925-3-3+2+4
20214030151536291718-4-1+2+3

It is not entirely clear why these gaps have emerged. Here we aim to explain some of the key features of the PNS methodology that might help explain the difference. In doing so, our aim is not to suggest that one approach is better than another, but rather to explain some of the decisions we have made that can have an impact on the figures we publish. 

Continue reading Understanding the Local Elections Projected National Share (PNS) in 2022

Forecasting Local Election net seat gains/losses 2022

by Stephen Fisher, 3rd May 2022.

There are local elections tomorrow in England, Scotland, and Wales. Table 1 below shows my forecasts for net seat changes for each. They are based on projecting changes in opinion poll performance since the last round of local elections, with different methods for the different countries as discussed below. They represent what we might expect if the changes in party performance in local elections are on par with changes in the opinion polls.

Table 1. Forecasts for English local election net seat gains/losses for 2022

EnglandWalesScotlandOverall
Con-165-80-36-281
Lab+140+80+84+304
LD+700+6+76
PC 0 0
SNP  -24-24

The Conservatives are expected to lose seats in all three countries. They are defending a strong 2017 base in Scotland and Wales, and dropped in the polls since both 2017 and 2018 when the English seats were last fought. 

Labour are expected to be the main beneficiaries from Conservative losses. The projections suggest Labour might recover most but not all the losses they suffered in 2017 in Scotland and Wales. In England, Labour are trying this week to build on cumulative gains from 2010, 2014 and 2018. They already control over half the seats up this year. Since Labour are at 40% in the polls, their poll support is no greater than is was in 2018. Instead of gaining seats from winning more voters, Labour are projected to make council seat gains in England primarily from the drop in the Conservative vote. But, as discussed below, last-year’s experience shows there are various reasons why that might not happen. 

Perhaps most surprising of the forecasts is the projected drop in SNP seats from what was considered a disappointing performance in 2017, winning only 32% of the first-preference vote when typically the party has been winning at least 45% of the vote in Scotland-wide elections since the independence referendum in 2014. My projection for the Scottish locals this week is based on changes since 2017 in local-election first-preference vote intention polls. Even general-election vote-intention polls show no advance on 2017 for the SNP. The party will be hoping that more of the people who vote for them in Westminster and Holyrood elections will support them in the locals this week. 

England

Typically, the Conservatives lose English council seats when their lead over Labour in the opinion polls drops from what it was the last time the election was fought. Similarly, if the Tories extend their lead, then they typically make net gains. The graph below shows that pattern for local elections in England when the Conservatives were in government and the local elections were not on the same day as a general election. There is a strong correlation, but with a lot of noise around it, meaning any forecast comes with a big range of uncertainty. This year either of the two main parties could be either up or down by more than 100 seats based on the variation in previous local elections.

Last year’s local elections contributed to that noise. The graph above distinguishes between what happened in the elections that were delayed from 2020 because of the Covid pandemic, and those that happened in 2021 as scheduled. The Conservatives substantially outperformed expectations from the historical pattern for both sets of elections. They made +248 gains in the “2020” set despite polls (in 2021) showing only a 1-point increase in the Con-Lab lead since 2016. For the 2021 set, the Tories suffered a net loss of only 14 seats despite the poll lead dropping by 13 points from the high that Theresa May enjoyed in the 2017 local elections (before losing most of it at the general election the following month).

Continue reading Forecasting Local Election net seat gains/losses 2022

What impact did the Brexit Party have in the 2019 general election?

by John Curtice, Stephen Fisher and Patrick English

One of the most dramatic developments during the campaign for the December 2019 general election, whose second anniversary is this weekend, was the decision announced by Nigel Farage on 11 November that the Brexit Party would not contest those seats being defended by the Conservatives. Instead it would concentrate its firepower on those seats currently in the hands of one of the opposition parties. This represented a substantial retreat for a party that just the previous May had topped the poll in the European elections.

But was the decision as important as it was dramatic? Part of the answer to that question depends, of course, on the impact the decision had on the overall popularity of the party where it was continuing to fight. It almost certainly helped to reduce it. In the week leading up to Mr Farage’s announcement, the Brexit Party stood on average in the polls at 16% among those who had voted Leave in the 2016 EU referendum. (It had no measurable support at all among Remain supporters.) A week later, its support among Leave voters had fallen by five points to 11% (while support for the Conservatives was up by five points). By polling day, the party was down to just 4%.

Of course, much of this drop would have been occasioned by the gradual realisation among voters in the relevant seats that the Brexit Party was not standing locally, though the scale of the drop by polling day is too big to be explained by that alone. Equally, some voters will have switched away from the party for reasons that had nothing to do with Mr Farage’s decision. But it certainly looks highly likely that the Brexit Party’s partial withdrawal helped to undermine its level of support where it continued to stand.

Even so, the party still managed to win an average of 5.7% of the vote in those seats in England & Wales that it did fight – potentially enough to have some impact on the relative fortunes of the other parties. But what impact did it have? Mr Farage’s decision appeared to reflect his oft made claim that his party was able to gain the support of voters in Labour-inclined, more working class constituencies that the Conservatives could never reach. Thus, while standing down in Conservative-held seats might help avoid a split in the Leave vote that could help the opposition parties to gain Conservative-held seats, continuing to contest opposition-held seats might help the pro-Brexit Conservatives to gain Labour-held seats and thus the overall majority that Boris Johnson was seeking.

The success or otherwise of this strategy is addressed in our analysis of the constituency election results in the recently published ‘Nuffield’ study of the last election, The British General Election of 2019. It proved far from an easy question to answer.

We could, of course, look at the choices that those who voted for the Brexit Party in 2019 had made at the previous election two years earlier. In fact, according to the 30,000 sample British Election Study internet panel (BESIP), rather more Brexit Party voters had voted Conservative (40%) than Labour (30%) in 2017, suggesting that there was little truth to Mr Farage’s assertion that this party was particularly effective at winning over Labour voters in Labour constituencies. However, what matters here is not how Brexit Party voters voted in 2017, but how they would have voted in the absence of a Brexit Party candidate in 2019.

One way of trying to address this counterfactual question is to compare the performance of the parties where the Brexit Party did and did not stand. At +5.3%, the swing from Labour to Conservative since 2017 was rather higher in seats (In England & Wales) that the Brexit Party contested than it was is those seats that it did not (+4.7%). That suggests that the presence of a Brexit Party candidate did do Labour somewhat more harm than the Conservatives, an observation apparently reinforced by the fact that the swing to the Conservatives tended to be higher the bigger the Brexit Party’ share of the vote.

But if the Brexit Party were winning over Labour voters who would never back the Conservatives, we would anticipate that Labour would have lost support more heavily among Leave voters in seats where the Brexit Party stood than it did in those that it did not. Yet the BESIP data suggest that, if anything, the opposite is true. In seats where the Brexit Party stood, 57% of those who voted Leave in 2016 and Labour in 2017 voted for the party again, compared with just 48% in seats the Brexit Party did not contest. Meanwhile, whereas in those seats the Brexit Party did not contest 39% of Labour Leave voters switched to the Conservatives, in seats where there was a Brexit Party presence only 26% did so. Much of the difference between these two figures is accounted for by the 10% who backed the Brexit Party in seats the party was contesting – suggesting that all the Brexit Party managed to achieve where it stood was to divert the support of Labour Leave voters away from the Conservatives.

This in turn, however, is also probably too bold a conclusion. When the BESIP asked respondents to give both the parties and the leaders a mark out of ten, those Leave voters who switched from Labour to the Brexit Party gave both the Conservatives (3.8) and Boris Johnson (5.3) a lower score than those who switched from Labour to the Conservatives (6.2 and 7.1 respectively), suggesting that perhaps not all of those who switched from Labour to the Brexit Party would have swung behind the Conservatives in the absence of a Brexit Party candidate.

In the absence of data on the second preferences of those who voted for the Brexit Party, we modelled the behaviour of Labour Leave voters in seats that the Brexit Party did not contest (taking into account demographics and evaluations of the parties and leaders) and then applied the resulting equation to those voters who elsewhere switched from Labour to the Brexit Party. This suggested that in the absence of a Brexit Party candidate, 70% of those who switched from Labour to the Brexit Party would have instead backed the Conservatives, while just 30% would have stuck with Labour.

It is on this basis that we conclude that the Brexit Party’s decision to continue to fight opposition held seats cost the Conservatives seats they might otherwise have won. If we apply that 70:30 ratio to the share of the Brexit Party’ share of the vote in each constituency, we find that there are around 25 seats that Labour managed to retain that might otherwise have fallen from its grasp – many of them in the North of England and the Midlands – and enough to give the Conservatives an overall majority of 130.

Meanwhile, because there was a national swing from Labour to the Conservatives anyway, the Brexit Party’s decision to stand down in Conservative-held seats helped save the Conservatives from defeat in at most a small handful of Remain-inclined seats.

The squeeze on Brexit Party support during the 2019 election campaign, a squeeze that was probably accelerated by its decision to vacate the contest in Conservative-held seats, played a role in enabling the Conservatives to unite much of the Leave vote behind it, an outcome that was central to their success in winning an overall majority of 80. Even so, the Brexit Party’s continued presence in seats that the opposition parties were defending still acted as something of a brake on that achievement. Small though its tally of votes might have been, under the first-past-the-post electoral system, it was still enough to make a significant dent in the size of the Conservative majority and in reducing the extent of the breach in Labour’s so-called ‘Red Wall’. Little wonder that, in the wake of the outcome of the Old Bexley & Sidcup by-election, commentators are now wondering what role, if any, the Brexit Party’s successor, Reform UK, might play at the next general election.

This blog is also to be found on the What UK Thinks website.

            The British General Election of 2019, by Robert Ford, Tim Bale, Will Jennings and Paula Surridge is published by Palgrave Macmillan.

What do citizens forecast for the 2021 German federal election?

Guest post by Andreas Murr, University of Warwick.

On Sunday 26 September Germans will go to the ballot box to cast several votes in state and federal elections.  The Economist and several academics have published their federal election forecasts.   But who do Germans think will win the election?  Two research teams have used such citizen forecasts to predict the upcoming federal election.

Murr and Lewis-Beck predict the parties’ national vote shares based on citizens’ responses to the question “Who will win the general election?”  Their prediction is based on a regression model of historical vote shares on citizen forecasts and whether there was a grand coalition.  The historical data goes back until 1980.  Combing their regression model with a recent Politbarometer survey from June, they predict CDU/CSU to win 34% of the vote and SPD to win 21%. of the vote.  According to them, a CDU/CSU/SPD coalition seems the safest bet statistically, though a CDU/CSU-Greens coalition is not out of the question.

Kayser et al. predict both constituency winners and parties’ national vote shares by simply aggregating citizen forecasts collected in a survey the last two weeks.  They collected two different kinds of citizen forecasts.  First, they asked citizens to forecast which candidate will win in their constituency.  And, second, they asked citizens what vote share each party will win nationally.  The authors then simply predict the constituency to be won by the candidate who most citizens say will win.  And, they predict the vote share of a party to be the average of citizens’ forecasted vote share.  In other words, only survey data and no historical data was used to forecast the election.  Citizens collectively predict that CDU/CSU will win more constituencies than the SPD (174 v. 98).  However, they also collectively predict that SPD will win a higher national vote share than CDU/CSU (25% v. 23%).  The Greens are predicted to win 16%.

Why do the forecasts of the two teams differ?  The two teams use different methods and forecast with different lead times.  However, we can update the forecast of Murr and Lewis-Beck by using the most recent Politbarometer survey from mid-September.  This way the forecasts differ only in method.  If we do this, then both forecasts go in the same direction.  The Murr and Lewis-Beck model then predicts CDU/CSU to win 25% and SPD to win 31%.  In other words, their updated model would also predict the SPD to win a majority of the votes, though a bigger one than predicted by Kayser et al.  This said, Murr and Lewis-Beck did some analysis of the optimal lead time of their model: they find that it forecasts more accurately with a lead time of two months (June) than one month (September) on average.  Of course, soon we will know which lead time or method forecasted better in this election.

What does it take to get elected in Scotland and Wales?

By Stephen Fisher, 6th May 2021.

There is speculation about how well various new and small parties will do in today’s elections to the Scottish and Welsh Parliaments. Former SNP First-Minister Alex Salmond has recently established the Alba Party, while George Galloway has created Unity 4 All. Both are contesting regional list seats in Scotland. Meanwhile, Abolish the Welsh Assembly has come fourth, with at least 6% of the regional list vote, in all four polls for the Welsh Parliament (Senedd Cymru) since mid-April. Also, Reform-UK, the renamed Brexit Party, is fielding candidates in both countries. These are just some of the various contenders.

This post explains some of the complications in trying to figure out what share of the vote a party needs to get elected in these institutions. It ends up pointing to past experience as a guide and drawing comparison with and the electoral system for the Greater London Assembly (GLA).  

TLDR: The experience of all five elections to the Scottish Parliament suggests that around 5.5% of a regional list vote is usually enough to win a seat. It would be rare, but not impossible, to miss out on a seat with 6%. Winning on 5.2% to 5.4% is not uncommon. The lowest D’Hondt ratio ever to yield a seat in Scotland was 4.6% (equivalent to a single party winning a seat on that share). That was in 2003 when the SNP won 5 seats, including 2 list seats, with 23.0% of the Mid Scotland and Fife list vote. For the Senedd, a share of around 6.5% in a region is likely to be enough, especially in North Wales and Mid and West Wales which have historically been more accessible to small parties. By comparison, the GLA has a 5% legal threshold, without which parties would get seats with 3.8% of the vote.

Scotland

The Scottish Parliament has 129 seats. The electoral system is sometimes described as proportional, but it is significantly different from pure proportional representation. If it were close then a party would win a seat for roughly every 0.8% of the vote, since 100/129=0.8. More precisely, if all the seats were elected by the D’Hondt method of proportional representation a party would be guaranteed a seat with more than 100/(1+Number of seats) per cent of the vote, that is 100/130 = 0.77% of the vote. As we shall see that is much lower than what is actually required to get elected.

Continue reading What does it take to get elected in Scotland and Wales?

Forecasting Local Election Net Seat Gains/Losses 2021

by Stephen Fisher, 6th May 2021.

Local election seat gains and losses are hideous to try to predict at the best of times. Various factors make this year especially hard. The Covid-19 pandemic means that the 2020 round of local elections was postponed to today, and there have been various boundary changes and restructuring that mean it is hard to allocate many seats to either the 2016-2020 or the 2017-2021 4-year cycle that my model requires. Nonetheless, and despite the weaknesses of my 2019 forecast (discussed below), I have ploughed on regardless. This is not due to the kind of stoic determination that the Finns call sisu: amusingly summarised by someone as, “chin down and press on to the next disappointment.” Rather, I am still curious as to how much of a guide (on a broad macro level) Westminster vote intention polls are to local election outcomes. 

Headline forecasts are in the table below together with the range of possible outcomes in the model. The table includes the projections from Michael Thrasher’s second scenario here. He and Colin Rallings would normally calculate an expected National Equivalent Vote (NEV) from the results of local by-elections, but those have not been happening. Instead Michael Thrasher has used adjusted opinion polls to estimate the NEV and then projected the implied changes at the ward/division level. Both mine and his methods use polls this year, but whereas Michael’s is a local projection based on previous ward and division level results, mine is based on regression models of the macro historical relationship between poll changes and net seat changes. (For a really micro, individual-level approach, predicting Labour losses in “red-wall” councils, but not overall net gains and losses, see the analysis from YouGov’s Patrick English here.)

Table 1. Forecasts for English local election net seat changes 2021

ForecastRangeThrasher
Con-210-770 to +350+120
Lab+70-290 to +430-50
LD+140-50 to +330-70
Others0 0

It is striking that my forecasts point in different directions from Michael Thrasher’s projections for every party. But with 4630 odd seats up for election, both sets of predictions are for modest net changes. The prediction intervals for my forecasts comfortably include zero for all three parties, so each of them could easily end either up or down when all the results are in. 

Continue reading Forecasting Local Election Net Seat Gains/Losses 2021

Calculating the Local Elections Projected National Share (PNS) and Projected House of Commons in 2021

By John Curtice and Stephen Fisher.

This post is a revised and updated version of a similar post from 2019 here.

A key summary statistic of the outcome of each year’s annual round of local elections is the so-called Projected National Share (PNS). This is an estimate of the share of the vote that the principal parties would have won in a GB-wide general election if voters across the country as a whole had behaved in the same way as those who actually voted in the local elections. It provides a single, seemingly straightforward measure of party performance that can tell us not only how well or badly a party has done as compared with four years ago (when, typically, most of the seats up for grabs were last contested), but also as compared with any previous local election for which a PNS is available – even though the places in which local elections are held varies considerably from one year to the next.

In this blog we provide some guidance as to how the PNS is being calculated this year by the BBC and how it will be used to project an outcome in terms of House of Commons seats.

Continue reading Calculating the Local Elections Projected National Share (PNS) and Projected House of Commons in 2021

Some thoughts on the US election forecasts 2020

By Stephen Fisher, 3rd November 2020.

Americans are evenly divided in their expectations of who will win today’s presidential election. YouGov found 40% expect Joe Biden to win, 39% think Donald Trump will win and 21% are “Not sure” (see p59 here). By contrast, in 2016 a similar question about expectations of the outcome from New York Times/CBS News polls here had 56% expecting Hilary Clinton to win and just 33% thinking Donald Trump would. Differences in question wording means we should be cautious about detailed comparison, but it appears that Americans collectively are much more unsure of the outcome of this election than they were last time.

They may be more cautious because the majority of them were wrong last time. Then their expectations were in line with the polls and the forecasters, some of whom were clearly over-confident for Hilary Clinton, as I argued before the fact

The problems of the polls in 2016 were diagnosed and methodology has improved. In particular, many polls are now weighting their samples so that the numbers of people with different levels of education are represented in proportion to their prevalence in the population. Part of the problem with some of the state-level polls last time was that they did not do so. Without that education weighting the polls this year would be showing even bigger Biden leads.

Some of the most over-confident forecasters from 2016 are no longer forecasting (including the Huffington Post), while others, such as Sam Wang, have adjusted their models and de-emphasised the predicted probabilities. 

Nate Silver’s FiveThirtyEight forecasts did best in 2016, mainly by having a model with more prediction uncertainty and so a larger probability of Clinton winning. Despite that, one of the most prominent debates about the election forecasts this year has been prompted by the superb statistician Andrew Gelman querying whether the FiveThirtyEight model has included too much uncertainty. 

Andrew Gelman is part of The Economist forecasting team. Their model has consistently had more confidence in a Biden win. It now gives Biden a 97% chance, to FiveThirtyEight’s 89% chance, of winning. Their central forecasts are extremely close, it is their uncertainty levels that differ.

More recently, Andrew Gelman has posted this on the effects of using Normal versus t distributions for the tail probabilities, and this on the value of learning from experience as a forecaster, and a fair few interesting posts about election forecasting in between. I can see the case for the modelling choices in The Economist model, but somehow the published level of uncertainty from FiveThirtyEight feels more reasonable to me. Since there will be sources of uncertainty that are not measurable from the polling and historical data, I think it is okay to err on the side of allowing more uncertainty when making choices between different justifiable modelling options. 

Another forecaster that has adapted their methods since 2016 is YouGov. Their MRP model forecast has a slightly higher forecast electoral college tally for Biden (364) than does The Economist (356) or FiveThirtyEight (348). The difference is based on a slightly larger predicted Biden lead in votes (9 points for YouGov compared with 8 points for FiveThirtyEight). 

YouGov has not published a predicted probability for a Biden win, but from their graph of simulations it seems that they have in the order of a 10% chance of a Trump win. That is roughly the same as FiveThirtyEight despite a lower central forecast for Trump. Moreover, in 2016 the corresponding YouGov model put very little probability on a Trump win despite a much closer forecast for the national vote share margin. The additional uncertainty estimation this time seems like an improvement. 

It is worth noting that whereas the FiveThirtyEight, Economist and similar models depend on a plethora of polls funded by others, YouGov build their model on their own polling data. They might do a very large poll, but the YouGov sample size would still pale by comparison with the cumulative sample size of all the other campaign polls at the national and state levels. YouGov’s ability to refine their methodology is important for understanding the potential for getting effective forecasts from more modest amounts of polling data.

Using perhaps the most data of all, PollyVote aggregate not only polls but models, betting markets, citizen expectations and expert forecasts. That website has very helpful descriptions of how the different data sources and models work with links to research on how they have fared in the past. Their overall combined forecast is for a Biden victory, but with a smaller margin than the polls and poll-based models point to. This is largely because citizens suggest the race is close (as discussed above) and the betting markets and experts are cautious about the polls. 

One model that points in completely the opposite direction from nearly all others is Helmut Norpoth’s Primary Model. It predicts a comfortable Trump victory because he is an incumbent president (they usually win) and because Biden did not do well in early primary contests, suggesting he is a weak candidate. The Primary Model has a very good track record of successful forecasts. It at least provides a basis for claiming that a Biden win would be remarkable by historical experience. 

Overall, the evidence from the (apparently improved) polls is a very strong indication that Biden will win, and win very comfortably. The dearth of media coverage of election forecasts and the cautious coverage of opinion polls this campaign, as a result of their failure to anticipate Trump’s victory in 2016, seems to have led to public uncertainty. If the polls are right, and especially if Biden does a bit better than they suggest, it will come as a shock to many, especially Republican voters who mainly expect Trump to win (see p59 here).

Where has the swing to Biden come from?

Evidence from the polls seems to be that partisan polarization between self-described liberals and Democrats on the one side and conservatives and Republicans on the other has worsened since 2016, with both groups more strongly backing their candidate than they did in 2016. What is helping Biden most is a big swing among those who describe themselves as independent and/or moderate. Since that big swing is mainly among whites and it is towards the Democrats the ethnic divide has narrowed as white vote intention moves closer to the strongly Democratic Latino and solidly Democratic Black vote.

At a macro level it looks like swing at the state level might be fairly uniform. The table below shows my quick calculations for the change in the lead in some of the larger more marginal states. The average lead for Biden in them is a bit bigger for YouGov than FiveThirtyEight, in keeping with the one-point larger lead YouGov expects in the national share of the vote. Relative to Hilary Clinton’s actual lead over Donald Trump in 2016, the predicted change this year is relatively similar in these states. There is variation between states in the extent of the swing expected, but the differences are not huge and they are not all consistent between the two forecasters. 

Table: FiveThirtyEight and YouGov predictions for selected swing states

 Biden lead  Biden lead – actual Clinton lead 
538YouGov538YouGov
Arizona2.65.46.18.9
Florida2.54.33.75.5
Georgia0.93.16.08.2
Michigan8.07.98.28.1
North Carolina1.72.45.46.1
Pennsylvania4.76.85.47.5
Wisconsin8.36.89.17.6
Average4.15.26.37.4
Note: All figures are in percentage points.

That said, both YouGov and FiveThirtyEight expect bigger than average swings in Michigan and Wisconsin, and a smaller than average swing in Florida. This would be the same as in 2016 but in the opposite direction. I have not had time to analyse whether there is a broader pattern of negative correlation between 2012-16 swing and 2016-20 swing, but it would not be surprising if the places that swung most to Trump in 2016 swing most heavily away from Trump today. 

House of Representatives

The House is largely expected to stay Democrat controlled. FiveThirtyEight give the party a 97% chance of retaining control. In 2018, the Democrats won 235 seats to the Republicans 199 on an 8.6 point lead in the popular vote, quite a margin to overturn. Currently, the Real Clear Politics generic congressional vote poll average has a Democrat lead of 6.8 points. Similarly, FiveThirtyEight predict a popular vote margin of 6.3 points for the Democrats. Since this is down what it was on 2018 it is remarkable that the Democrats are also predicted to win slightly more seats (239). 

If this comes to pass, I for one will be wondering whether the failure of a swing to the Republicans to yield any net seat gain will be due to incumbency advantage, especially “sophomore surges” given that most of the Republican target seats are being defended by first-term Democrat incumbents. 

While a swing to the Republicans in the House without net Republican seat gains would seem to be a sign of problematic unresponsiveness of the electoral system, it would represent a partial unwinding of a pro-Republican bias in the distribution of seats given votes and so arguably a sign of the system working more fairly.

Senate

There is much more uncertainty as to whether the Senate will flip from Republican to Democrat. Of the 35 Senate seats up for election this time 21 are begin defended by Republican incumbents, and incumbency effects in US Senate elections are very strong. However, the seats up this time were last fought in 2014, which was a midterm year when Obama was president. The big c.9.7 point swing to the Republicans then appears to be being partially undone. Also, the Democrats have more of the seats not up for election: 35 out of 65 including two independents caucusing with the Democrats.

Sam Wang’s Princeton Election Consortium is predicting the Senate to go 53 D 47 R, while Fivethirtyeight’s central prediction is 51.5 D 48.5 R, with a 75% chance of Democratic control. 

Overall, based on the polling evidence the Democrats are likely win control all three branches of government, with the Senate the least certain to go their way.

Further links:

PS: Political Science & Politics academic forecasts (from September 2020) and related articles: https://www.cambridge.org/core/journals/ps-political-science-and-politics/2020-presidential-election-forecasting-symposium

The review, by Alfred Cuzán, of previous forecasts in that series is especially worth a read.

Pluralvote.com claims to have a powerful model based on polls and media tracking that improves on polls only. 

Patrick English has a very confident model for Biden here.

Final combined forecast for the 2019 general election

By Stephen Fisher, John Kenny and Rosalind Shorrocks

Since our first combined forecast at the start of the campaign, the number of forecasts for this general election has grown substantially. All of the combined forecasts – seats, vote shares, and probabilities – are pointing to a Conservative majority. However, some individual forecasts do predict a hung parliament, and there is variation within each forecast type over how certain this majority is, and how large it is predicted to be.

SEATS

Seat projections from the betting markets, complex models, and simple models are all very similar, forecasting a Conservative majority of between 343 and 351 seats. The average number of seats across all forecasts that the Conservatives are expected to win – 341 – is slightly lower but ultimately very similar to the forecast last week.

Since last week the Political Studies Association have published their Expert Survey, in which the average expected number of Conservative seats suggests a hung parliament with the Conservatives just shy of a majority. It is interesting that the experts surveyed by the PSA predict the Conservatives will win fewer seats than is currently suggested by the polls. Perhaps they are factoring in the same kind of late-campaign changes as observed in 2017 – although it should be noted that when a similar kind of survey was run for the EU referendum in 2016, the average predicted vote share for Remain and Leave amongst experts was the furthest away from the actual result than any of the other types of forecast. They also predicted a Conservative majority in 2017, although that prediction was made much more earlier in the campaign when the Conservatives had considerable leads over Labour in the polls.

Seats Betting Markets Complex models Simple models Experts Average
Con 346 343 350 324 341
Lab 221 225 219 233 224
LD 18 17 18 25 19
Brexit 0 0 0 2 1
Green 1 1 1 2 1
SNP 43 44 41 42 43
PC 4 3 3 4 4
Con majority 42 37 49 -2 31

 

Conservative Seats - 11th December

The similarity between the seat projections from most sources hides considerable variation within one particular forecast type – complex models. These models range from predicting 311 Conservative seats to 366 – the difference between a hung parliament and a healthy Conservative majority. They also range between 190 and 268 for Labour. It is particularly noteworthy that the voter expectation model, from Murr, Stegmaier, and Lewis-Beck, which uses citizen forecasts to predict the number of seats, forecasts one of the highest number of Conservative seats (360) and the lowest number of Labour seats (190). This is in contrast to our implied probability calculated from the citizen forecasts, which suggest that citizens are in general the least convinced about the likelihood of a Conservative majority compared to other forecasting methods. This suggests these surveys also suffer from being open to multiple interpretations and methods of analysis, as well as the question wording effects we discussed last week.

Continue reading Final combined forecast for the 2019 general election

Election analysis and forecasting