Some thoughts on the US election forecasts 2020

By Stephen Fisher, 3rd November 2020.

Americans are evenly divided in their expectations of who will win today’s presidential election. YouGov found 40% expect Joe Biden to win, 39% think Donald Trump will win and 21% are “Not sure” (see p59 here). By contrast, in 2016 a similar question about expectations of the outcome from New York Times/CBS News polls here had 56% expecting Hilary Clinton to win and just 33% thinking Donald Trump would. Differences in question wording means we should be cautious about detailed comparison, but it appears that Americans collectively are much more unsure of the outcome of this election than they were last time.

They may be more cautious because the majority of them were wrong last time. Then their expectations were in line with the polls and the forecasters, some of whom were clearly over-confident for Hilary Clinton, as I argued before the fact

The problems of the polls in 2016 were diagnosed and methodology has improved. In particular, many polls are now weighting their samples so that the numbers of people with different levels of education are represented in proportion to their prevalence in the population. Part of the problem with some of the state-level polls last time was that they did not do so. Without that education weighting the polls this year would be showing even bigger Biden leads.

Some of the most over-confident forecasters from 2016 are no longer forecasting (including the Huffington Post), while others, such as Sam Wang, have adjusted their models and de-emphasised the predicted probabilities. 

Nate Silver’s FiveThirtyEight forecasts did best in 2016, mainly by having a model with more prediction uncertainty and so a larger probability of Clinton winning. Despite that, one of the most prominent debates about the election forecasts this year has been prompted by the superb statistician Andrew Gelman querying whether the FiveThirtyEight model has included too much uncertainty. 

Andrew Gelman is part of The Economist forecasting team. Their model has consistently had more confidence in a Biden win. It now gives Biden a 97% chance, to FiveThirtyEight’s 89% chance, of winning. Their central forecasts are extremely close, it is their uncertainty levels that differ.

More recently, Andrew Gelman has posted this on the effects of using Normal versus t distributions for the tail probabilities, and this on the value of learning from experience as a forecaster, and a fair few interesting posts about election forecasting in between. I can see the case for the modelling choices in The Economist model, but somehow the published level of uncertainty from FiveThirtyEight feels more reasonable to me. Since there will be sources of uncertainty that are not measurable from the polling and historical data, I think it is okay to err on the side of allowing more uncertainty when making choices between different justifiable modelling options. 

Another forecaster that has adapted their methods since 2016 is YouGov. Their MRP model forecast has a slightly higher forecast electoral college tally for Biden (364) than does The Economist (356) or FiveThirtyEight (348). The difference is based on a slightly larger predicted Biden lead in votes (9 points for YouGov compared with 8 points for FiveThirtyEight). 

YouGov has not published a predicted probability for a Biden win, but from their graph of simulations it seems that they have in the order of a 10% chance of a Trump win. That is roughly the same as FiveThirtyEight despite a lower central forecast for Trump. Moreover, in 2016 the corresponding YouGov model put very little probability on a Trump win despite a much closer forecast for the national vote share margin. The additional uncertainty estimation this time seems like an improvement. 

It is worth noting that whereas the FiveThirtyEight, Economist and similar models depend on a plethora of polls funded by others, YouGov build their model on their own polling data. They might do a very large poll, but the YouGov sample size would still pale by comparison with the cumulative sample size of all the other campaign polls at the national and state levels. YouGov’s ability to refine their methodology is important for understanding the potential for getting effective forecasts from more modest amounts of polling data.

Using perhaps the most data of all, PollyVote aggregate not only polls but models, betting markets, citizen expectations and expert forecasts. That website has very helpful descriptions of how the different data sources and models work with links to research on how they have fared in the past. Their overall combined forecast is for a Biden victory, but with a smaller margin than the polls and poll-based models point to. This is largely because citizens suggest the race is close (as discussed above) and the betting markets and experts are cautious about the polls. 

One model that points in completely the opposite direction from nearly all others is Helmut Norpoth’s Primary Model. It predicts a comfortable Trump victory because he is an incumbent president (they usually win) and because Biden did not do well in early primary contests, suggesting he is a weak candidate. The Primary Model has a very good track record of successful forecasts. It at least provides a basis for claiming that a Biden win would be remarkable by historical experience. 

Overall, the evidence from the (apparently improved) polls is a very strong indication that Biden will win, and win very comfortably. The dearth of media coverage of election forecasts and the cautious coverage of opinion polls this campaign, as a result of their failure to anticipate Trump’s victory in 2016, seems to have led to public uncertainty. If the polls are right, and especially if Biden does a bit better than they suggest, it will come as a shock to many, especially Republican voters who mainly expect Trump to win (see p59 here).

Where has the swing to Biden come from?

Evidence from the polls seems to be that partisan polarization between self-described liberals and Democrats on the one side and conservatives and Republicans on the other has worsened since 2016, with both groups more strongly backing their candidate than they did in 2016. What is helping Biden most is a big swing among those who describe themselves as independent and/or moderate. Since that big swing is mainly among whites and it is towards the Democrats the ethnic divide has narrowed as white vote intention moves closer to the strongly Democratic Latino and solidly Democratic Black vote.

At a macro level it looks like swing at the state level might be fairly uniform. The table below shows my quick calculations for the change in the lead in some of the larger more marginal states. The average lead for Biden in them is a bit bigger for YouGov than FiveThirtyEight, in keeping with the one-point larger lead YouGov expects in the national share of the vote. Relative to Hilary Clinton’s actual lead over Donald Trump in 2016, the predicted change this year is relatively similar in these states. There is variation between states in the extent of the swing expected, but the differences are not huge and they are not all consistent between the two forecasters. 

Table: FiveThirtyEight and YouGov predictions for selected swing states

 Biden lead  Biden lead – actual Clinton lead 
538YouGov538YouGov
Arizona2.65.46.18.9
Florida2.54.33.75.5
Georgia0.93.16.08.2
Michigan8.07.98.28.1
North Carolina1.72.45.46.1
Pennsylvania4.76.85.47.5
Wisconsin8.36.89.17.6
Average4.15.26.37.4
Note: All figures are in percentage points.

That said, both YouGov and FiveThirtyEight expect bigger than average swings in Michigan and Wisconsin, and a smaller than average swing in Florida. This would be the same as in 2016 but in the opposite direction. I have not had time to analyse whether there is a broader pattern of negative correlation between 2012-16 swing and 2016-20 swing, but it would not be surprising if the places that swung most to Trump in 2016 swing most heavily away from Trump today. 

House of Representatives

The House is largely expected to stay Democrat controlled. FiveThirtyEight give the party a 97% chance of retaining control. In 2018, the Democrats won 235 seats to the Republicans 199 on an 8.6 point lead in the popular vote, quite a margin to overturn. Currently, the Real Clear Politics generic congressional vote poll average has a Democrat lead of 6.8 points. Similarly, FiveThirtyEight predict a popular vote margin of 6.3 points for the Democrats. Since this is down what it was on 2018 it is remarkable that the Democrats are also predicted to win slightly more seats (239). 

If this comes to pass, I for one will be wondering whether the failure of a swing to the Republicans to yield any net seat gain will be due to incumbency advantage, especially “sophomore surges” given that most of the Republican target seats are being defended by first-term Democrat incumbents. 

While a swing to the Republicans in the House without net Republican seat gains would seem to be a sign of problematic unresponsiveness of the electoral system, it would represent a partial unwinding of a pro-Republican bias in the distribution of seats given votes and so arguably a sign of the system working more fairly.

Senate

There is much more uncertainty as to whether the Senate will flip from Republican to Democrat. Of the 35 Senate seats up for election this time 21 are begin defended by Republican incumbents, and incumbency effects in US Senate elections are very strong. However, the seats up this time were last fought in 2014, which was a midterm year when Obama was president. The big c.9.7 point swing to the Republicans then appears to be being partially undone. Also, the Democrats have more of the seats not up for election: 35 out of 65 including two independents caucusing with the Democrats.

Sam Wang’s Princeton Election Consortium is predicting the Senate to go 53 D 47 R, while Fivethirtyeight’s central prediction is 51.5 D 48.5 R, with a 75% chance of Democratic control. 

Overall, based on the polling evidence the Democrats are likely win control all three branches of government, with the Senate the least certain to go their way.

Further links:

PS: Political Science & Politics academic forecasts (from September 2020) and related articles: https://www.cambridge.org/core/journals/ps-political-science-and-politics/2020-presidential-election-forecasting-symposium

The review, by Alfred Cuzán, of previous forecasts in that series is especially worth a read.

Pluralvote.com claims to have a powerful model based on polls and media tracking that improves on polls only. 

Patrick English has a very confident model for Biden here.

Final combined forecast for the 2019 general election

By Stephen Fisher, John Kenny and Rosalind Shorrocks

Since our first combined forecast at the start of the campaign, the number of forecasts for this general election has grown substantially. All of the combined forecasts – seats, vote shares, and probabilities – are pointing to a Conservative majority. However, some individual forecasts do predict a hung parliament, and there is variation within each forecast type over how certain this majority is, and how large it is predicted to be.

SEATS

Seat projections from the betting markets, complex models, and simple models are all very similar, forecasting a Conservative majority of between 343 and 351 seats. The average number of seats across all forecasts that the Conservatives are expected to win – 341 – is slightly lower but ultimately very similar to the forecast last week.

Since last week the Political Studies Association have published their Expert Survey, in which the average expected number of Conservative seats suggests a hung parliament with the Conservatives just shy of a majority. It is interesting that the experts surveyed by the PSA predict the Conservatives will win fewer seats than is currently suggested by the polls. Perhaps they are factoring in the same kind of late-campaign changes as observed in 2017 – although it should be noted that when a similar kind of survey was run for the EU referendum in 2016, the average predicted vote share for Remain and Leave amongst experts was the furthest away from the actual result than any of the other types of forecast. They also predicted a Conservative majority in 2017, although that prediction was made much more earlier in the campaign when the Conservatives had considerable leads over Labour in the polls.

Seats Betting Markets Complex models Simple models Experts Average
Con 346 343 350 324 341
Lab 221 225 219 233 224
LD 18 17 18 25 19
Brexit 0 0 0 2 1
Green 1 1 1 2 1
SNP 43 44 41 42 43
PC 4 3 3 4 4
Con majority 42 37 49 -2 31

 

Conservative Seats - 11th December

The similarity between the seat projections from most sources hides considerable variation within one particular forecast type – complex models. These models range from predicting 311 Conservative seats to 366 – the difference between a hung parliament and a healthy Conservative majority. They also range between 190 and 268 for Labour. It is particularly noteworthy that the voter expectation model, from Murr, Stegmaier, and Lewis-Beck, which uses citizen forecasts to predict the number of seats, forecasts one of the highest number of Conservative seats (360) and the lowest number of Labour seats (190). This is in contrast to our implied probability calculated from the citizen forecasts, which suggest that citizens are in general the least convinced about the likelihood of a Conservative majority compared to other forecasting methods. This suggests these surveys also suffer from being open to multiple interpretations and methods of analysis, as well as the question wording effects we discussed last week.

Continue reading Final combined forecast for the 2019 general election

The 2018 and 2019 local election results suggested the Conservatives might struggle to get a majority at the next general election

By Chris Prosser and Stephen Fisher.

Every May local election results are analysed as indicators of the state of the political parties and scrutinised for what they tell might tell us about the outcome of the next general election. That general election is tomorrow. Although a lot has happened in politics since May 2019, and especially since May 2018, it might be worthwhile reminding ourselves of what happened in the local elections then and what those results portend for the outcome this week.

In 2015, when the polls failed to anticipate a Conservative majority, one of the more successful forecasting models was Chris Prosser’s one based on the 2013 and 2014 rounds of local elections. Unlike the polls – which showed the Conservatives and Labour neck and neck – that model forecast a four point Conservative lead in vote share. Using a uniform change projection, the forecast shares predicted a Conservative tally of 296 seats for 2015, short of an overall majority but better than a uniform projection from the final opinion polls and the best of a set of twelve academic forecasting models for that election.

Applying the same method again, the table below shows that both the 2018 and 2019 rounds of local elections point to a clear lead for the Conservatives in a subsequent general election. However, the forecast shares of the vote from both rounds do not suggest a big enough lead for the Conservatives to be sure of an overall majority. On the average of the two set of vote shares, coupled with a uniform change projection (also using last night’s YouGov MrP projected SNP and PC vote shares) points to a very narrow Conservative majority of 8.

Party Forecast share based on 2018 results Forecast share based on 2019 results Average forecast share Standard Error of share forecast Seats forecast
Con 40.8 36 38.4 4 329
Lab 33.8 29 31.4 4 231
LD 13.4 15.4 14.4 4 23
Other 12 19.6 15.8 4 65

The fact that a forecast based on local elections 7 and 19 months ago should be so close to last night’s YouGov MrP projection of Con 339, Lab 231, LD 15 is remarkable. There is just a difference of 10 seats for the Tories and none for Labour.

Given that since the 2018 local elections we’ve had May’s Deal and the first missed Brexit deadline, and since the 2019 local elections we’ve seen Brexit Party and Lib Dem success in the European parliament elections, a new prime minister, a new Brexit withdrawal agreement, and another missed Brexit deadline, it is even more surprising that most opinion polls now do not differ profoundly from what previous local elections suggested would happen.

The Party Leadership Model predicts a Conservative overall majority

By Andreas Murr and Stephen Fisher.

Two years ago Andrew Adonis wrote a piece in Prospect arguing that Labour should ditch Jeremy Corbyn because of the importance of party leadership for electoral success. The piece claimed that “the best leader wins and nothing else matters,” and in Lord Adonis’s view Mr Corbyn is the worst Labour leader since Michael Foot. So, Adonis concluded already in September 2017, that regarding the next election “Corbyn will lose it decisively if he contests it.”

In response to Adonis’ claims many, including Danny Finkelstein, expressed skepticism about the power of leadership and pointed out that it is difficult to properly evaluate the quality of leaders in retrospect. Once we know who won and who lost we have a tendency to convince ourselves that the winner was a better leader than the loser. But there are ways of producing measures of leadership quality prior to elections which have historically been useful for forecasting election outcome.

The Party Leadership Model, was devised by Andreas Murr in the run up to the 2015 general election. It successfully anticipated David Cameron’s victory unlike the vast majority of other forecasts at the time. Murr has elaborated the model further for this election to produce a seats forecast, not just a prediction as to who will emerge as prime minister. His new model forecasts that the Conservatives will win an overall majority this week with 342 seats, and that Labour will win 254 seats.

Continue reading The Party Leadership Model predicts a Conservative overall majority

Why are polls from different pollsters so different?

By Stephen Fisher and Dan Snow.

On average the polls have had a fairly consistent and comfortable lead for the Conservatives in this general election campaign. However, around that average there are substantial differences between polls. Some suggest the Conservatives might fail to win a big enough lead to secure a majority, while others point to a Tory landslide with a majority over a hundred. What’s going on?

In short, since this is a long and complicated blog, our tentative conclusion is that the big systematic differences between pollsters are due primarily to systematic differences in the kinds of people they have in their samples, even after weighting. Some of the sample profile differences translate straightforwardly into headline differences. For instance, having more 2017 Conservatives in a sample means there will be more 2019 Conservatives. In other areas there are more puzzling findings. Polls vary in the extent to which women are more undecided than men and in the extent to which young adults are less certain to vote, but neither source of variation has the effect on headline figures that we would expect. Nonetheless for most of the aspects of the poll sample profiles we have inspected, it is remarkable the extent to which polls differ primarily between pollsters, with relatively little change over time for each pollster. This suggests that the way different pollsters have been going about collecting responses has yielded systematically different kinds of respondents for different pollsters. With a couple of exceptions, it seems as though it has been the process of data collection rather than post-hoc weighting and adjustment that may be driving pollster differences in this campaign.

As the graph below shows, a large part of the variation between polls is between pollsters. The pollsters have shown a similar pattern of change in the Conservative-Labour lead over time, most with a peak in mid-November and a slight decline since. The headline Conservative-Labour lead – the basis for the swingometer – is the main guide to seat outcomes. So an important question is why pollsters differ systematically in the size of their published Conservative leads.

ConLabLead

In this blog post we use data from the standard tables that pollsters have to publish as part of the requirements of British Polling Council membership. They contain a wealth of information about the profiles of the different survey samples both before and after weighting and adjustment. We collected data from such tables for all polls between the 30th of October (when parliament voted for an early general election) to the 4th of December (just over a week before the end of the campaign). There have been more polls since then but so far as we can tell they do not substantially change the issues we raise here.

Continue reading Why are polls from different pollsters so different?

Fifth combined forecast for the 2019 general election

By Stephen Fisher, John Kenny and Rosalind Shorrocks 

Since our update last week there have been several new forecasts, most notably including the YouGov MRP (multilevel regression and post-stratification) model. That was a nowcast rather than a forecast, but the same is true of most of our “forecasts”. More on differences between forecasting models below, along with some observations about intriguing question wording effects for citizen forecasts.

But first, overall, the seats projections overall have tightened for the Conservatives, who are down from a 353 average last week to 346 this week, while Labour are up from 209 to 218. The Liberal Democrat forecast total has dropped yet again (from 23 to 19). Now they are estimated to return fewer MPs than they had going into the election (20), but still more than the number of seats they won in 2017 (12).

Seats Betting Markets Complex models Simple models Average
Con 343 347 348 346
Lab 220 218 217 218
LD 19 18 19 19
Brexit 0 0 0
Green 1 1 1 1
SNP 45 45 44 45
PC 5 4 3 4
Con majority 36 45 46 42

Conservative Seats - 4th December

There is now remarkably little difference between the betting markets, complex and simple models in the expected size of the Conservative majority. Particularly striking is that on average the complex models differ by only a seat for each party from the simple uniform change projections based on the average of the opinion polls.

Continue reading Fifth combined forecast for the 2019 general election

Fourth combined forecast for the 2019 general election

By Stephen Fisher, John Kenny and Rosalind Shorrocks 

There has not been much change in our combined forecast over the last week. The Conservatives are still apparently headed towards a comfortable majority (55 on average) based on an average forecast vote share lead over Labour of 12 points. The average predicted probability of a Tory majority has crept up to 72%, partly due to increasing confidence in the betting markets and the quantitative forecasting models, as well as the polls. Citizens remain much more sceptical. Concerns that the Liberal Democrats might make little advance continue, and were compounded by the Datapraxis MrP forecast of just 14 seats for the party. Otherwise the Datapraxis forecast was largely in line with other forecasts of headline seats totals. Further MrP based forecasts are due this week, including YouGov’s.

Seats Betting Markets Complex models Simple models Average
Con 346 353 360 353
Lab 210 212 204 209
LD 25 21 22 23
Brexit 0 0 0
Green 1 1 1 1
SNP 45 45 43 44
PC 4 4 4 4
Con majority 42 55 69 55

Conservative Seats - 27th November   Continue reading Fourth combined forecast for the 2019 general election

Third combined forecast for the 2019 general election

By Stephen Fisher, John Kenny and Rosalind Shorrocks 

Once again all three sources of seat forecasts suggest the Conservatives are heading to a comfortable majority, while Labour are on course for a result on par with their previous post-war low of 209 seats in 1983. The Liberal Democrat forecast has been dropping steadily, so that they are now expected to end up with only a few more MPs than the twenty they had when they chose to support the election.

Seats Betting Markets Complex models Simple models Average
Con 346 354 363 354
Lab 209 211 200 206
LD 30 22 22 25
Brexit 0 0 0
Green 2 1 1 1
SNP 46 46 44 45
PC 4 4 4 4
Con majority 42 58 75 58

Con seats 20 NovLib Dem seats 20 Nov 

Continue reading Third combined forecast for the 2019 general election

How did the election forecasts do in 2017?

By Stephen Fisher, Rosalind Shorrocks and John Kenny.

Lots of the forecasts for the 2017 British general election were wrong in pointing to a Conservative majority. Most even suggested an increased majority when the party actually lost one. The well-known exceptions are the exit poll and YouGov multilevel regression and post-stratification (MRP) model. Less commonly remembered are the polls that were within a point of the Conservative-Labour lead and so, by standard uniform change calculations, provided an indication that the Tory majority was in jeopardy.

During the election campaign we ran an exercise combining the different forecasts for shares of the vote, seat tallies, and probabilities of a Conservative majority. Inevitably the combined forecast is only as good as the average of what goes into it, and that average was poor. But since the exercise involved classifying and averaging forecasts by the methodology they used, it is possible to reflect on which types of forecasting method performed better than others.

The figure below shows predictions of the Conservative share of the vote over the course of the campaign from the Political Studies Association (PSA) expert survey, betting markets, forecasting models and opinion polls. Strictly speaking polls are snapshots not forecasts, but the final polls performed as well as the experts and better than all the other methods. Consistently the worst source of predictions for the Tory share were the betting markets.

Conservative Vote Share 2017Note: Due to an error in our 2ndJune 2017 betting market calculation (and no way of correcting it at this stage) the betting market line runs direct (interpolates) from 26thMay to the final 8thJune 2017 estimate.

Continue reading How did the election forecasts do in 2017?

Second combined forecast for the 2019 general election

By Stephen Fisher, John Kenny and Rosalind Shorrocks

There are still just a few different forecasts for the general election. Perhaps the big changes during the 2017 campaign have made people more hesitant about predicting early this time. Perhaps the problems of the polls in 2015, 2016 and 2017 have put some off the idea entirely. (More on this in our post-mortem for the 2017 combined forecast which we’re still working on.) Nonetheless, this is the second of our weekly blogs where we review the different forecasts from different methods and combine them into an overall forecast.

Here we aggregate seats and vote share forecasts from a variety of sources including betting markets, polls, statistical forecasting models and citizen forecasts. As well as updating weekly, the methodology (as detailed below) might well evolve. So comments and suggestions on our approach and for new forecasts to include are welcome.

Just as they did last week, all the different sources point to the Conservatives being comfortably the largest party, with heavy losses for Labour and modest gains for the Liberal Democrats and Scottish National Party (SNP). Last week the betting markets suggested a smaller Tory tally than did the forecasting models. The models haven’t changed much but the betting markets have moved into line with forecasts. On average across betting markets, and complex and simple models, the Tories are expected to win with a comfortable majority of 60, barely changed from 57 last week. All three sources now suggest a majority of 50 or more.

Seats Betting Markets Complex models Simple models Average
Con 352 360 353 355
Lab 210 201 193 201
LD 37 25 30 31
Brexit 0 0 0
Green 2 1 1 1
SNP 46 48 50 48
PC 5 4 4 4
Con majority 54 70 56 60

Conservative Seats - 13th Nov

Continue reading Second combined forecast for the 2019 general election

Election analysis and forecasting