Category Archives: US elections

Some thoughts on the US election forecasts 2020

By Stephen Fisher, 3rd November 2020.

Americans are evenly divided in their expectations of who will win today’s presidential election. YouGov found 40% expect Joe Biden to win, 39% think Donald Trump will win and 21% are “Not sure” (see p59 here). By contrast, in 2016 a similar question about expectations of the outcome from New York Times/CBS News polls here had 56% expecting Hilary Clinton to win and just 33% thinking Donald Trump would. Differences in question wording means we should be cautious about detailed comparison, but it appears that Americans collectively are much more unsure of the outcome of this election than they were last time.

They may be more cautious because the majority of them were wrong last time. Then their expectations were in line with the polls and the forecasters, some of whom were clearly over-confident for Hilary Clinton, as I argued before the fact

The problems of the polls in 2016 were diagnosed and methodology has improved. In particular, many polls are now weighting their samples so that the numbers of people with different levels of education are represented in proportion to their prevalence in the population. Part of the problem with some of the state-level polls last time was that they did not do so. Without that education weighting the polls this year would be showing even bigger Biden leads.

Some of the most over-confident forecasters from 2016 are no longer forecasting (including the Huffington Post), while others, such as Sam Wang, have adjusted their models and de-emphasised the predicted probabilities. 

Nate Silver’s FiveThirtyEight forecasts did best in 2016, mainly by having a model with more prediction uncertainty and so a larger probability of Clinton winning. Despite that, one of the most prominent debates about the election forecasts this year has been prompted by the superb statistician Andrew Gelman querying whether the FiveThirtyEight model has included too much uncertainty. 

Andrew Gelman is part of The Economist forecasting team. Their model has consistently had more confidence in a Biden win. It now gives Biden a 97% chance, to FiveThirtyEight’s 89% chance, of winning. Their central forecasts are extremely close, it is their uncertainty levels that differ.

More recently, Andrew Gelman has posted this on the effects of using Normal versus t distributions for the tail probabilities, and this on the value of learning from experience as a forecaster, and a fair few interesting posts about election forecasting in between. I can see the case for the modelling choices in The Economist model, but somehow the published level of uncertainty from FiveThirtyEight feels more reasonable to me. Since there will be sources of uncertainty that are not measurable from the polling and historical data, I think it is okay to err on the side of allowing more uncertainty when making choices between different justifiable modelling options. 

Another forecaster that has adapted their methods since 2016 is YouGov. Their MRP model forecast has a slightly higher forecast electoral college tally for Biden (364) than does The Economist (356) or FiveThirtyEight (348). The difference is based on a slightly larger predicted Biden lead in votes (9 points for YouGov compared with 8 points for FiveThirtyEight). 

YouGov has not published a predicted probability for a Biden win, but from their graph of simulations it seems that they have in the order of a 10% chance of a Trump win. That is roughly the same as FiveThirtyEight despite a lower central forecast for Trump. Moreover, in 2016 the corresponding YouGov model put very little probability on a Trump win despite a much closer forecast for the national vote share margin. The additional uncertainty estimation this time seems like an improvement. 

It is worth noting that whereas the FiveThirtyEight, Economist and similar models depend on a plethora of polls funded by others, YouGov build their model on their own polling data. They might do a very large poll, but the YouGov sample size would still pale by comparison with the cumulative sample size of all the other campaign polls at the national and state levels. YouGov’s ability to refine their methodology is important for understanding the potential for getting effective forecasts from more modest amounts of polling data.

Using perhaps the most data of all, PollyVote aggregate not only polls but models, betting markets, citizen expectations and expert forecasts. That website has very helpful descriptions of how the different data sources and models work with links to research on how they have fared in the past. Their overall combined forecast is for a Biden victory, but with a smaller margin than the polls and poll-based models point to. This is largely because citizens suggest the race is close (as discussed above) and the betting markets and experts are cautious about the polls. 

One model that points in completely the opposite direction from nearly all others is Helmut Norpoth’s Primary Model. It predicts a comfortable Trump victory because he is an incumbent president (they usually win) and because Biden did not do well in early primary contests, suggesting he is a weak candidate. The Primary Model has a very good track record of successful forecasts. It at least provides a basis for claiming that a Biden win would be remarkable by historical experience. 

Overall, the evidence from the (apparently improved) polls is a very strong indication that Biden will win, and win very comfortably. The dearth of media coverage of election forecasts and the cautious coverage of opinion polls this campaign, as a result of their failure to anticipate Trump’s victory in 2016, seems to have led to public uncertainty. If the polls are right, and especially if Biden does a bit better than they suggest, it will come as a shock to many, especially Republican voters who mainly expect Trump to win (see p59 here).

Where has the swing to Biden come from?

Evidence from the polls seems to be that partisan polarization between self-described liberals and Democrats on the one side and conservatives and Republicans on the other has worsened since 2016, with both groups more strongly backing their candidate than they did in 2016. What is helping Biden most is a big swing among those who describe themselves as independent and/or moderate. Since that big swing is mainly among whites and it is towards the Democrats the ethnic divide has narrowed as white vote intention moves closer to the strongly Democratic Latino and solidly Democratic Black vote.

At a macro level it looks like swing at the state level might be fairly uniform. The table below shows my quick calculations for the change in the lead in some of the larger more marginal states. The average lead for Biden in them is a bit bigger for YouGov than FiveThirtyEight, in keeping with the one-point larger lead YouGov expects in the national share of the vote. Relative to Hilary Clinton’s actual lead over Donald Trump in 2016, the predicted change this year is relatively similar in these states. There is variation between states in the extent of the swing expected, but the differences are not huge and they are not all consistent between the two forecasters. 

Table: FiveThirtyEight and YouGov predictions for selected swing states

 Biden lead  Biden lead – actual Clinton lead 
North Carolina1.
Note: All figures are in percentage points.

That said, both YouGov and FiveThirtyEight expect bigger than average swings in Michigan and Wisconsin, and a smaller than average swing in Florida. This would be the same as in 2016 but in the opposite direction. I have not had time to analyse whether there is a broader pattern of negative correlation between 2012-16 swing and 2016-20 swing, but it would not be surprising if the places that swung most to Trump in 2016 swing most heavily away from Trump today. 

House of Representatives

The House is largely expected to stay Democrat controlled. FiveThirtyEight give the party a 97% chance of retaining control. In 2018, the Democrats won 235 seats to the Republicans 199 on an 8.6 point lead in the popular vote, quite a margin to overturn. Currently, the Real Clear Politics generic congressional vote poll average has a Democrat lead of 6.8 points. Similarly, FiveThirtyEight predict a popular vote margin of 6.3 points for the Democrats. Since this is down what it was on 2018 it is remarkable that the Democrats are also predicted to win slightly more seats (239). 

If this comes to pass, I for one will be wondering whether the failure of a swing to the Republicans to yield any net seat gain will be due to incumbency advantage, especially “sophomore surges” given that most of the Republican target seats are being defended by first-term Democrat incumbents. 

While a swing to the Republicans in the House without net Republican seat gains would seem to be a sign of problematic unresponsiveness of the electoral system, it would represent a partial unwinding of a pro-Republican bias in the distribution of seats given votes and so arguably a sign of the system working more fairly.


There is much more uncertainty as to whether the Senate will flip from Republican to Democrat. Of the 35 Senate seats up for election this time 21 are begin defended by Republican incumbents, and incumbency effects in US Senate elections are very strong. However, the seats up this time were last fought in 2014, which was a midterm year when Obama was president. The big c.9.7 point swing to the Republicans then appears to be being partially undone. Also, the Democrats have more of the seats not up for election: 35 out of 65 including two independents caucusing with the Democrats.

Sam Wang’s Princeton Election Consortium is predicting the Senate to go 53 D 47 R, while Fivethirtyeight’s central prediction is 51.5 D 48.5 R, with a 75% chance of Democratic control. 

Overall, based on the polling evidence the Democrats are likely win control all three branches of government, with the Senate the least certain to go their way.

Further links:

PS: Political Science & Politics academic forecasts (from September 2020) and related articles:

The review, by Alfred Cuzán, of previous forecasts in that series is especially worth a read. claims to have a powerful model based on polls and media tracking that improves on polls only. 

Patrick English has a very confident model for Biden here.

Forecasts for the US Midterm Elections 2018

By Stephen Fisher.

The most striking thing about the forecasts for today’s midterm elections in the United States is that they have been much less talked of in the media than in previous campaigns. This is partly because in 2016 most of the forecasters put very high probabilities (90%+) on Hilary Clinton winning the presidency. (See here for a post-mortem.)

This post reviews the main statistical model based forecasts for the US House and Senate, with some discussion of the methodology and comparison with other forecasts. Overall, and as usual, there is not much variation between the forecasters in their central forecasts. They all point to the Democrats taking control of the house and the Republican retaining control of the Senate. The striking exception is a Gallup poll suggesting 50% think the Republicans will retain control of the House and only 44% think the Democrats will win it.

Despite the forecasts differing from the expectations of the American people, the forecasts appear to have been widely accepted in the media. So much so that some journalists suggest it will be a vindication of Donald Trump if Republicans maintain control of the House. However, if that happens it will most likely be despite a clear lead for the Democrats in the popular vote. In which case, it would be the electoral system, not Trump, that thwarts the Democrats. Meanwhile, if there are net Republican gains in the Senate it will be primarily because the Democrats are defending a big haul from 2012.

House forecasts

As James Campbell has noted, in all but three midterm elections since 1900 the President’s party has lost seats. Since 1950 the average loss has been 24 seats. The Democrats need to make net gains of 23 or more to take control.

Continue reading Forecasts for the US Midterm Elections 2018

Why was Trump elected?

by Stephen Fisher.

This question can be addressed from various angles and at different levels. I will start with the basic features of the result, then discuss the patterns of change since 2012, all before trying to address the bewilderment of many who find it hard to understand how anyone could vote for him.

Trump won the presidency by holding on to states that Romney won and winning some states that Obama previously won, including Iowa, Florida, Ohio, Pennsylvania and Wisconsin. He did this by a combination of a national vote swing of around 1.5 points since 2012, plus above average swings in some key battleground states, such as Pennsylvania 3 points, Michigan 5 points and Iowa 10 points.*

At the same time, Trump lost the national popular vote but won the presidency by narrowly winning Michigan, Wisconsin, Pennsylvania and Florida with leads of less that 1.5 points each. So, had the swing to Trump in these states been smaller by just 1 point then Hilary Clinton would have won outright.  This was a close election. Continue reading Why was Trump elected?

What to make of the US election forecasts?

By Stephen Fisher

The majority of forecasts point to Hilary Clinton winning tomorrow’s US presidential election. Several of the poll, market and expert forecasts with probabilities for who will win are helpfully summarised by the New York Times here.  The polls-based predictions are all, apart from one, pretty confident that Clinton will win. At the time of writing, Drew Linzer’s model at Daily Kos puts the probability of a Clinton win at 87%, HuffPost has 98% and Sam Wang at the Princeton Election Consortium estimate is as high as 99%. The New York Times’ own model is slightly less confident, on 84%. The exception is Nate Silver’s FiveThirtyEight model which puts Clinton’s chances at just 67%.

The lower probability for Clinton in the FiveThirtyEight model is partly (by comparison with most but not all models) due to 538 having a smaller estimate of her likely lead in votes. By comparison with HuffPost, for instance, this is because FiveThirtyEight put more weight on more recent polls that haven’t been so great for Clinton in wake of the FBI announcement last week of further investigations into her emails.

But the discrepancy is mostly due to the FiveThirtyEight model allowing for more uncertainty. So 538 have a higher probability of a Clinton landslide than other models as well as a higher probability of a triumphant Trump. Continue reading What to make of the US election forecasts?

Republican seat gains in the House off a swing to Democrats since 2010?

by Stephen Fisher

There was clearly a swing to the Republicans since 2012 in yesterday’s US midterm elections. However, the most comparable recent election for the House contest was not the general election but the 2010 mid terms.

The Democrats seem to have recovered votes since the 2010 big wave Republican takeover. According to yesterday’s exit polls, there has been a small (1 %) swing to the Democrats. And yet, the number of Republican House seats has increased from 242 in 2010, to a likely (at the time of writing) final tally of around 248 this year. Continue reading Republican seat gains in the House off a swing to Democrats since 2010?

A guide to the US midterm election forecasts

Stephen Fisher and Jonathan Jones

Tomorrow Americans go to the polls for the midterm elections. All 435 seats in the House of Representatives are up for election, as well as 36 Senate seats and a host of state and local offices.

The historic tendency for the President’s party to lose ground in midterm elections, is sufficiently strong to say that there is virtually no chance of the Democrats regaining control of the House. Meanwhile the experience of previous midterm elections with a second-term President suggests that the Republicans should have a decent of winning the extra six seats they need to take control of the Senate.

Not only history but also political and economic circumstances in recent months, and especially current polls for individual Senate races suggest the Republicans have, according to the main forecasters, at least a two-thirds chance of achieving a Senate majority.

We are not attempting to forecast the outcomes of any of these elections, but several others with excellent track records for US election forecasting are. This article purely provides some introduction and links to the forecasts and offers some commentary from a British election forecasting perspective. We consider the House forecasts before turning to those for the Senate. Continue reading A guide to the US midterm election forecasts

US Elections 2012: Some initial thoughts on the results and polls

The big political story is no change. Obama is still the president, there looks like there will be very little change in the party composition of the Republican held House and Democrat led Senate. Given the polarisation of US parties at the moment, this seems to be a recipe for continued legislative gridlock. The last Congress was the least productive in over 120 years. Continue reading US Elections 2012: Some initial thoughts on the results and polls