By Stephen Fisher.

The most striking thing about the forecasts for today’s midterm elections in the United States is that they have been much less talked of in the media than in previous campaigns. This is partly because in 2016 most of the forecasters put very high probabilities (90%+) on Hilary Clinton winning the presidency. (See here for a post-mortem.)

This post reviews the main statistical model based forecasts for the US House and Senate, with some discussion of the methodology and comparison with other forecasts. Overall, and as usual, there is not much variation between the forecasters in their central forecasts. They all point to the Democrats taking control of the house and the Republican retaining control of the Senate. The striking exception is a Gallup poll suggesting 50% think the Republicans will retain control of the House and only 44% think the Democrats will win it.

Despite the forecasts differing from the expectations of the American people, the forecasts appear to have been widely accepted in the media. So much so that some journalists suggest it will be a vindication of Donald Trump if Republicans maintain control of the House. However, if that happens it will most likely be despite a clear lead for the Democrats in the popular vote. In which case, it would be the electoral system, not Trump, that thwarts the Democrats. Meanwhile, if there are net Republican gains in the Senate it will be primarily because the Democrats are defending a big haul from 2012.

House forecasts

As James Campbell has noted, in all but three midterm elections since 1900 the President’s party has lost seats. Since 1950 the average loss has been 24 seats. The Democrats need to make net gains of 23 or more to take control.

Economic growth and unemployment levels have been relatively good and Trump’s tax cut has been popular. Despite this his approval rating is only around 42% (by Charles Franklin’s latest estimate.) The historical record apparently shows that approval ratings are more important than economic factors as predictors of midterm election outcomes. That record however does not suggest that Trump’s approval rating are so bad that the Democrats ought, on that basis, to win enough seats to take control of the House. After all, Republicans managed to maintain control of the House in 2016 despite Trump having similarly bad approval ratings then. Part of the reason for this is the pro-Republican bias in the electoral system whereby the Democrats need about a 6 point lead in votes to take the lead in seats.

The table below gives a summary (in no particular order) of the model based forecasts for the House, along with a couple of betting markets and the academic forecasts that were presented at the American Political Science Association (APSA) conference in early September (and published here). The fact that the up to date forecasts are so similar to the much older ones is a sign of how little has changed over the course of the campaign: a conclusion that is also supported by the trend lines for those forecasts that have been regularly updated.

	Methodology	House Dem seats	Dem seats prediction interval (with probability)	Probability Dem control House
CNN	Polls and other info modelling	225	202-263 (95%)
Fivethirtyeight (Nate Silver et al)	Polls and other info modelling	233	215-253 (80%)	87.5%
CBS/YouGov (Lauderdale and Rivers et al)	MrP with YouGov data	225	212-238 (95%)	c.89%
The Economist	Polls and other info modelling	229		86%
Sky (Erikson, Wlezien and Bafumi)	Polls and history	221
Crosstab (Elliot Morris)	Polls and history	229		79%
Decision desk	Polls and history	233		96%
Iowa Electronic Markets	Betting market			82%
PredictIt	Betting market	225		68%
APSA academic forcasts
Abramowitz (Sept)		225		67%
Bafumi et al (July)		222		54%
Campbell (Aug)		239
Lewis-Beck & Tien (Aug)		239

Data collected 5^th November 2018.

Fivethirtyeight actually have three different forecasts: polls only “lite” version, classic and a deluxe version including the expert ratings. The classic is the one quoted in the table. The average seats forecast from the three are very similar, 231.5, 233 and 230 respectively. At this late stage all three should be dominated by the polls and so unsurprisingly pretty similar. They are more likely to be different earlier in the campaign, when the latter two put less weight on the polls. On this occasion, all three headline seat forecasts have been very similar, and unchanging, since August.

As this exchange between Nate Silver and Doug Rivers indicates, a key difference between the 538 (and similar) models and the MrP model estimates is more uncertainty around the central forecast in will poll-aggregation forecasts compared with MrP. While poll-aggregation models use the historical experience of polling error to inform the prediction uncertainty, the MrP model uses only the contemporary polling data and, effectively so far as I can tell, sampling theory. In 2016 the same YouGov team used a similar multilevel regression and post-stratification (MrP) model for the presidential election and the narrow prediction interval meant that they put an extremely low probability on the event of a Clinton win (see figure 2 here).Nate Silver’s model did better partly by having a lower central forecast (302 rather than 318 electoral college votes) but mainly by having wider prediction intervals. This time the two teams have similar probabilities for control of the house, but as a result of different prediction distributions. The narrower prediction intervals from the MrP model mean that CBS/YouGov put a smaller probability on a Republican hold despite predicting a smaller Democratic majority in their central forecast.

The only citizen forecast I know of is a Gallup poll that intriguingly shows 50% expect the Republicans and 44% expect the Democrats to win control of the House. This not only goes against all the other forecasts but also is strikingly at a time when polls were showing a 10-point lead in the generic ballot. The academic literature, and the figures in the linked table, suggest that such citizen forecasts have a fairly good record. In advance of the 2016 election, 33% of poll respondents thought that Trump would win, and only 56% thought Clinton would. While these figures do not constitute successful citizen prediction, they suggest a Trump victory was a much higher probability event than most of the forecasters indicated.

Good Judgement Open gives the Democrats just a 68% chance of winning the House. It is not clear from the website whether Philip Tetlock considers the forecasts on that website to be equally good as those from his “superforecasters” at his commercial website, or whether there are midterm forecasts available from the commercial site.

The betting markets look reasonably in line with the poll-based forecasts, probably because the market prices are partly informed by the polls and forecasters.

Senate forecasts

Despite predictions of substantial Democratic gains in the House, the Republicans are forecast to maintain control of the Senate. The main reason for this is that there are only nine Republican seats up for election. Also, as James Campbell noted, since they made gains in the previous three elections for this cycle of Senate seats (2012, 2006 and 2000), the Democrats are defending a relatively large tally. They did particularly well last time because that was the same time Obama was being re-elected as president. Although they only need two extra seats to take control, that means winning 80% of the seats up (28 of 35), and so winning in more hostile states.

As the summary of the state-by-state forecasts here shows, it is not a simple matter of expecting the Democrats to hold all the places they won last time and having better chance in states with retiring Republican senators. For instance, Heidi Heitkamp, who narrowly won North Dakota in 2012, is expected to lose. Democratic senators in a handful of other states are apparently facing tight races.

Indeed fivethirtyeight, predict that the Democrats are most likely to suffer a net loss of one seat, leaving them on 48 (out of 100). That model gives just a 19% chance of the Democrats of getting to the 51 seats they need to take control. The betting markets (e.g. PredictIt) similarly expect 48 seats for the Democrats, but suggest a slightly lower chance, 13%, of Democratic control.

Philip Tetlock’s at Good Judgement Open gives the Democrats an 18% chance of winning the Senate. Participants in the Monkey Cage midterm forecasting challenge give the Democrats a 17% chance.

Will the forecasts this time be any better?

The forecasts above are primarily driven by opinion polls and so their accuracy will depend heavily on the accuracy of the polls on which they depend.

Despite much talk beforehand of the polls potentially underestimating the Democrats, in the 2014 midterms the polls overestimated that party’s vote share and so also their seat tally. The Republicans gained 13 seats in the house, instead of the 8 predicted. For most of the forecasts, but not for Sky, a similar error this year would still see the Democrats take control of the house. That year the forecasters also underestimated the number of Republican Senate seats, by an average of two seats. A similar error this year would see the Republicans strengthen their control with three net gains.

That the poll-based forecasters failed to predict Donald Trump’s victory in 2016 is well known. For that race the national-level polls were broadly correct in pointing to a lead for Hilary Clinton in the popular vote, but some of the state-level polls were seriously out.

Also in 2016 the Republicans did better in both the House and the Senate than the polls and forecasters suggested. In the House, the Democrats made only 6 seat gains from a 2.3 point swing, when they were expected to take around 13 seats from a 3.6 point swing. In the Senate the Republicans dropped by only two seats from 54 to 52, not to 50 as the forecasters suggested. On average they wrongly had a higher probability of Democratic than Republican control.

So, in both 2014 and 2016 the forecasters underestimated the eventual Republican performance substantially. It could happen again this week. If it does, then the main question is over control of the House. Nate Silver notes that a systematic polling error is needed this time for Republicans to retain control of the House. He says, “if there’s a typical polling error of 2 to 3 percentage points and it works in Republicans’ favor, the House would be a toss-up.” Such an outcome, or even a little more error than that, in either direction, should not be a great surprise.

Given the shock two years ago, pollsters might have upped their game. It is not clear from this discussionof recent changes in polling practice that things have improved much. However, some of the issues in the American Association of Public Opinion Research (AAPOR) reporton the problems of 2016 are easy to solve, such as weighting for educational attainment. So there is reason to hope that forecasts will be more accurate this time.

Perhaps the hardest thing for pollsters to predict is turnout, and the partisan consequences of turnout. Some pollsters are experimenting with and publishing results based on different turnout prediction methods. Early voting and other indications suggestthat turnout will be unusually high. This might benefit Democrats.

Votes and seats

The meaning of vote shares and the relationship between votes and seats is complicated by the fact that, as Ben Lauderdale notes, there are 37 seats without a Republican candidate and four without a Democrat. He says that his own CBS/YouGov forecast Democratic vote share lead of of 5.5 points is based on the hypothetical situation where both parties have candidates in every seat and so they, “expect the actual national vote shares to be more like D+7.5 than D+5.5.” That is quite a difference in the lead. Theres is a corresponding issue of interpretation when comparing polls based on the so-called generic ballot and actual vote shares.

Nate Silver has noted that if the polls are right then the Democrats might win the popular vote by 8-9 points, a bigger margin than the Republicans achieved in 1994 and 2010. But that would not deliver a big majority in seats. His model also shows that there is a substantial probability that the Republicans will lose the popular vote but win the most seats, and very little chance that the Republicans will win both. The graph on how votes translate into seats is not clear on what these probabilities are, but they note that the Democrats are only favoured to win a majority if they win the popular vote by a margin of, as much as, 5.6 points.

By contrast, in 2014 the Republicans won a majority of 59 on a lead of 5.7. That a similar sized lead should produce no majority for the Democrats is a measure of the pro-Republican bias in the system. Another sign is that, in 2012, the Republicans won control of the House without winning the popular vote. That could happen again this year.

Thanks to Roberto Cerina, Rob Ford, and Matt Singh for pointers to various forecasts.