Category Archives: Combined forecast

Final combined forecast for the 2019 general election

By Stephen Fisher, John Kenny and Rosalind Shorrocks

Since our first combined forecast at the start of the campaign, the number of forecasts for this general election has grown substantially. All of the combined forecasts – seats, vote shares, and probabilities – are pointing to a Conservative majority. However, some individual forecasts do predict a hung parliament, and there is variation within each forecast type over how certain this majority is, and how large it is predicted to be.

SEATS

Seat projections from the betting markets, complex models, and simple models are all very similar, forecasting a Conservative majority of between 343 and 351 seats. The average number of seats across all forecasts that the Conservatives are expected to win – 341 – is slightly lower but ultimately very similar to the forecast last week.

Since last week the Political Studies Association have published their Expert Survey, in which the average expected number of Conservative seats suggests a hung parliament with the Conservatives just shy of a majority. It is interesting that the experts surveyed by the PSA predict the Conservatives will win fewer seats than is currently suggested by the polls. Perhaps they are factoring in the same kind of late-campaign changes as observed in 2017 – although it should be noted that when a similar kind of survey was run for the EU referendum in 2016, the average predicted vote share for Remain and Leave amongst experts was the furthest away from the actual result than any of the other types of forecast. They also predicted a Conservative majority in 2017, although that prediction was made much more earlier in the campaign when the Conservatives had considerable leads over Labour in the polls.

Seats Betting Markets Complex models Simple models Experts Average
Con 346 343 350 324 341
Lab 221 225 219 233 224
LD 18 17 18 25 19
Brexit 0 0 0 2 1
Green 1 1 1 2 1
SNP 43 44 41 42 43
PC 4 3 3 4 4
Con majority 42 37 49 -2 31

 

Conservative Seats - 11th December

The similarity between the seat projections from most sources hides considerable variation within one particular forecast type – complex models. These models range from predicting 311 Conservative seats to 366 – the difference between a hung parliament and a healthy Conservative majority. They also range between 190 and 268 for Labour. It is particularly noteworthy that the voter expectation model, from Murr, Stegmaier, and Lewis-Beck, which uses citizen forecasts to predict the number of seats, forecasts one of the highest number of Conservative seats (360) and the lowest number of Labour seats (190). This is in contrast to our implied probability calculated from the citizen forecasts, which suggest that citizens are in general the least convinced about the likelihood of a Conservative majority compared to other forecasting methods. This suggests these surveys also suffer from being open to multiple interpretations and methods of analysis, as well as the question wording effects we discussed last week.

Continue reading Final combined forecast for the 2019 general election

Fifth combined forecast for the 2019 general election

By Stephen Fisher, John Kenny and Rosalind Shorrocks 

Since our update last week there have been several new forecasts, most notably including the YouGov MRP (multilevel regression and post-stratification) model. That was a nowcast rather than a forecast, but the same is true of most of our “forecasts”. More on differences between forecasting models below, along with some observations about intriguing question wording effects for citizen forecasts.

But first, overall, the seats projections overall have tightened for the Conservatives, who are down from a 353 average last week to 346 this week, while Labour are up from 209 to 218. The Liberal Democrat forecast total has dropped yet again (from 23 to 19). Now they are estimated to return fewer MPs than they had going into the election (20), but still more than the number of seats they won in 2017 (12).

Seats Betting Markets Complex models Simple models Average
Con 343 347 348 346
Lab 220 218 217 218
LD 19 18 19 19
Brexit 0 0 0
Green 1 1 1 1
SNP 45 45 44 45
PC 5 4 3 4
Con majority 36 45 46 42

Conservative Seats - 4th December

There is now remarkably little difference between the betting markets, complex and simple models in the expected size of the Conservative majority. Particularly striking is that on average the complex models differ by only a seat for each party from the simple uniform change projections based on the average of the opinion polls.

Continue reading Fifth combined forecast for the 2019 general election

Third combined forecast for the 2019 general election

By Stephen Fisher, John Kenny and Rosalind Shorrocks 

Once again all three sources of seat forecasts suggest the Conservatives are heading to a comfortable majority, while Labour are on course for a result on par with their previous post-war low of 209 seats in 1983. The Liberal Democrat forecast has been dropping steadily, so that they are now expected to end up with only a few more MPs than the twenty they had when they chose to support the election.

Seats Betting Markets Complex models Simple models Average
Con 346 354 363 354
Lab 209 211 200 206
LD 30 22 22 25
Brexit 0 0 0
Green 2 1 1 1
SNP 46 46 44 45
PC 4 4 4 4
Con majority 42 58 75 58

Con seats 20 NovLib Dem seats 20 Nov 

Continue reading Third combined forecast for the 2019 general election

How did the election forecasts do in 2017?

By Stephen Fisher, Rosalind Shorrocks and John Kenny.

Lots of the forecasts for the 2017 British general election were wrong in pointing to a Conservative majority. Most even suggested an increased majority when the party actually lost one. The well-known exceptions are the exit poll and YouGov multilevel regression and post-stratification (MRP) model. Less commonly remembered are the polls that were within a point of the Conservative-Labour lead and so, by standard uniform change calculations, provided an indication that the Tory majority was in jeopardy.

During the election campaign we ran an exercise combining the different forecasts for shares of the vote, seat tallies, and probabilities of a Conservative majority. Inevitably the combined forecast is only as good as the average of what goes into it, and that average was poor. But since the exercise involved classifying and averaging forecasts by the methodology they used, it is possible to reflect on which types of forecasting method performed better than others.

The figure below shows predictions of the Conservative share of the vote over the course of the campaign from the Political Studies Association (PSA) expert survey, betting markets, forecasting models and opinion polls. Strictly speaking polls are snapshots not forecasts, but the final polls performed as well as the experts and better than all the other methods. Consistently the worst source of predictions for the Tory share were the betting markets.

Conservative Vote Share 2017Note: Due to an error in our 2ndJune 2017 betting market calculation (and no way of correcting it at this stage) the betting market line runs direct (interpolates) from 26thMay to the final 8thJune 2017 estimate.

Continue reading How did the election forecasts do in 2017?

Second combined forecast for the 2019 general election

By Stephen Fisher, John Kenny and Rosalind Shorrocks

There are still just a few different forecasts for the general election. Perhaps the big changes during the 2017 campaign have made people more hesitant about predicting early this time. Perhaps the problems of the polls in 2015, 2016 and 2017 have put some off the idea entirely. (More on this in our post-mortem for the 2017 combined forecast which we’re still working on.) Nonetheless, this is the second of our weekly blogs where we review the different forecasts from different methods and combine them into an overall forecast.

Here we aggregate seats and vote share forecasts from a variety of sources including betting markets, polls, statistical forecasting models and citizen forecasts. As well as updating weekly, the methodology (as detailed below) might well evolve. So comments and suggestions on our approach and for new forecasts to include are welcome.

Just as they did last week, all the different sources point to the Conservatives being comfortably the largest party, with heavy losses for Labour and modest gains for the Liberal Democrats and Scottish National Party (SNP). Last week the betting markets suggested a smaller Tory tally than did the forecasting models. The models haven’t changed much but the betting markets have moved into line with forecasts. On average across betting markets, and complex and simple models, the Tories are expected to win with a comfortable majority of 60, barely changed from 57 last week. All three sources now suggest a majority of 50 or more.

Seats Betting Markets Complex models Simple models Average
Con 352 360 353 355
Lab 210 201 193 201
LD 37 25 30 31
Brexit 0 0 0
Green 2 1 1 1
SNP 46 48 50 48
PC 5 4 4 4
Con majority 54 70 56 60

Conservative Seats - 13th Nov

Continue reading Second combined forecast for the 2019 general election

First combined forecast for the 2019 general election

By Stephen Fisher, John Kenny and Rosalind Shorrocks (Universities of Oxford, Southampton and Manchester respectively)

There aren’t so many different forecasts for the general election out yet, but enough to start looking at how they compare. This is the first of hopefully weekly blogs where we review the different forecasts from different methods and combine them into an overall forecast. At the moment, the polls and seats forecasts suggest a comfortable Conservative majority but citizens and betting markets are not so sure it will happen.

The idea of combining forecasts from different sources has a good track record, though it has to be admitted that our attempts for the 2017 general election and the 2016 Brexit referendum did not work out well. We will write more about those experiences soon. Also worth noting is the experience of the pollyvote.com combined forecast of the US presidential elections.

Here we aggregate seats and vote share forecasts from a variety of sources including betting markets, polls, statistical forecasting models and citizen forecasts. As well as updating weekly, the methodology (as detailed below) might well evolve. So comments and suggestions on our approach and for new forecasts to include are welcome.

Continue reading First combined forecast for the 2019 general election

Final Combined Forecast for GE2017

By Stephen Fisher, John Kenny and Rosalind Shorrocks.

Our final combined forecast is for a Conservative majority 66. This estimate is down only slightly from 70 on Friday last week, but well down on the 132 majority we first forecast on 12th May. All three sources on average each suggest that net gains for the Conservatives will largely be at the expense of Labour and the SNP, with tallies for other parties little changed from 2015.

Seats Betting Markets Complex models Simple models Average
Con 371 361 343 358
Lab 199 214 228 214
LD 11 7 7 8
UKIP 0 0 0 0
Green 1 1 1 1
SNP 46 46 50 47
PC 3 3 3 3
Con majority 92 72 36 66

The combined probability of a Conservative majority is now 72%. This is little changed from last week but down from 91% in our first combined forecast. The probability of a 100+ majority Conservative landslide is, at 11%, the same as last week but well down on the 71% we reported on 12th May. Betting markets are the most confident of a Tory majority. Citizens collectively express the most uncertainty, with both the highest probability on a Tory landslide and the second highest probability on the Conservatives losing their majority. Taking the most recent poll from each pollster, only just over half of them show the Conservatives with the lead of 6 points or more that would be needed for the party to win a majority on uniform change calculations. The 54% figure is down from 63% last week, but it reflects a change in who has done a poll this week more than how the pollsters from last week have seen their figures change.

Betting markets Models Polls Citizen forecast Average
Conservative Majority 0.83 0.82 0.54 0.69 0.72
Conservative landslide 0.17 0.00 0.17 0.11

For our final forecast we have taken a straight average of the final polls published at the time of writing instead of relying on other poll aggregators. This average is for a strikingly similar Conservative lead as in the actual outcome last time 2015, 6.5 compared with 6.6 points. But the final polls last time pointed to a tie. The markets and models predict that the polls are over estimating Labour and under estimating the Conservatives again. Overall our combined forecast this week now suggests that the Conservatives will win with a 9.6 point lead. This estimate is down only slightly from 10.2 points last week. But there has been a massive drop from the 18.7 point lead predicted in our first combined forecast on 12th May.

Final Poll average Betting markets Models Average
Con 43.0 45.8 44.3 44.4
Lab 36.5 34.7 33.1 34.8
Lib Dem 7.6 8.3 9.7 8.5
UKIP 4.4 3.4 4.7 4.1
Green 2.1 2.2 2.1
SNP 4.0 4.1 4.0
PC 0.6 0.8 0.7
Con-Lab lead 6.5 11.1 11.2 9.6

Methodology

The main changes in method this week are as follows.

Nigel Marriott in his final model now includes a forecast for party vote shares as well as the previous seat forecasts and so this is included; we have also added new models from Election Data, Elections Etc, Janta-Lipinski, Kantar Public, Michael Thrasher, Andreas Murr et al and Number Cruncher Politics. YouGov’s forecast model has been removed because they have officially dropped it. Betting market data for the forecasted party vote share of Labour, the Liberal Democrats and UKIP is gathered from Betfair/Betfair Exchange where such data is not available on Oddschecker.

Our polling average is now a simple average of the final polls.

As there have been five citizen surveys published since the 31st May, we have decided to limit the inclusion criteria for this so that only the most recent citizen survey from each survey company during this period is included.

Overall description of the method

Our basic approach is to combine forecasts by averaging them within each category and then taking the average across categories. Since the different sources do not all present equally clear figures that can be averaged on a like for like basis we have made various judgement calls on how to treat the data.

Historically the idea of combining forecasts from different sources has had a good track record, though it has to be admitted that our attempt to do one for the EU referendum did not work out well. Most recently the pollyvote.com combined forecast of the US presidential election last year was 2 points out on the share of the vote. 

Polls

Our polling average is now a simple average of the final polls. Previously for vote shares, we use the various available polling averages, or ‘polls of polls’, and take their average. We exclude polling averages for whom the most recently published polling average is more than a week old. There are a number of different polling averages. They are in truth nowcasts rather than forecasts, but we are in effect treating them as forecasts. Some admittedly are quite sophisticated, allowing for pollster (aka house) effects, but they are nonetheless estimates of current public opinion and not future votes.

We do not attempt to say what seats outcome is implied by polls (that is the job of the modellers). However, since statistical models are rarely if ever clear about the probabilities their models place on key events like a Conservative majority and a 100+ majority, we have included in the probabilities table some pseudo-probabilities from the polls. Taking the most recent poll published by each pollster in the last week, we calculate the proportion showing a Conservative lead over Labour of more than 6 points as the pseudo-probability of a Conservative majority, but we allow polls with exactly a 6 point lead to contribute a 0.5 to the average. Using the same polls, we use the proportion showing a Conservative lead over Labour of 16 points or more as the pseudo-probability of a Conservative landslide. These thresholds of 6 and 16 points are based on what would be required under uniform swing assumptions for the Conservatives to win a bare majority and a 100+ majority respectively.

Betting markets

There are numerous betting markets for the various outcomes in the election. We have taken those that are most helpful for the four forecasts we want to produce. For seat shares, we take the mid-point of the spread as the seat share, and average these mid-points between different sources. Note that the markets imply fewer seats forecast than there are actually are in the House of Commons. This is because the markets are separate for each party and do not need to be consistent collectively.

For vote share, we use betting markets for the Conservatives, Labour, the Liberal Democrats and UKIP. (Vote share markets for other parties are unavailable either on the betting market aggregation site Oddschecker of Betfair/Betfair Exchange). Odds are given for 5-point ranges of vote share. We take a weighted sum of the mid-points of these ranges where the weights are the implied probabilities. For the top and bottom options we use 2.5% above/below the upper/lower bound (e.g. 52.5 for “Above 50” and 27.5 for “Below 30”). The weighted sum is calculated just using the three categories with the largest implied probabilities, because the probabilities for other categories are so small and unstable. For UKIP we use just the two most likely categories.

For the probability of a Conservative majority we give an average of the implied probability from sites offering this market. For the probability of a Conservative landslide, we use the combined prices PredictIt that the Conservatives will win 370-379 seats, 380-389 seats, and 390 or more seats. This really represents a majority of 90 or more but that was as close as we could get to 100.

Statistical models

There are numerous statistical forecasting models this year. We have divided them into two categories: simple (poll average plus uniform swing seats projection) and complex (anything more elaborate than the simple models, although they are not necessarily particularly complex). Some make adjustments for long run differences between pollsters, for constituency variation, and some estimate by how much things will differ between current polls and the eventual result. Chris Hanretty’s forecast at electionforecast.co.uk does all of these things. Within these categories we simply average the available estimates of seats and shares. In addition, as discussed above, we have separated those models based on the local election results into a separate category.

We should note that not all of the models are the modellers’ favourite. Some are counterpoints to their main models for comparison. We have included these on the basis that they are still talked of and expected to be reasonable estimates, for example, at Electoral Calculus, Martin Baxter has a local election results based model. We have not excluded any models based on our judgement of quality, but they do have to be statistical models as opposed to personal guesses.

Citizen forecasts

Some polls ask people what they think that the outcome will be on June 8th. Different pollsters use different survey questions but they can be combined to generate pseudo-probabilities. We use the proportion of poll respondents who think the Conservatives will win/there will be a Conservative majority, excluding don’t knows and re-percentaging, as the pseudo-probability of a Conservative majority. We similarly use the proportion of poll respondents who think that there will be a Conservative landslide/the Conservatives will win more than 100 seats, for the probability of a Conservative landslide.

Note: Estimates come from around 9am on the morning of  8th June 2017. In all seat estimates, the Speaker’s seat is counted as Conservative.

 Sources

There sources we used are listed below in no particular order. Please let us know of any that you think we have missed or misclassified. Some polling averages we know of were not included because they were more than a week old.

Prediction markets:
IG
Predictit
Sporting index
Oddschecker
Betfair Predicts

Betfair

Betfair Exchange

 

Complex Forecasting Models:
Chris Prosser

ElectionForecast.co.uk (Chris Hanretty)
Electoral Calculus (main and local election forecast)
Election Data

Elections Etc

Forecast UK 
Janta-Lipinski

Kantar Public

Lord Ashcroft

Michael Thrasher
Murr, Stegmaier and Lewis-Beck

Nigel Marriott

Number Cruncher Politics

PME Politics (Patrick English)

UK-Elect

 

Simple forecasting models (polling average + uniform swing):
Electionpolling
Principalfish

 

Polling Averages (less than a week old):
Telegraph
FT
Britain Elects
Electionforecast.co.uk
NewStatesman
Electionpolling
Principalfish
The CrossTab

 

Citizen forecasts:
YouGov (here)
ComRes (here)
ICM (here)

Opinium (here)

Survation (here)

The three authors are equal contributors and our names are in alphabetical order.

 

Final combined EU Referendum forecast

by Stephen Fisher and Rosalind Shorrocks.

This forecast is based on data collected late on 22nd June. Yet again this method suggests Remain is most likely to win but there is a considerable chance that Leave may win. The probability that Remain will win is up from 62.3% on Sunday to 66.5% now. This reflects both better opinion polls in recent days and so polling models more favourable for Remain, and greater market confidence in a status quo victory.
The only component forecast that is less favourable to Remain is the citizen forecasts, and that is because we have restricted our sample to citizen forecasts over the last week. They are noticeably more equivocal.
The forecast share of the vote is once again little changed, with Remain predicted to get 53.3% and Leave 46.7% per cent of the vote.
Remain % share Leave % share Probability Remain wins
Betting markets 53.5 46.5 76.7
Prediction markets 73.4
Citizen forecasts  52.0  48.0 55.2
Expert forecasts 55.1 44.9  62.0
Volunteer forecasts 54.0 46.0 74.0
Polls 50.6 49.4 55.6
Poll based models 52.5 47.5 68.5
Non-poll based models 55.6 44.4
Combined forecast (mean) 53.3 46.7 66.5

(Individual forecasts collected on the evening of 22nd June 2016.)

There is another poll due during polling day but that would be unlikely to change our polling average. The markets may change, possibly dramatically during the day if people on the betting and prediction markets are following trends financial markets.

METHODOLOGY Continue reading Final combined EU Referendum forecast

Updated combined EU Referendum Forecast

by Stephen Fisher and Rosalind Shorrocks.

We offer our condolences to the family and friends of Jo Cox MP. We hope that it is not disrespectful of us to post this updated forecast now that the official campaigns have resumed.

Remain % share Leave % share Probability Remain wins
Betting markets 52.0 48.0 66.5
Prediction markets 65.6
Citizen forecasts  52.0  48.0 62.1
Expert forecasts 55.1 44.9  62.0
Volunteer forecasts 54.3 45.7 71.3
Polls 48.5 51.5 46.0
Poll based models 50.5 49.5 55.0
Non-poll based models 55.6 44.4
Combined forecast (mean) 52.6 47.4 62.3

(Individual forecasts collected on 18th June 2016.)

METHODOLOGY Continue reading Updated combined EU Referendum Forecast