Category Archives: Forecast updates

2019 General Election, Combined forecast, Forecast updates, GE19 Combined Forecast

First combined forecast for the 2019 general election

2 Comments

By Stephen Fisher, John Kenny and Rosalind Shorrocks (Universities of Oxford, Southampton and Manchester respectively)

There aren’t so many different forecasts for the general election out yet, but enough to start looking at how they compare. This is the first of hopefully weekly blogs where we review the different forecasts from different methods and combine them into an overall forecast. At the moment, the polls and seats forecasts suggest a comfortable Conservative majority but citizens and betting markets are not so sure it will happen.

The idea of combining forecasts from different sources has a good track record, though it has to be admitted that our attempts for the 2017 general election and the 2016 Brexit referendum did not work out well. We will write more about those experiences soon. Also worth noting is the experience of the pollyvote.com combined forecast of the US presidential elections.

Here we aggregate seats and vote share forecasts from a variety of sources including betting markets, polls, statistical forecasting models and citizen forecasts. As well as updating weekly, the methodology (as detailed below) might well evolve. So comments and suggestions on our approach and for new forecasts to include are welcome.

Continue reading First combined forecast for the 2019 general election →

Forecast updates, Local elections

Forecasting Local Election Seat Gains/Losses 2018

2 Comments

by Stephen Fisher

My forecasting model for seat gains/losses at local elections has previously been a simple model based on change in party support in the polls. While the historical data show that changes in the percentage of the council seats that a party wins is reasonably strongly correlated with changes in that party’s poll share, that basis for forecasting this year would not work.

The Conservatives are up by 8 points and Labour up by 6 points in the polls since just before the 2014 local elections, when most the seats up for election this week were last fought. As in last year’s general election, support for both main parties has increased largely because of a big drop in UKIP support. Forecasting the parties separately based on changes in their respective poll shares would produce misleading predictions of big gains for both main parties. In first-past-the-post elections it is typically relative not absolute performance that matters for seat outcomes. Last year, despite winning more votes the Conservatives lost their parliamentary majority because the Labour vote share went up by more than did that for the Conservatives.

For this reason, my forecasting models this year are based on changes in the gaps between polls shares.  For the Conservatives, who have traditionally faced many contests with the Liberal Democrats, their leads over both Labour and the Liberal Democrats matter. For Labour, the model is primarily based on the Labour lead over the Conservatives. Meanwhile, for the Liberal Democrats, their changing opinion poll performance relative to the Conservatives, but not Labour, has historically been correlated with headline local election seat changes.

Continue reading Forecasting Local Election Seat Gains/Losses 2018 →

2017 General Election, Combined forecast, Forecast updates

Final Combined Forecast for GE2017

2 Comments

By Stephen Fisher, John Kenny and Rosalind Shorrocks.

Our final combined forecast is for a Conservative majority 66. This estimate is down only slightly from 70 on Friday last week, but well down on the 132 majority we first forecast on 12th May. All three sources on average each suggest that net gains for the Conservatives will largely be at the expense of Labour and the SNP, with tallies for other parties little changed from 2015.

Seats Betting Markets Complex models Simple models Average
Con 371 361 343 358
Lab 199 214 228 214
LD 11 7 7 8
UKIP 0 0 0 0
Green 1 1 1 1
SNP 46 46 50 47
PC 3 3 3 3
Con majority 92 72 36 66

The combined probability of a Conservative majority is now 72%. This is little changed from last week but down from 91% in our first combined forecast. The probability of a 100+ majority Conservative landslide is, at 11%, the same as last week but well down on the 71% we reported on 12th May. Betting markets are the most confident of a Tory majority. Citizens collectively express the most uncertainty, with both the highest probability on a Tory landslide and the second highest probability on the Conservatives losing their majority. Taking the most recent poll from each pollster, only just over half of them show the Conservatives with the lead of 6 points or more that would be needed for the party to win a majority on uniform change calculations. The 54% figure is down from 63% last week, but it reflects a change in who has done a poll this week more than how the pollsters from last week have seen their figures change.

Betting markets Models Polls Citizen forecast Average
Conservative Majority 0.83 0.82 0.54 0.69 0.72
Conservative landslide 0.17 0.00 0.17 0.11

For our final forecast we have taken a straight average of the final polls published at the time of writing instead of relying on other poll aggregators. This average is for a strikingly similar Conservative lead as in the actual outcome last time 2015, 6.5 compared with 6.6 points. But the final polls last time pointed to a tie. The markets and models predict that the polls are over estimating Labour and under estimating the Conservatives again. Overall our combined forecast this week now suggests that the Conservatives will win with a 9.6 point lead. This estimate is down only slightly from 10.2 points last week. But there has been a massive drop from the 18.7 point lead predicted in our first combined forecast on 12th May.

Final Poll average Betting markets Models Average
Con 43.0 45.8 44.3 44.4
Lab 36.5 34.7 33.1 34.8
Lib Dem 7.6 8.3 9.7 8.5
UKIP 4.4 3.4 4.7 4.1
Green 2.1 2.2 2.1
SNP 4.0 4.1 4.0
PC 0.6 0.8 0.7
Con-Lab lead 6.5 11.1 11.2 9.6

Methodology

The main changes in method this week are as follows.

Nigel Marriott in his final model now includes a forecast for party vote shares as well as the previous seat forecasts and so this is included; we have also added new models from Election Data, Elections Etc, Janta-Lipinski, Kantar Public, Michael Thrasher, Andreas Murr et al and Number Cruncher Politics. YouGov’s forecast model has been removed because they have officially dropped it. Betting market data for the forecasted party vote share of Labour, the Liberal Democrats and UKIP is gathered from Betfair/Betfair Exchange where such data is not available on Oddschecker.

Our polling average is now a simple average of the final polls.

As there have been five citizen surveys published since the 31st May, we have decided to limit the inclusion criteria for this so that only the most recent citizen survey from each survey company during this period is included.

Overall description of the method

Our basic approach is to combine forecasts by averaging them within each category and then taking the average across categories. Since the different sources do not all present equally clear figures that can be averaged on a like for like basis we have made various judgement calls on how to treat the data.

Historically the idea of combining forecasts from different sources has had a good track record, though it has to be admitted that our attempt to do one for the EU referendum did not work out well. Most recently the pollyvote.com combined forecast of the US presidential election last year was 2 points out on the share of the vote. 

Polls

Our polling average is now a simple average of the final polls. Previously for vote shares, we use the various available polling averages, or ‘polls of polls’, and take their average. We exclude polling averages for whom the most recently published polling average is more than a week old. There are a number of different polling averages. They are in truth nowcasts rather than forecasts, but we are in effect treating them as forecasts. Some admittedly are quite sophisticated, allowing for pollster (aka house) effects, but they are nonetheless estimates of current public opinion and not future votes.

We do not attempt to say what seats outcome is implied by polls (that is the job of the modellers). However, since statistical models are rarely if ever clear about the probabilities their models place on key events like a Conservative majority and a 100+ majority, we have included in the probabilities table some pseudo-probabilities from the polls. Taking the most recent poll published by each pollster in the last week, we calculate the proportion showing a Conservative lead over Labour of more than 6 points as the pseudo-probability of a Conservative majority, but we allow polls with exactly a 6 point lead to contribute a 0.5 to the average. Using the same polls, we use the proportion showing a Conservative lead over Labour of 16 points or more as the pseudo-probability of a Conservative landslide. These thresholds of 6 and 16 points are based on what would be required under uniform swing assumptions for the Conservatives to win a bare majority and a 100+ majority respectively.

Betting markets

There are numerous betting markets for the various outcomes in the election. We have taken those that are most helpful for the four forecasts we want to produce. For seat shares, we take the mid-point of the spread as the seat share, and average these mid-points between different sources. Note that the markets imply fewer seats forecast than there are actually are in the House of Commons. This is because the markets are separate for each party and do not need to be consistent collectively.

For vote share, we use betting markets for the Conservatives, Labour, the Liberal Democrats and UKIP. (Vote share markets for other parties are unavailable either on the betting market aggregation site Oddschecker of Betfair/Betfair Exchange). Odds are given for 5-point ranges of vote share. We take a weighted sum of the mid-points of these ranges where the weights are the implied probabilities. For the top and bottom options we use 2.5% above/below the upper/lower bound (e.g. 52.5 for “Above 50” and 27.5 for “Below 30”). The weighted sum is calculated just using the three categories with the largest implied probabilities, because the probabilities for other categories are so small and unstable. For UKIP we use just the two most likely categories.

For the probability of a Conservative majority we give an average of the implied probability from sites offering this market. For the probability of a Conservative landslide, we use the combined prices PredictIt that the Conservatives will win 370-379 seats, 380-389 seats, and 390 or more seats. This really represents a majority of 90 or more but that was as close as we could get to 100.

Statistical models

There are numerous statistical forecasting models this year. We have divided them into two categories: simple (poll average plus uniform swing seats projection) and complex (anything more elaborate than the simple models, although they are not necessarily particularly complex). Some make adjustments for long run differences between pollsters, for constituency variation, and some estimate by how much things will differ between current polls and the eventual result. Chris Hanretty’s forecast at electionforecast.co.uk does all of these things. Within these categories we simply average the available estimates of seats and shares. In addition, as discussed above, we have separated those models based on the local election results into a separate category.

We should note that not all of the models are the modellers’ favourite. Some are counterpoints to their main models for comparison. We have included these on the basis that they are still talked of and expected to be reasonable estimates, for example, at Electoral Calculus, Martin Baxter has a local election results based model. We have not excluded any models based on our judgement of quality, but they do have to be statistical models as opposed to personal guesses.

Citizen forecasts

Some polls ask people what they think that the outcome will be on June 8th. Different pollsters use different survey questions but they can be combined to generate pseudo-probabilities. We use the proportion of poll respondents who think the Conservatives will win/there will be a Conservative majority, excluding don’t knows and re-percentaging, as the pseudo-probability of a Conservative majority. We similarly use the proportion of poll respondents who think that there will be a Conservative landslide/the Conservatives will win more than 100 seats, for the probability of a Conservative landslide.

Note: Estimates come from around 9am on the morning of  8th June 2017. In all seat estimates, the Speaker’s seat is counted as Conservative.

 Sources

There sources we used are listed below in no particular order. Please let us know of any that you think we have missed or misclassified. Some polling averages we know of were not included because they were more than a week old.

Prediction markets:
IG
Predictit
Sporting index
Oddschecker
Betfair Predicts

Betfair

Betfair Exchange

 

Complex Forecasting Models:
Chris Prosser

ElectionForecast.co.uk (Chris Hanretty)
Electoral Calculus (main and local election forecast)
Election Data

Elections Etc

Forecast UK 
Janta-Lipinski

Kantar Public

Lord Ashcroft

Michael Thrasher
Murr, Stegmaier and Lewis-Beck

Nigel Marriott

Number Cruncher Politics

PME Politics (Patrick English)

UK-Elect

 

Simple forecasting models (polling average + uniform swing):
Electionpolling
Principalfish

 

Polling Averages (less than a week old):
Telegraph
FT
Britain Elects
Electionforecast.co.uk
NewStatesman
Electionpolling
Principalfish
The CrossTab

 

Citizen forecasts:
YouGov (here)
ComRes (here)
ICM (here)

Opinium (here)

Survation (here)

The three authors are equal contributors and our names are in alphabetical order.

 

2017 General Election, Forecast updates, Historical & polls based forecast

Polls-based forecast for the 2017 British general election

3 Comments

by Stephen Fisher and Josh Goldenberg.

Our central forecast for the result of today’s general election is as follows.

  GB share of the vote Seats 90% prediction intervals
Conservative 44 349 318-385
Labour 34.5 223 192-252
Liberal Democrat 9 9 3-15
SNP 4 47 39-53
PC 1 3  
UKIP 4 0  
Green 2.5 1  
 

This implies a Conservative vote share lead over Labour of 9.5 points and a majority of 48.  This represents only a modest improvement over the party’s performance 2015 when they achieved a majority of 12 with a 6.6 point lead.

From this central forecast, our estimated probability of a Conservative majority is 87%. Our analysis gives just a 1% chance of the Conservatives winning a 100+ landslide majority.

Continue reading Polls-based forecast for the 2017 British general election →

2017 General Election, Forecast updates

Combined forecast for GE2017: second update

Leave a comment

By Stephen Fisher, John Kenny and Rosalind Shorrocks.

This is our first post since the Manchester bombing. We would like to take this opportunity to extend our sympathy to all those affected, directly and indirectly. Now that the political parties are campaigning again we hope that it is not insensitive of us to update our forecasts.

In truth, because of Monday’s terrible events it is not clear how our forecast should be interpreted. Only one poll conducted since the attacks has been published, so most of the changes in the opinion poll data, and the models that are built on them, reflect polls conducted late last week; shortly after the Conservative manifesto launch and mostly before Theresa May’s announcement of a cap on social care funding. Those polls showed a considerable tightening of the Conservative lead and so a reduction in the predicted Tory majority.

Overall, our combined forecast of the Conservative majority has dropped to 100, down from 123 last week and from 132 two weeks ago.

Seats Betting Markets Complex models Simple model Volunteered Average
Con 380 382 356 381 375
Lab 181 189 210 173 188
LD 15 8 6 10
UKIP 0 0 0 0
Green 1 1 1 1
SNP 46 48 55 50
PC 3 3 3
Con majority 110 114 62 112 100

The combined probability of a Conservative majority, at 87% has correspondingly taken a small dip from 91% last week. More strikingly the probability of a Conservative landslide (a 100+ seat majority) has fallen from 64% (and 71% two weeks ago) to just 34% this week. The citizen forecast component of this average has not changed due to lack of new data. Previously that component was pulling down the average. Now it is pushing up. The markets and our pseudo-probability from the polls are both around 30%. Intriguing there is some inconsistency between the PredictIt market suggesting a 27% probability of a 100+ majority and the spread-betting markets pointing to a 110 seat majority.

Betting markets Models Polls Citizen forecast Average
Con majority 0.87 0.87 0.94 0.79 0.87
Con landslide 0.27 0.31 0.43 0.34

Continue reading Combined forecast for GE2017: second update →

2017 General Election, Forecast updates

Combined forecast for GE2017: first update

3 Comments

By Stephen Fisher, John Kenny and Rosalind Shorrocks.

Since our first forecast combining different indicators of the election outcome last week, the Conservatives and particularly Labour have edged up in the polls at the expense of UKIP and the Liberal Democrats. The crucial Conservative-Labour lead in the polls this week has varied from between 13 to 20 points. On average the lead has narrowed by just under two points to 16.5.

In line with this movement in the polls, the predicted Conservative majority from the betting markets and the simple models has dropped, but not, intriguingly, in the complex model forecasts. On average, our combined forecast is for a Tory majority of 123; down a little from 132 last week but still a big win.

Seats Betting

Markets

 Complex

models

 Simple

models

 Volunteered Average
Con 395 391 378 381 386
Lab 167 178 191 173 177
LD 15 10 13 13
UKIP 0 0 0 0
Green 1 1 1 1
SNP 46 47 45 46
PC 3 3 3
Con majority 140 133 106 112 123

The combined probability of a Conservative majority, at 91%, has remained unchanged from last week. However, the probability of a Conservative landslide has taken a small dip to a 64% chance (down from 71%). This drop is almost entirely due to Labour’s better performance in more recent polls, resulting in fewer leads for the Conservatives of 16 points or more required for a 100+ majority (the basis of our pseudo-probability from the polls).

Betting

markets

 Statistical

Models

 Polls Citizen

forecasts

 Average
Conservative Majority 0.95 0.91 1.00 0.79 0.91
Conservative landslide 0.81 0.67 0.43 0.64

Continue reading Combined forecast for GE2017: first update →

2017 General Election, Forecast updates

First Combined Forecast for the 2017 general election

4 Comments

By Stephen Fisher, John Kenny and Rosalind Shorrocks.

The betting markets, polls, statistical forecasting models and volunteer forecasters are all pointing towards a landslide majority of around 132 for the Conservatives in the UK general election on 8th June. Surveys of citizen expectations are more limited and less clear, but by our analysis they suggest that 43% of people in society at large are expecting a Tory majority of over 100.

Historically the idea of combining forecasts from different sources has had a good track record, though it has to be admitted that our attempt to do one for the EU referendum did not work out well. Most recently the pollyvote.com combined forecast of the US presidential election last year was 2 points out on the share of the vote.

This is the first of our Combined Forecasts for next month’s general election in which we aggregate seats and vote share forecasts from a variety of sources including betting markets, polls, statistical forecasting models, volunteer forecast aggregators, and citizen forecasts. We will update weekly and the methodology (as detailed below) will evolve.

All the different sources point to a big Conservative win and there is little sign of doubt that the Conservatives will get at least a majority. Various different complex forecasting models are pointing to majorities for Theresa May of between 110 and 174, with an average of 130. Betting and prediction markets are just a little more bullish with an average majority of 146.  Volunteers contributing forecasts to the Times Red Box sweepstake are only slightly more cautious. Details of the average estimate from each source, and the overall average number of seats expected for each party are in the table below.  In addition to the big Conservative majority they collectively suggest that the Liberal Democrats seat tally will only increase by four and that the SNP will be down seven.

Seats Betting Markets Complex models Simple models Volunteered Average
Con 398 390 394 380 391
Lab 158 178 173 172 170
LD 18 11 9 13
UKIP 0 0 0 0
Green 2 1 1 1
SNP 45 49 51 49
PC 3 3 3
Con majority 146 130 138 110 132

Not only are the headline forecasts from these sources remarkably strong for the Conservatives but the evidence from the polls and the models is consistent. There is also little sign of doubt among punters. The table below shows the implied probabilities of there being a Conservative majority and a Conservative landslide (100+ majority) from the betting markets, forecasting models (where available) and the polls (as judged by the proportions of polls showing the required lead). The citizen forecasts in the table show the proportion of respondents (excluding Don’t Knows) expecting each outcome. They are noticeably much more cautious and both ICM and YouGov surveys have shown as few as 30 per cent expecting a Conservative landslide.

Probabilities Betting markets Statistical Models Polls Citizen forecasts Average
Conservative Majority 0.96 0.92 1.00 0.77 0.91
Conservative landslide 0.83 0.86 0.43 0.71

The sources for vote-share forecasts are more limited but again remarkably consistent across polls, markets and forecasting models. The polling average is an average of seven different polling averages. They are at best nowcasts rather than forecasts. The statistical models give actual forecasts. Even though the polls have underestimated the Conservatives and overestimated in the large majority of general elections, and substantially so in 2015, all but three of the four forecasting models with vote shares are expecting Labour to outperform the polls and the Tories to underperform. The betting markets are expecting the opposite; an even bigger Conservative lead than the polls are showing.

% vote Polling average Betting markets Forecast Models Average
Con 46.8 47.4 45.0 46.4
Lab 28.7 26.9 27.6 27.7
Lib Dem 9.6 12.2 10.9
UKIP 6.4 3.9 6.3 5.5
Green 2.8 2.6 2.7
SNP 4.3 4.1 4.2
PC 0.6 0.6 0.6

Methodology

Continue reading First Combined Forecast for the 2017 general election →

EU referendum, Forecast updates, Historical & polls based forecast

Final forecast from the Historical Referendums and Polls based method

18 Comments

by Stephen Fisher and Alan Renwick.

The polls this week have been better for Remain than they were last week. Since this is our final forecast it makes sense for us to restrict our sample of polls to include in our polling average just the most recent poll from each company (or company-mode combination) over the last week. If we do this then our polling average finds Remain at 51 per cent after setting aside Don’t Knows. This is up two points from our polling average on Sunday. The two-point difference is partly due to restricting the sample from two weeks to just one, partly rounding error and partly to the fact that more of the polls than previously include Northern Ireland. So it is not clear whether the apparent movement towards Remain is real or not.

Our forecast share of the vote is 52 per cent for Remain, 48 per cent for Leave. This reflects an expectation of a 1.5-point rise in support for the status quo, based on the change that is visible on average between the final polls and the actual result in previous referendums in Britain or on the EU elsewhere. While this reflects the average historical experience we have explained here and here why the average may not be a very reliable guide.

The unreliability means there is a lot of uncertainty in our forecast. The 95 per cent prediction interval is considerably narrower than it was at the beginning of the week. But at ±10 points it is still very wide. So wide that Remain could reasonably be expected to get anywhere between 42 per cent and 62 per cent of the vote. Neither a comfortable Remain victory nor a comfortable Leave victory can be ruled out.

That said not all the possible outcomes in this range are equally likely. Our forecast probability that Remain will win the referendum is 64 per cent.

The methods behind our forecast

Continue reading Final forecast from the Historical Referendums and Polls based method →

Combined forecast, EU referendum, Forecast updates

Final combined EU Referendum forecast

17 Comments

by Stephen Fisher and Rosalind Shorrocks.

This forecast is based on data collected late on 22nd June. Yet again this method suggests Remain is most likely to win but there is a considerable chance that Leave may win. The probability that Remain will win is up from 62.3% on Sunday to 66.5% now. This reflects both better opinion polls in recent days and so polling models more favourable for Remain, and greater market confidence in a status quo victory.
The only component forecast that is less favourable to Remain is the citizen forecasts, and that is because we have restricted our sample to citizen forecasts over the last week. They are noticeably more equivocal.
The forecast share of the vote is once again little changed, with Remain predicted to get 53.3% and Leave 46.7% per cent of the vote.
Remain % share Leave % share Probability Remain wins
Betting markets 53.5 46.5 76.7
Prediction markets 73.4
Citizen forecasts  52.0  48.0 55.2
Expert forecasts 55.1 44.9  62.0
Volunteer forecasts 54.0 46.0 74.0
Polls 50.6 49.4 55.6
Poll based models 52.5 47.5 68.5
Non-poll based models 55.6 44.4
Combined forecast (mean) 53.3 46.7 66.5

(Individual forecasts collected on the evening of 22nd June 2016.)

There is another poll due during polling day but that would be unlikely to change our polling average. The markets may change, possibly dramatically during the day if people on the betting and prediction markets are following trends financial markets.

METHODOLOGY Continue reading Final combined EU Referendum forecast →