Monthly Archives: June 2017

2017 General Election

How Theresa May lost her majority and how Scots have kept the Tories in government

1 Comment

By Stephen Fisher.

At the time of writing it looks like the Conservatives will win 319 seats, short of the 326 seats they needed to win an overall majority. (The following numbers are provisional.)

Most likely they will come to some agreement with the Democratic Unionist Party in order to remain in government.

Given that Sinn Fein won 7 seats that they will not take, there are effectively 643 MPs returning to the House of Commons.  The Conservatives 319 + 10 DUP seats constitutes an effective majority of 15.

This is intriguingly similar to the actual majority of 12 that David Cameron won in 2015.

By comparison with a tally of 331 seats in 2015, the Conservatives lost 27 seats to Labour and 5 to the Liberal Democrats.

29 of those 32 Conservative losses were in England.

They were partly compensated for by gains of 6 seats from Labour, 1 from the Liberal Democrats, 1 from UKIP in England.  As a result the Tories suffered a net loss of 21 seats in England.

They lost a further 3 seats in Wales, making a net loss of 24 seats in England and Wales.

Were that the end of the story then the Conservatives would have ended up with just 310 seats. That would not have been enough to remain in government.

Only thanks to 12 gains from Scottish National Party are the Tories able to remain in government.

The Conservatives in England and Wales owe their continuing position in government to the popularity and success of Ruth Davidson and the Scottish Conservatives.

Numbers will be updated when all results are in.

 

 

 

2017 General Election, Combined forecast, Forecast updates

Final Combined Forecast for GE2017

2 Comments

By Stephen Fisher, John Kenny and Rosalind Shorrocks.

Our final combined forecast is for a Conservative majority 66. This estimate is down only slightly from 70 on Friday last week, but well down on the 132 majority we first forecast on 12th May. All three sources on average each suggest that net gains for the Conservatives will largely be at the expense of Labour and the SNP, with tallies for other parties little changed from 2015.

Seats Betting Markets Complex models Simple models Average
Con 371 361 343 358
Lab 199 214 228 214
LD 11 7 7 8
UKIP 0 0 0 0
Green 1 1 1 1
SNP 46 46 50 47
PC 3 3 3 3
Con majority 92 72 36 66

The combined probability of a Conservative majority is now 72%. This is little changed from last week but down from 91% in our first combined forecast. The probability of a 100+ majority Conservative landslide is, at 11%, the same as last week but well down on the 71% we reported on 12th May. Betting markets are the most confident of a Tory majority. Citizens collectively express the most uncertainty, with both the highest probability on a Tory landslide and the second highest probability on the Conservatives losing their majority. Taking the most recent poll from each pollster, only just over half of them show the Conservatives with the lead of 6 points or more that would be needed for the party to win a majority on uniform change calculations. The 54% figure is down from 63% last week, but it reflects a change in who has done a poll this week more than how the pollsters from last week have seen their figures change.

Betting markets Models Polls Citizen forecast Average
Conservative Majority 0.83 0.82 0.54 0.69 0.72
Conservative landslide 0.17 0.00 0.17 0.11

For our final forecast we have taken a straight average of the final polls published at the time of writing instead of relying on other poll aggregators. This average is for a strikingly similar Conservative lead as in the actual outcome last time 2015, 6.5 compared with 6.6 points. But the final polls last time pointed to a tie. The markets and models predict that the polls are over estimating Labour and under estimating the Conservatives again. Overall our combined forecast this week now suggests that the Conservatives will win with a 9.6 point lead. This estimate is down only slightly from 10.2 points last week. But there has been a massive drop from the 18.7 point lead predicted in our first combined forecast on 12th May.

Final Poll average Betting markets Models Average
Con 43.0 45.8 44.3 44.4
Lab 36.5 34.7 33.1 34.8
Lib Dem 7.6 8.3 9.7 8.5
UKIP 4.4 3.4 4.7 4.1
Green 2.1 2.2 2.1
SNP 4.0 4.1 4.0
PC 0.6 0.8 0.7
Con-Lab lead 6.5 11.1 11.2 9.6

Methodology

The main changes in method this week are as follows.

Nigel Marriott in his final model now includes a forecast for party vote shares as well as the previous seat forecasts and so this is included; we have also added new models from Election Data, Elections Etc, Janta-Lipinski, Kantar Public, Michael Thrasher, Andreas Murr et al and Number Cruncher Politics. YouGov’s forecast model has been removed because they have officially dropped it. Betting market data for the forecasted party vote share of Labour, the Liberal Democrats and UKIP is gathered from Betfair/Betfair Exchange where such data is not available on Oddschecker.

Our polling average is now a simple average of the final polls.

As there have been five citizen surveys published since the 31st May, we have decided to limit the inclusion criteria for this so that only the most recent citizen survey from each survey company during this period is included.

Overall description of the method

Our basic approach is to combine forecasts by averaging them within each category and then taking the average across categories. Since the different sources do not all present equally clear figures that can be averaged on a like for like basis we have made various judgement calls on how to treat the data.

Historically the idea of combining forecasts from different sources has had a good track record, though it has to be admitted that our attempt to do one for the EU referendum did not work out well. Most recently the pollyvote.com combined forecast of the US presidential election last year was 2 points out on the share of the vote. 

Polls

Our polling average is now a simple average of the final polls. Previously for vote shares, we use the various available polling averages, or ‘polls of polls’, and take their average. We exclude polling averages for whom the most recently published polling average is more than a week old. There are a number of different polling averages. They are in truth nowcasts rather than forecasts, but we are in effect treating them as forecasts. Some admittedly are quite sophisticated, allowing for pollster (aka house) effects, but they are nonetheless estimates of current public opinion and not future votes.

We do not attempt to say what seats outcome is implied by polls (that is the job of the modellers). However, since statistical models are rarely if ever clear about the probabilities their models place on key events like a Conservative majority and a 100+ majority, we have included in the probabilities table some pseudo-probabilities from the polls. Taking the most recent poll published by each pollster in the last week, we calculate the proportion showing a Conservative lead over Labour of more than 6 points as the pseudo-probability of a Conservative majority, but we allow polls with exactly a 6 point lead to contribute a 0.5 to the average. Using the same polls, we use the proportion showing a Conservative lead over Labour of 16 points or more as the pseudo-probability of a Conservative landslide. These thresholds of 6 and 16 points are based on what would be required under uniform swing assumptions for the Conservatives to win a bare majority and a 100+ majority respectively.

Betting markets

There are numerous betting markets for the various outcomes in the election. We have taken those that are most helpful for the four forecasts we want to produce. For seat shares, we take the mid-point of the spread as the seat share, and average these mid-points between different sources. Note that the markets imply fewer seats forecast than there are actually are in the House of Commons. This is because the markets are separate for each party and do not need to be consistent collectively.

For vote share, we use betting markets for the Conservatives, Labour, the Liberal Democrats and UKIP. (Vote share markets for other parties are unavailable either on the betting market aggregation site Oddschecker of Betfair/Betfair Exchange). Odds are given for 5-point ranges of vote share. We take a weighted sum of the mid-points of these ranges where the weights are the implied probabilities. For the top and bottom options we use 2.5% above/below the upper/lower bound (e.g. 52.5 for “Above 50” and 27.5 for “Below 30”). The weighted sum is calculated just using the three categories with the largest implied probabilities, because the probabilities for other categories are so small and unstable. For UKIP we use just the two most likely categories.

For the probability of a Conservative majority we give an average of the implied probability from sites offering this market. For the probability of a Conservative landslide, we use the combined prices PredictIt that the Conservatives will win 370-379 seats, 380-389 seats, and 390 or more seats. This really represents a majority of 90 or more but that was as close as we could get to 100.

Statistical models

There are numerous statistical forecasting models this year. We have divided them into two categories: simple (poll average plus uniform swing seats projection) and complex (anything more elaborate than the simple models, although they are not necessarily particularly complex). Some make adjustments for long run differences between pollsters, for constituency variation, and some estimate by how much things will differ between current polls and the eventual result. Chris Hanretty’s forecast at electionforecast.co.uk does all of these things. Within these categories we simply average the available estimates of seats and shares. In addition, as discussed above, we have separated those models based on the local election results into a separate category.

We should note that not all of the models are the modellers’ favourite. Some are counterpoints to their main models for comparison. We have included these on the basis that they are still talked of and expected to be reasonable estimates, for example, at Electoral Calculus, Martin Baxter has a local election results based model. We have not excluded any models based on our judgement of quality, but they do have to be statistical models as opposed to personal guesses.

Citizen forecasts

Some polls ask people what they think that the outcome will be on June 8th. Different pollsters use different survey questions but they can be combined to generate pseudo-probabilities. We use the proportion of poll respondents who think the Conservatives will win/there will be a Conservative majority, excluding don’t knows and re-percentaging, as the pseudo-probability of a Conservative majority. We similarly use the proportion of poll respondents who think that there will be a Conservative landslide/the Conservatives will win more than 100 seats, for the probability of a Conservative landslide.

Note: Estimates come from around 9am on the morning of  8th June 2017. In all seat estimates, the Speaker’s seat is counted as Conservative.

 Sources

There sources we used are listed below in no particular order. Please let us know of any that you think we have missed or misclassified. Some polling averages we know of were not included because they were more than a week old.

Prediction markets:
IG
Predictit
Sporting index
Oddschecker
Betfair Predicts

Betfair

Betfair Exchange

 

Complex Forecasting Models:
Chris Prosser

ElectionForecast.co.uk (Chris Hanretty)
Electoral Calculus (main and local election forecast)
Election Data

Elections Etc

Forecast UK 
Janta-Lipinski

Kantar Public

Lord Ashcroft

Michael Thrasher
Murr, Stegmaier and Lewis-Beck

Nigel Marriott

Number Cruncher Politics

PME Politics (Patrick English)

UK-Elect

 

Simple forecasting models (polling average + uniform swing):
Electionpolling
Principalfish

 

Polling Averages (less than a week old):
Telegraph
FT
Britain Elects
Electionforecast.co.uk
NewStatesman
Electionpolling
Principalfish
The CrossTab

 

Citizen forecasts:
YouGov (here)
ComRes (here)
ICM (here)

Opinium (here)

Survation (here)

The three authors are equal contributors and our names are in alphabetical order.

 

2017 General Election, Forecast updates, Historical & polls based forecast

Polls-based forecast for the 2017 British general election

3 Comments

by Stephen Fisher and Josh Goldenberg.

Our central forecast for the result of today’s general election is as follows.

  GB share of the vote Seats 90% prediction intervals
Conservative 44 349 318-385
Labour 34.5 223 192-252
Liberal Democrat 9 9 3-15
SNP 4 47 39-53
PC 1 3  
UKIP 4 0  
Green 2.5 1  
 

This implies a Conservative vote share lead over Labour of 9.5 points and a majority of 48.  This represents only a modest improvement over the party’s performance 2015 when they achieved a majority of 12 with a 6.6 point lead.

From this central forecast, our estimated probability of a Conservative majority is 87%. Our analysis gives just a 1% chance of the Conservatives winning a 100+ landslide majority.

Continue reading Polls-based forecast for the 2017 British general election →

2017 General Election, Gender

Labour poll surge mainly thanks to younger women, but also old

3 Comments

By Rosalind Shorrocks and Stephen Fisher.

Coverage of recent polls has suggested that women are becoming more supportive of Labour and that this is driving the recent tightening of the election race. The figure below shows the average vote intention separately for men and women on average using data from a range of different pollsters (see methodological note below).Shares_by_genderAt the beginning of May there was very little gender gap. The Conservative lead was much the same for men as for women. For polls conducted in the past week, on average the Conservatives still had a large, 14-point lead amongst men, but only a small, 4-point lead amongst women. Compared with the start of May, women are now 7 points more likely to vote Labour than men, and 3 points less likely to vote Conservative.

There was no appreciable difference between the trends for men and the trends for women up to 18th May, when the Conservative manifesto was launched. It was in the polls in the immediate aftermath of that launch that we see a much more dramatic narrowing of the Conservative lead among women than among men.

This opening up of a gender gap seems to be related to Conservative policy proposals. For example, women are more negative about the party’s social care plans than men. In a Survation poll from 19th-20th May, 34% of men supported the plans, versus just 22% of women. Women were also more likely to say that the plans made them more anxious about getting older, caring for older relatives, owning a house, and securing a future for their children.

It is not surprising that men and women should have reacted differently to Mrs May’s social care proposals. For those aged 70, men have a 1 in 10 chance of needing social care whereas for women the chances are 1 in 4.

The Conservative policy proposals are about formal social care costs, but most social care is informal within extended families and done mainly by middle-aged women. Without them the formal care system would truly crumble. If formal social care is to be increasingly paid for by families, then we can expect that increasing pressure to be put on women especially to take on caring responsibilities for elderly relatives to avoid the costs of social care.

The Conservative manifesto also proposed converting the triple lock on pensions into a double lock and means testing the Winter fuel allowance. It looked to many like a broader attack on pensioners. Whatever the rights and wrongs of the policies with regard to intergenerational fairness, women live longer than men and are much more likely to be affected by these proposed reductions in future income than men.

This all suggests we might expect bigger changes in vote intention amongst middle-aged and elderly women. Panelbase is the only polling company that breaks down their vote intention figures by gender and age in their standard published tables. The figures should be treated with caution because of the small numbers of respondents within each group, but they are illustrative. The Con-Lab lead by age and gender from these polls is shown in the figure below.age_by_genderThey show that the Conservative lead has dropped for both genders and for all age groups. (For those under 35 the Conservative lead is negative, and so dropping means Labour moving even further into the lead for this group). The drop for women is much larger than it is for men, and it is concentrated in particular age groups.

Older women have long been one of the most supportive groups for the Conservatives – and they still are. But whilst at the start of the campaign the Conservatives had a lead of 50 points over Labour amongst women aged 55+, this lead has now been dramatically reduced to around 20 points. This may be related to the Conservative plans with respect to social care and pensions, although in the Panelbase data it does seem to have come slightly late.

What is most striking in the figure, however, is that is it not the elderly or middle-aged who have moved most during this campaign, but women aged 18-34. They have become much more Labour. Even though the primary movement for this group was in the aftermath of the manifesto launch, this is hard to link to the Conservative’s social care or pension plans, since this group is the least likely to be immediately affected by them.

Instead, perhaps younger women are influenced by general perceptions of the Conservatives aiming to cut social and health services versus Labour aiming to extend service provision. This may have altered particularly younger women’s views. Women do tend to prioritise issues such as the NHS, education, and welfare at elections more than men, and data from numerous pollsters suggests that this election is no exception in this regard. Moreover, some policies in the Labour manifesto may have been particularly appealing to young women – for example the party’s plans for more free childcare.

In 2015, younger women were much more likely to vote Labour than younger men, whilst older women were much more likely to vote Conservative than older men. In the most recent polling, this pattern is replicated for younger women and indeed the gender gap in the younger age group appears to be growing. However, the same cannot be said of the older age group: older men and women now appear to be similar in their relative support for the Conservatives and Labour. The decline in relative support for the Conservatives amongst older women, a traditionally staunch Conservative group, may have serious repercussions for the party. The combined increased in relative support for Labour from both older and younger women after the manifesto launches is one of the key drivers behind the recent narrowing of the polls.

Methodological Note

The first figure is based on data from ICM, Survation, YouGov, ORB, ComRes, Opinium, and Kantar, who have conducted regular polls throughout the campaign and, crucially for our analysis, conducted polls both shortly before and shortly after the Conservative manifesto was launched. At each time point in the graph we take an average across pollsters. Where a pollster conducted multiple polls in the period the average of their polls are taken. All figures are after those who were undecided are set aside

The pollsters have a range of different methodologies, with ICM tending to be the most favourable to the Conservatives, and YouGov tending to be the most favourable to Labour. Despite their differences each of the individual pollsters show similar trends by gender. This suggests to us that the findings for the first figure are fairly robust.