by Stephen Fisher and Alan Renwick

A month ago we issued our first forecast for the EU membership referendum on 23rd June 2016. Based on an analysis of referendums in the UK and on the EU outside the UK, and on vote intention opinion polls we forecast that Remain had an 87% chance of winning, and that Remain would get 58% of the vote, plus or minus 14. This was in part based on our polling average (excluding Don’t Knows) of 55% for Remain on 11th March.

Our current forecast suggests the contest is a fair bit closer. Our polling average now puts Remain on 52%. We now give Remain a 73% chance of winning and estimate that the Remain share of the vote will be 54% plus or minus 13 points.

The key change here is the drop from 55% to 52% Remain in the polling average. The main reasons for this are as much or more methodological than substantive.

Over the past couple of months the telephone polls have shown declining support for Remain (before and after excluding Don’t Knows). But there has been little or no trend in the online polls over the same period. So rather than both kinds of poll moving in lock-step, the much discussed gap between phone and online polls in levels of support for Remain (aka the mode effect, see here, here and here) has declined substantially.

Ultimately it is impossible to tell for sure which if either of these kinds of poll has been accurately reflecting changes in public opinion. It would be too bold to assume that online polls were getting it right and not phone, or vice versa. Instead we try to estimate the average pattern of change across pollsters. Assuming that pollster (house) effects have been stable over the past month and a half, averaging over some regression models with slightly different specifications suggests around a one-point drop in support for Remain (excluding DK) over the last month.

Over the same period our estimate of the difference in the Remain share of the vote between telephone and online polls has shrunk from 9 to 5 points. For our polling average, this means that whereas we previously adjusted telephone polls down and online polls up by 4.4 points for Remain, the adjustment is now just 2.5 points. In retrospect we think that our previous mode effect estimate for our first forecast was too large and so we have switched to a more conservative method for estimating the mode effect. Still most of the 2 drop in the size of the adjustment is due to new polls over the last month suggesting that the mode effect has genuinely diminished.

In sum, the main source of change since our 11th March forecast ­– a 3-point drop for Remain (exc DK) in our polling average – is primarily made up of a (roughly) 2-point narrowing in the telephone vs online mode-effect adjustment. There appears to have been one-point drop in support for Remain in actual public opinion over the last month.

The other component of the forecast share of the vote that has changed is the amount by which we expect Remain to rise in the polls over the course of the campaign. Whereas we previously expected Remain to gain 1.5 points, we now expect only half a point. This purely reflects the difference between analysing historical polls over the final 75 days instead of the final 105 days. It does not mean that we were expecting Remain to have risen by a percentage point over the last month specifically. Although clearly the polls provide no evidence for any of the progress that Remain was expected to make.

As before there is still substantial uncertainty. With such a tiny (0.5 point) estimate for the direction of change now, the forecast is essentially that trends in the polls for the remainder of the campaign could go either way.

We are still expecting Remain to outperform the final polls in the actual referendum by about a point and half. Again that is a modest central estimate and accompanied by substantial uncertainty. Meaning that it is only just a bit more likely that the polls under- rather than overestimate Remain.

The forecasting method still makes no attempt to anticipate the possibility that supporters of one side turn out at a greater rate than supporters of the opposing side. Polling evidence so far suggests that Leave supporters are more likely to vote than those who favour Remain. One reason for us not doing anything about this (for the moment) is that some pollsters are already doing turnout weighting while others are not, and trying to account properly for which pollsters have done what is tricky. Turnout weighting will ultimately be a responsibility for pollsters and as the referendum approaches presumably more of them will be doing it.

While differential turnout has the potential to have a massive effect on the outcome, trying to estimate what that effect will be this far out is difficult. If this becomes a referendum that people care a lot about and nearly everyone turns out for, then, mathematically, there will little scope for different groups to turnout at very different rates. The lower overall turnout is the more differential turnout is likely to matter. Predicting the extent to which people will be interested and care about the outcome is important for estimating both overall and differential turnout. There are ways of going about this but all fraught with considerable uncertainty. So another reason why, at the moment, we are refraining from making differential turnout adjustments is that current patterns of reporting of likelihood to vote in the polls may not prove that reliable an indicator of future overall or differential turnout.

Thanks again to Jack Sheldon, Roberta Damiani and Johnny Runge for data gathering. We are still planning to write more about both our analyses of public opinion in the run-up to referendums and our forecasting method.