By Andreas Murr and Stephen Fisher.

Two years ago Andrew Adonis wrote a piece in Prospect arguing that Labour should ditch Jeremy Corbyn because of the importance of party leadership for electoral success. The piece claimed that “the best leader wins and nothing else matters,” and in Lord Adonis’s view Mr Corbyn is the worst Labour leader since Michael Foot. So, Adonis concluded already in September 2017, that regarding the next election “Corbyn will lose it decisively if he contests it.”

In response to Adonis’ claims many, including Danny Finkelstein, expressed skepticism about the power of leadership and pointed out that it is difficult to properly evaluate the quality of leaders in retrospect. Once we know who won and who lost we have a tendency to convince ourselves that the winner was a better leader than the loser. But there are ways of producing measures of leadership quality prior to elections which have historically been useful for forecasting election outcome.

The Party Leadership Model, was devised by Andreas Murr in the run up to the 2015 general election. It successfully anticipated David Cameron’s victory unlike the vast majority of other forecasts at the time. Murr has elaborated the model further for this election to produce a seats forecast, not just a prediction as to who will emerge as prime minister. His new model forecasts that the Conservatives will win an overall majority this week with 342 seats, and that Labour will win 254 seats.

The essence of the model is very simple. The party leader who won their internal party-leadership election by the biggest margin among MPs is most likely to win the general election. The idea is that MPs have the capacity to identify election winners and strong incentives to pick them as leaders. Table 2 from Murr’s paper (replicated below) shows that the model has successfully predicted the winner in 10 out of 13 elections since 1966.

Table 2: The Party Leadership Model correctly predicts 10 out of 13 past elections when data is sufficiently available to make a forecast.

		% point lead among MPs in party leadership election
General election	Incumbent	Conservative	Labour	GE Prediction	GE Winner
1966	Labour	5.7	16.6	Labour	Labour
1970	Labour	5.7	16.6	Labour	Conservative
1974 (Feb)	Conservative	5.7	16.6	Labour	Labour
1974 (Oct)	Labour	5.7	16.6	Labour	Labour
1979	Labour	24.3	12.4	Conservative	Conservative
1983	Conservative	24.3	3.8	Conservative	Conservative
1987	Conservative	24.3	23.2	Conservative	Conservative
1992	Conservative	14.5	65.6	Labour	Conservative
1997	Conservative	39.2	40.6	Labour	Labour
2001	Labour	13.6	40.6	Labour	Labour
2005	Labour	—	40.6	—	Labour
2010	Labour	16.7	80.0	Labour	Conservative
2015	Conservative	16.7	-6.8	Conservative	Conservative
2017	Conservative	35.0	-13.8	Conservative	Conservative
2019	Conservative	26.5	-13.8	Conservative	?

Note: Conservative leader in 2005, Michael Howard, was appointed leader without a formal vote, so there is no forecast for that year.

Despite a very good track record for the model, especially compared with other approaches to election forecasting, there is substantial uncertainty in the forecast for this week’s election. After all it is based on just thirteen cases and the three failures of the model so far were rather spectacular. As a result, the model suggests there is a 59% chance of a Conservative majority.

Although this forecast is being published only just before the election itself, it is worthwhile noting that it could have been made in July after Boris Johnson’s election as Conservative Party leader. An important part of the value of the model is that it yields useful predictions well in advance of general elections, about four years in advance on average.

We now go into more detail on the model mechanics.

Why should MPs be good at forecasting who wins?

MPs have the means, motive, and opportunity to select a party leader with the highest electoral appeal.

The MPs know the leadership candidates like nobody else does (“means”).  The candidates are MPs themselves and so selectors and candidates observe each other on a daily basis.

The MP’s future career depends on selecting a party leader with the highest electoral appeal (“motive”).  A weak party leader is likely to lead to fewer votes at the next general election, putting the MP’s job at risk.

Finally, since 1963 both the Conservatives and Labour have allowed MPs to formally vote in party leadership elections (“opportunity”). The number of votes for the selected leader should indicate her electoral appeal.

The Party Leadership Model

To translate the voting behaviour of MPs at party leadership elections into forecasts of general elections, the Party Leadership Model calculates the difference in vote shares between the selected leader and the main contender (“performance”).  It then compares the performance of Conservative and Labour party leaders, and predicts the leader with the better performance to become Prime Minister.

Are MPs actually good at forecasting?

According to a Bayesian analysis, there is a 95 per cent probability that having the larger margin in party leadership elections increases the chances of winning the General Election.

How well did the Party Leadership Model predict the 2015 and 2017 General Elections?

The above results were all based on ex post forecasts.  How well did the Party Leadership Model perform in its first ex ante forecasts, the 2015 and 2017 British General Election?

It correctly forecasted that David Cameron and Theresa May would become Prime Minister again in 2015 and 2017, respectively.

In 2005, David Cameron was elected as Conservative party leader. In the MPs’ vote Cameron was 16.7 percentage points ahead of his main contender, David Davies.  In 2010, Ed Miliband was elected as Labour party leader. But he lost the MPs’ vote. His brother, David Miliband was more popular among Labour MPs, by 6.8 percentage points.  On the basis that David Cameron was more popular than Ed Miliband among their own MPs (by 16.7 to -6.8 points), the Party Leadership Model correctly predicted Cameron’s re-election.

In 2017, Theresa May was elected as Conservative party leader with a winning margin of 35.0 percentage points over Andrea Leadsom.  In 2015, Jeremy Corbyn was elected as Labour party leader. Under the party’s then new system, MPs did not have a vote but candidates needed to secure nominations from at least 15% of the parliamentary party. Taking the numbers of nominations as equivalent to MPs’ votes, Jeremy Corbyn lost the MPs’ section of his initial leadership contest still more heavily than Ed Miliband lost his. Corbyn received the fewest nominations of all four of the candidates who passed the 15% threshold. His tally was some 13.8 percentage points behind Andy Burnham’s. If anything this understates Corbyn’s unpopularity among Labour MPs, especially given that 172 of them voted no confidence in him the following year. Nonetheless, based on the 2015 Labour-leadership contest and the 2016 Conservative-leadership contests the Party Leadership Model still successfully predicted that Theresa May would win the 2017 general election because she was initially more popular among Tory MPs than Corbyn was among Labour MPs at the times of their party leadership elections.

What does the Party Leadership Model predict for the 2019 General Election?

In July 2019, Boris Johnson was elected as Conservative party leader, 26.5 percentage points ahead of his main contender, Jeremy Hunt, in the fifth and final ballot of Tory MPs.  Still using the initial 2015 Labour-leadership contest as the measure of Corbyn’s leadership, Boris Johnson is therefore predicted by the Party Leadership Model to win the 2019 general election.

How many seats will each party win?

To predict seats, the Party Leadership model regresses the the seat share of a party on whether its leader was more popular than the other (L: 0=no, 1=yes). Using continuous measures of the leadership gap did not work so well. Also, to improve prediction quality, the regressions are done separately for principal government and opposition parties.

Predicted incumbent-party seat share (%) = 41.4 + 11.3*L (SE = 8.9)

Predicted opposition-party seat share (%) = 39.1 + 12.4*L (SE = 8.5)

Because the Conservatives are going into this election as the incumbent and have the more internally popular leader among MPs, the model predicts the Tories will win 41.4 + 11.3*1 = 52.7 per cent of seats. That is 342 seats.  Meanwhile, Labour as the principal opposition with the least internally popular leader among MPs is predicted them to win 39.1 + 12.4*0 = 39.1 per cent of seats. That is just 254 seats.

Given the prediction uncertainty in the model, Boris Johnson has a 59% chance of winning an overall majority.