How to forecast the Conservative leadership election

The last Conservative leadership election took place following the height of the Brexit process in Parliament. At the time, I began analysing MPs’ records on various Brexit votes (whether they supported a second referendum, a customs union, no deal etc.) and grouped every Tory MP into one of seven Brexit tribes to predict the outcome. If 50 per cent of the No Dealers who backed a candidate threw their weight behind Boris Johnson, I expected 50 per cent of No Dealers who hadn't publicly backed anyone to also support him.

I was able to predict the result with great accuracy — my forecast a few days into the contest predicted the top three to within one vote of the actual outcome.

This time around, I am using the same basic principles with my election forecasts — which have been proving popular on Twitter (@patrickjfl) — but there are many more variables at play. By looking at who each MP backed last time, how they voted on social and economic issues, when they became an MP, their Brexit position in 2016 and whether they served in Johnson’s cabinet, among others, I can find the statistically significant factors driving each candidate’s support. For example, Rishi Sunak is getting the majority of his support from Remain-supporting MPs, those on the economic right of the party and those in Remain or ‘soft Leave’ constituencies. Penny Mordaunt, on the other hand, does not fare particularly well with Remainers, and the overwhelming majority of her advocates are backbenchers.

For each candidate, I use these factors to create a formula which predicts levels of support from MPs who haven’t yet endorsed anyone. The model has performed well so far, correctly forecasting that Grant Shapps, Sajid Javid, Jeremy Hunt, Nadhim Zahawi and Suella Braverman would all be eliminated at the stages they were.

One of the reasons I am able to cut through the noise of Westminster speculation and predict this race accurately is because for me, the facts will always trump the rhetoric. My model doesn’t care that Sunak voted to leave the EU, because most of his parliamentary support comes from those who didn’t. Likewise, it’s not interested in whether Liz Truss calls for serious change if she’s disproportionately backed by government ministers.

The hard numbers tell us more than the words of candidates, and looking forward, they are indicating that the gap between the top three will shrink.

I predicted in my post-round one forecast that Mordaunt would close the gap with Sunak when all is said and done, and she made a start on that in the last round, with the distance between the two now down to 18 from 21.

Assuming Mordaunt's seeming loss of momentum over the weekend doesn't make an impact, this should continue because the backers of Braverman, Tugendhat and Badenoch have more in common with the supporters of Mordaunt than they do with Sunak. There are just eight public backers of those three candidates who voted Remain in 2016 (where Sunak beats Mordaunt), yet there are 55 who did not (in this group, Mordaunt beats Sunak). This segment of MPs is also disproportionately made up of backbenchers, where Mordaunt gets the vast majority of her support. Simply put, there is a much larger pool of supporters for Mordaunt to swim in from the eliminated candidates than there is for Sunak.

There are other ways of forecasting this election, too. At Smarkets, the prices on our platform were indicating that even though Sunak was the early favourite, he was likely to go into the runoff as the underdog. This was borne out by YouGov polling last week that showed Mordaunt beating Sunak handsomely in a head-to-head. In turn, this had an effect on the outcome of the first round of voting. Following the poll’s release, Sunak underperformed with MPs by about 10 votes compared to my forecast and Mordaunt overperformed by the same amount, leading me to believe that the ex-Chancellor’s survey results led a handful of MPs to switch their votes.

Since then, the markets have been in a state of flux, reflecting the high levels of volatility in this race. Both the markets and polls have moved significantly in Rishi Sunak's favour in the last week, and he is now the favourite to win once again.

Statistical models, betting markets and polls all provide important insights. If you can understand these methods and the interplay between them, you’ll see a much clearer picture of who our next Prime Minister could be.