Canada and UK 2019: District level fragmentation

With two of the big Westminster parliamentary democracies having had general elections in 2019, we have a good opportunity to assess the state of district-level competition in FPTP electoral systems.

(Caution: Deep nerd’s dive here!)

Before we turn to the district level, a short overview of what is expected at the national level is in order.

As noted previously, Canada’s election produced a nationwide seat balance that was extremely close to what we expect from the Seat Product Model (SPM), yet the nationwide votes were exceedingly fragmented (and, anomalously, the largest seat-winning party was second in votes). The UK election, on the other hand, was significantly less fragmented in the parliamentary outcome than we expect from the SPM, even if it was in key respects a “typical” FPTP outcome in terms of manufacturing a majority for a party with less than a majority of the vote.

In general, over decades, Canada tends to conform well to the SPM expectation for the shape of its parliamentary party system, whereas the UK is a more challenging case from the SPM’s perspective.

The SPM states that the effective number of seat-winning parties (N_S) should be the seat product, raise to the power, 1/6. The seat product is the assembly size, times the mean district magnitude. The SPM predictions for N_S explain around 60% of the variance in actual outcomes for elections around the world under a wide variety of electoral systems. SPM predictions for other output quantities also explain in the neighborhood of 60%. So the SPM is both successful at explaining the real world of seat and vote fragmentation, and leaves plenty of room for country-specific or election-specific “other factors” (i.e., the other 40%). The SPM is based on deductive logic, starting from the minimum and maximum possible outcomes for a given number of seats at stake (in a district or an assembly). The logic is spelled out in Votes from Seats.

In the case of a FPTP system, the SPM makes the bold claim that we can understand the shape of a party system by knowing only the assembly size. That is because with district magnitude, M=1, the seat product is fully described by the country’s total number of seats, S, which is also the number of districts in which the voting is carried out. Thus we expect N_S=S^1/6. Let’s call this “Equation 1.”

For Canada’s current assembly size (338), this means N_S=2.64, as an average expectation. Actual elections have tended to come pretty close–again, on average. Of course, individual elections might vary in one direction or the other. (The assembly size was also formerly smaller, but in recent times, not by enough to concern ourselves too much for purposes of this analysis.) For the UK, the corresponding expectation would be 2.94 based on a seat product of 650.

The actual Canadian election of 2019 resulted in N_S=2.79; for the UK it was 2.39. Thus for Canada, we have a result very close to the expectation (ratio of actual to expected is 1.0578). For the UK, the actual result was quite short (ratio of 0.8913). As I said, the UK is a challenging, even aberrant, case– at least at the national level.

What about the district level? A national outcome is obviously somehow an aggregation of all those separate district-level outcomes. The SPM, however, sees it differently. It says that the districts are just arenas in which the nationwide election plays out. That is, we have a logical grounding that says, given a national electoral system with some seat product, we know what the nationwide party system should look like. From that we can further deduce what the average district should look like, given that each district is “embedded” in the very same national electoral system. (The logic behind this is spelled out in Votes from Seats, Chapter 10).

The crazy claim of the SPM, district-level extension, is that under FPTP, assembly size alone shapes the effective number of votes-earning parties in the average district (N’_V, where the prime mark reminds us that we are talking about the district-level quantity rather than the nationwide one). (Note that for FPTP, it must be the case that N’_S=1, always and in every district).

The formula for expected N’_V under FPTP is: N’_V=1.59S^1/12 (Equation 2). It has a strictly logical basis, but I am not going to take the space to spell it out here; I will come back to that “1.59” below, however. It is verified empirically on a wide set of elections, including those from large-assembly FPTP cases like Canada, India, and the UK. So what I want to do now is see how the elections of 2019 in Canada and UK compare to this expectation. (Some day I will do this for India’s 2019 election, too.)

If the effective number of seat-winning parties at the national level (N_S) is off, relative to the SPM, then it should be expected that the average district-level effective number of vote-earning parties (N’_V) would be off as well. They are, after all, derived from the same underlying factor–the number of single-seat districts, i.e., the assembly size (S). We already know that N_S was close to expectation in Canada, but well off in the UK in 2019. So how about the districts? In addition to checking this against the expectation from S alone, we can also check one other way: from actual national N_S. We can derive an expected connection of N’_V to N_S via basic algebra. We just substitute the value from one equation into the other (using Equations 1 and 2). If we have N_S=S^1/6 then it must be that S= N_S⁶. So we can substitute:

N’_V=1.59(N_S⁶)^1/12= 1.59√N_S (Equation 3).

In a forthcoming book chapter, Cory L. Struthers and I show that this works not only algebraically, but also empirically. We also suggest a logical foundation to it, which would require further analysis before we would know if it is really on target. The short version suggested by the equation is that the voting in any given district tends to be some function of (1) the basic tendency of M=1 to yield two-candidate competition (yes, Duverger!) in isolation and (2) the extra-district viability of competing parties due to the district’s not being isolated, but rather embedded in the national system. The 1.59, which we already saw in Equation 2, is just 2^2/3; it is the expected N’_V if there were exactly two vote-earning parties, because it is already established–by Taagepera (2007)–that the effective number tends to be the actual number, raised to the power, two thirds. And the square root of N_S suggests that parties that win some share of seats (i.e., can contribute more or less to the value of N_S) tend to attract votes even though they may have no chance of winning in any given district. By having some tendency to attract votes based on their overall parliamentary representation, they contribute to N’_V because voters tend to vote based on the national (expected, given it is the same election) outcome rather than what is going on in their district (about which they may have poor information or simply not actually care about). If the parliamentary party system were fully replicated in each district, the exponent on N_S would be 1. If it were not replicated at all, the exponent would be zero. On average, and in absence of any other information, it can be expected to be 0.5, i.e., the square root.

How does this hold up in the two elections we are looking at in 2019? Spoiler alert: quite well in the UK, and quite badly in Canada. Here are graphs, which are kernel density plots (basically, smoothed histograms). These plots show how actual districts in each election were distributed across the range of observed values of N’_V, which in both elections ranged from around 1.35 to just short of 4.5. The curve peaks near the median, and I have marked the arithmetic mean with a thin gray line. The line of most interest, given the question of how the actual parliamentary outcome played out in each district is the long-dash line–the expected value of N’_V based on actual N_S. This corresponds to Equation 3. I also show the expectation based solely on assembly size (light dashed line); we already have no reason to expect this to be close in the UK, but maybe it would be in Canada, given that the actual nationwide N_S was close to the SPM expectation, based on S (Equation 2).

Here is the UK, then Canada, 2019.

What we see here is interesting (OK, to me) and also a little unexpected. It is the UK in which the actual mean N’_V is almost the same as the expectation from nationwide N_S (i.e., Equation 3). We have actual mean N’_V=2.485 compared to expected N’_V from actual N_S of 2.45; the ratio of actual to expected is 1.014. We can hardly ask for better than that! So, the nationwide party system (as measured by N_S) itself may be well off the SPM expectation, but the vote fragmentation of the average district (N’_V) closely tracks the logic that seems to stand behind Equation 3. Voters in the UK 2019 election tended to vote in the average district as if parties’ national viability mattered in their choice.

In Canada, on the other hand, even though national N_S was very close to SPM expectation, the actual average district’s N’_V (2.97) was really nowhere near either the expectation solely from S (the light dashed line, at 2.58) or the expectation from the actual N_S (2.66). The average district was just so much more fragmented than it “should be” by either definition of how things ought to be! (The ratio of actual to that expected from Equation 3 is 1.116; the Equation 3 expectation is almost exactly the 25th percentile of the distribution.)

The Canadian outcome looks as if the exponent on actual N_S in Equation 3 were around 0.64 instead of 0.5. Why? Who knows, but one implication is that the NDP (the third national party) performed far better in votes than the party’s contribution to N_S implies that it should have. Such an overvaluing of a party’s “viability” would result if voters expected the party to do much better in terms of seats than it did. This is probably a good description of what happened, given that pre-election seat extrapolations implied the NDP would win many more seats than it did (and the Liberals fewer). The NDP also underperformed its polling aggregate in votes (while Liberals over-performed), but it held on to many more voters than it “should have” given its final seat-winning ability would imply. That is, the actual result in votes suggests a failure to update fully as the parties’ seat prospects shifted downward at the very end of the campaign. In fact, if we compare the final CBC poll tracker and seat projections to the ultimate result, we find that their actual votes dropped by 13.6% but their seats dropped by 31.7% (percent change, not percentage points!). In other words, this was just an unusually difficult context for voters to calibrate the expectations that Equation 3 implies they tend to make. (I am assuming the polls were “correct” at the time they were produced; however, if we assume they were wrong and the voters believed them anyway, I think the implications would be the same.)

It should be understood that the divergence from expectation is not caused by certain provinces, like Quebec, having a different party system due to a regional party, as some conventional expectations might point towards. While Quebec’s size is sufficient to exert a significant impact on the overall mean, it is not capable of shifting it from an expected 2.6 or 2.7 towards an observed 3.0! In fact, if we drop the Quebec observations, we still have a mean N’_V=2.876 for the rest of Canada. The high fragmentation of the average district in the 2019 Canadian election is thus due to a Canada-wide phenomenon of voters voting for smaller parties at a greater rate than their actual viability would suggest they “should”. In other words, voters seem to have acted as if Trudeau’s promise that 2015 would be the last election under FPTP had actually come true! It did not, and the electoral system did its SPM-indued duty as it should, even if the voters were not playing along.

On the other hand, in the UK, voters played along just as they should. Their behavior produced a district-level mean vote fragmentation that logically fits the actual nationwide seat balance resulting from how their votes translated into seats under FPTP. There’s some solace in that, I suppose.

Votes, seats, and exit polls: UK 2019 edition

Two political scientists, Pippa Norris and Patrick Dunleavy, have accused the BBC and others of “systemic media bias” on the recent UK election night for not emphasizing the voting outcome and instead focusing on the seats. Their claims appear at the LSE blog. Of course, I am very much inclined to agree that votes and seats both matter–I’ve (co-) written two books that have both words, votes and seats, in their titles, after all! Thus I largely agree with Norris and Dunleavy’s bigger point that media coverage in majoritarian electoral systems tends to exaggerate the notion that a party that wins the seat outcome has a “mandate”. As I said in my own election post-mortem, the “mandate” claim is a stretch, at best, and very much depends on how the electoral system manufactures majorities–not only for Conservatives overall but also for the SNP among Westminster constituencies within Scotland.

Nonetheless, the claims in the LSE blog piece are somewhat hard to swallow. The main argument is that at 10:00 p.m., when polls closed, only the seats were mentioned. The votes did not come till 5:00 a.m., they claim. Anthony B. Masters has already shown that is not actually true, in a really excellent rebuttal. I won’t repeat Masters many points regarding misleading evidence that the LSE blog authors present to make their case.

The deeper issue here is that the exit poll is bound to be more accurate for seats–the initial projection almost nailed the result for the UK as a whole–than for votes. The voting estimates are subject to more error, because of uncertainty about turnout. Moreover, seats are the currency of power. Votes are relevant as a “currency of legitimacy” (as Jonathan Hopkin put it on Twitter), which is important for the subsequent narratives and intraparty soul-searching for the losers. That is, however, very much the kind of stuff that can only happen once the full results are known (not that it stops the media talking heads from engaging in speculation all night long). Basically, it is just very odd to slam as “biased” the media for reporting what was proven to be an actually accurate projection of the one thing the poll was designed to do and that matters most on election night–who won the most seats, was it a majority, and if so, how big?

Besides, as Masters notes in his rebuttal, it is not even true that votes were not being reported all night long. They simply are subject to more revisions as the picture gets clearer because, as noted above, the vote estimate is subject to more error.

Finally, I’d note that it could be much worse. In US elections, the topic of votes hardly comes up in the media, particularly for congressional elections. Even if you stayed up till 5:00 a.m. on election night (not that I ever have), you would not hear what percentage of the House votes each party had.

Reminder from UK 2019 result: Electoral systems matter

Keep this in mind about the UK result. The Conservatives won less than 44% of the vote. Polling has consistently shown that if there were another referendum on Brexit, a majority would vote for Remain. But the Conservatives won 56% of the seats, so Johnson is banging on about his great “mandate” to “get Brexit done”.

You see, electoral systems matter.

Even if you add in the Brexit Party votes (which got no seats), the combined votes cast for parties still advocating outright for leaving the EU do not reach a majority. In fact, it barely breaks 45%.

Meanwhile, the SNP has won 81% of the Scottish seats, with 45% of the votes cast in Scotland. And their leader, Nicola Sturgeon, is going on and on about the mandate for Scotland to decide on independence. It’s a fishy claim.

Which party gained the most in votes, relative to the last general election? That would be the Liberal Democrats. But the party suffered a net loss of one seat (and its leader was defeated).

The first-past-the-post (FPTP) electoral system makes a country seem more divided than it is, and often leads to policy outcomes a majority of voters actually oppose.

FPTP certainly is not very representative. But it can produce a decisive government, and Boris Johnson now looks like he could take his place among the significant Prime Ministers in the country’s recent history.

At least this result means my old lectures about British majoritarianism do not to be heavily caveated as they’ve been for the past several years.

The Brexit Party

Just a quick add-on to my previous remarks on the UK 2019 election. Via @kiwiting on Twitter comes this example of a Brexit Party local leaflet.

Look closely and you might actually see the local candidate’s name! As I stress in the preceding post, I expect parties under FPTP (at least in parliamentary systems) to require a national presence in the party system in order normally to do well at the constituency level. That is a key insight of the Seat Product Model, and how it stands apart from “bottom-up” approaches that stress local district-level “coordination” as what drives a party system. But this is pretty extreme: the Brexit Party is not only a single-issue party, it is also a one-man band!

Even though this party at one point was polling above 20% (and won a plurality of the UK vote in the European Parliament elections), it was always hard for me to take the Brexit Party seriously. On the one hand, it certainly is a nationally focused party. On the other hand, the leader Nigel Farage made a decision not to contest any constituency, or to target even one seat somewhere that some candidate of the party might win. The process behind the SPM implies that voters respond to the “viability” of a smaller party, and tend to vote for it without too much regard for the viability of its candidate in their own district. But for that to work, it has to be viable–and preferably winning–somewhere. Not only did the Brexit Party not even try this, it pulled its candidates out of seats the Conservatives hold, while retaining candidates only in districts held by other parties. It is a bizarre strategy if the party was serious, and it is no wonder the party is on life support. Of course, they are going to get their one policy issue enacted (even if not as “hard” as they would like), precisely by not posing too big a risk to the incumbent government’s pursuit of a (manufactured) majority.

UK election 2019

The UK general election is almost here. At this point, it seems quite unlikely that the result will be anything other than a good old fashioned FPTP manufactured majority. Boris Johnson and his Conservatives will win a majority of seats, barring a surprise, despite under 45% of the votes, and will be able to pass their Brexit deal.

If one looks at the polling aggregate graph by the Economist, one might be tempted to conclude it was also a good old fashioned “Duvergerian” pattern at work. As recently as early October, before the election was legislated, the Conservatives were leading on about 33% of the votes, and three other parties ranged from 12% to 25%. Go back further, to June, and all for were in the 18–25% range (with Labour then on top, and the Brexit Party ahead of the Conservatives). Since the latter part of October, and especially since the campaign formally got underway, Conservatives and Labour have both taken off, at the expense of the LibDem and Brexit parties. Notably, the gap between the top two has been quite steady, at 8-10 percentage points. Unlike 2017, there is no evidence at all that Labour is closing the gap. Labour simply are hoovering up the non-Tory (and Remain or second-referendum) votes at the same time as Leave voters have realized there’s no point in voting for a single-issue Brexit Party when the Tories have a pretty “hard” Brexit deal already to go, if only they win a majority of seats.

So, on the one hand, a far more “normal” election for a FPTP-parliamentary system than seemed possible during the long parliamentary deadlock of the past year or more. Just like Duverger’s “law” predicts, right? Desertion of the third and fourth parties for the top two.

Only sort of. Let’s take the current polling estimates for the parties (and not forgetting to include the current 5% “other”, which I will treat as one party, given most of it is one party–the Scottish National Party). It results in an effective number of vote-earning parties of 3.05. That’s a little high for a supposedly classic two-party system! It is, however, lower than seen at any election from 1997 through 2015. In 2017, however, it was 2.89, which was the lowest since 1979. The top two would be combining for 78% of the votes, which is a little higher than most elections from 1974 (February, in a two-election year) through 2001. Even in 2017, hailed by many at the time as the return to two-party politics–albeit dubiously–had a combined top-two of just 82.4%. (It looks like a high figure only compared to 2005-2015, when it ranged from 65.1% to 67.6%.)

Of course, it is the seats that really matter. Seat projections based on election polls under FPTP are never easy. There are various ones out there, but I will go with YouGov‘s.* It has the Conservatives with a projected 359 seats, which is 55.2%, with Labour on 211 (32.5%). Taking all the parties (and here breaking the “Northern Ireland” bloc down a bit, as we know it will consist of more than one such party), we get an effective number of seat-winning parties around 2.4. That is even lower than 2015, driven mainly by the presence of an expected single-party majority.

[*Note: just after I posted this, YouGov posted an update of their projections. I am not going to revise the numbers here. The differences are small, though potentially politically significant. See my first comment below this post.]

The problem with the standard Duvergerian claims about FPTP is that they ignore assembly size: In a larger assembly, we should expect more parties, other things (like district magnitude and formula) equal. While we could argue over how much the expected results of the 2019 election correspond to the so-called law, I’d rather not. What is of interest to me is that the UK case continues its long-term defiance of the Seat Product Model (SPM), and that’s something that I can’t take lying down.

While the conventional wisdom would see 2017 and 2019 as some sort of return to normalcy, it’s actually a challenging case for me. From the SPM (which explains over 60% of the variation in party-system outcomes worldwide, including FPTP systems), we should expect:

Effective number of seat-winning parties: 2.95.

Seat share of the largest party: 0.445.

Effective number of vote-earning parties: 3.33.

The seat outcomes actually never have come very close to the expectations. As for votes, the 1987 election got it right, but was a terrible performer in terms of seats (effective N=2.17!). Taking all the indicators together, the 2010 election is about the closest to what should be “normal” for a FPTP system with such a large assembly: effective N on votes 3.72, seats 2.57, and largest seat share of 0.47. So why was that not finally the start of the kind of party system the country “should” have? I guess we need to blame Nick Clegg. Or David Cameron. (I’d rather blame the latter; he was the one, after all, who thought a Brexit referendum was a good enough idea to go ahead with it.) More to the point, voters’ reaction to Clegg and the LibDems entering a coalition and–gasp–making policy compromises. After which, voters reverted to supporting the big two in greater shares than they are supposed to. In other words, contingency and path dependency overcome the SPM in this case. I hate to admit it, but it’s the best I’ve got!

Speaking of the LibDems, they should have had an opportunity here. Labour has the most unpopular opposition leader in decades. (Deservedly so, but I digress.) And the best hope for stopping Brexit would be tactical voting to increase their chances to win seats where Labour is not best positioned to defeat a Tory. Yet, despite lots of constituency-level tactical voting advice being offered in this campaign, there’s little evidence the message is getting though.

There is tactical voting happening, but as Rob Johns points out in a short video, it is happening based on the national outcome and not on district level. Under the Duvergerian conventional wisdom, voters are alleged to think of their constituency, and vote tactically (strategically) to effect the local outcome. Yet in real life, only a relatively small minority of voters behave that way. That voters use a strategy based on who is best placed to defeat a party they do not like on the national level, instead of at the constituency level, is a point made forcefully by Richard Johnston in his book, The Canadian Party System. It is also the underlying logic of the SPM itself.

So from the standpoint of the SPM, what is surprising is not that there isn’t more tactical voting at the constituency level. It is that there does not remain (so to speak) a strong enough third party, such as the Liberal Democrats, to appear viable nationally so that voters would be willing to vote for its district candidates. Quite apart from the legacy of the coalition that I referred to above, the case for the LibDems as a viable counterweight probably was not helped by a tactical decision it made in this campaign. Its leader, Jo Swinson, declared that a LibDem government would revoke the Article 50 notification and cancel Brexit. Put aside the ridiculous idea that there would have been a LibDem government. If one had resulted from this election, it would have been on far less than 50% of the votes. So you have a government resting on a minority promising to go back on the majority voice of the 2016 referendum without even bothering with a second referendum. That seemed at the time like a dumb position for the party to take. Only recently has Swinson offered the message of what the LibDems could accomplish in a no-majority parliament. But it’s too late. There almost certainly won’t be such a parliament.

The UK really needs a national third party (and fourth…). Contrary to the Duvergerian conventional wisdom, the electoral system actually could sustain it; we would expect the party system to look more like Canada’s (which conforms to the SPM very well, both over time and, in terms of seats, in 2019). Given the larger assembly, the British party system should be even less two-party dominated than Canada’s actually is. It is by now rather apparent that the LibDems are not the third party the system needs to realize its full potential. Will one emerge? Alas, not soon enough to stop a hard Brexit from being implemented by a manufactured majority (for a leader who is pretty unpopular himself) while Labour gobbles up most of the opposition, but falls well short.

Rediscovering an old publication

Believe it or not, I just noticed an article by me, published in an academic journal, has been missing from my CV for over twenty years! In fact, I had to search on the web to find it.

“The Jenkins Paradox: A complex system, yet only a timid step towards PR,” Representation 36:2 (1999).

I thought of it when wanting to link to it in my previous note about the Quebec proposal. And then I could not find the link because it was not on my CV (or website)!

My personal favorite passage from my forgotten article, after commenting on the Jenkins Commission proposal for the UK and its flaws:

It would seem, therefore, preferable to use MMP with a small percentage of PR seats, or MMP with multiple regional PR compensation regions, or straightforward alternative vote, but not some combination of all three!

The other thing I realized in searching for this is just how dreadfully bad the interface of the Taylor and Francis journals website is.

Meanwhile in the UK…

I think I’ve said this a time or two before… but things in the UK seem to be getting pretty interesting this week, and in the weeks to come.

Strong and stable, and all that.

So, any predictions, comments, etc.? Here’s the place.