The UK 2017 result–Comparative data forays

Well, the election that I thought would be just a boring “typical” snap election in which the incumbent takes advantage of the unprepared opposition… did not quite turn out that way.

Some things happened that are not supposed to happen. And some things about the result are glaringly off the mark of what we should expect from the Seat Product Model (which, of course, is meant to predict average trends, not individual elections).

Top two dominance but no majority

A party is not supposed to gain votes, but lose seats. It is hard to exaggerate how extraordinary this is. The top two parties combined for 82.4% of the votes, the highest in the UK in a long time. The last time it was over 80% was in 1979. The last time over 75% was 1992, and in the three elections immediately before this one, the future had been around two thirds.

Yet, despite the recovery of the top-two vote shares, there is no majority party. Parliamentary majorities have been won on far less in the past, and one would not expect such dominance of the two leading parties (42.4% and 40.0%) under FPTP to fail to produce majority government. But here we are.

I was curious to know just how common it was for both parties in a FPTP parliamentary system to have at least 40% of the vote, but there to be no parliamentary majority. In my dataset of FPTP elections, consisting of 210 observations, I find one case: Trinidad and Tobago 1995 (two top parties on 48.8% and 47.2%, tied in seats with 17/36). (I have not kept this updated in recent years, and perhaps I am failing to remember one that would be included if I had.)

If I drop my threshold a little lower, to the top two parties both being at at least 38% (but no seat majority), I get one more case: Canada 1957. Of course, the main reason why a leading party with 40% or even 38% of the votes so often gets a majority under FPTP is that it tends to have a more substantial lead over the runner-up, implying many districts are competitive.

Thus it is not only the top two absolute sizes that matter for getting a majority, but also their ratios. How common is it for the top two parties to have votes so similar? First of all, let’s define a ratio of the top two in votes; the mean of this ratio in the data sample is 1.67 (median 1.26). In this UK election, it was 1.06. Approximately 15% of the elections are this close. However, only around 3% of all the elections are both this close and result in no majority party, including UK 1974 (Feb.).

Thus the UK17 combination of two-party dominant, close, and no seat majority is pretty unusual!

Campaigns and leaders

Campaigns and leaders matter. That is not in itself surprising, but many political scientists (sometimes including me) consider them less important than “fundamentals”–whatever those might be. But May did not look like someone who could provide “strong and stable” government. And indeed, she may not get to provide any government at all, if she can’t survive a seemingly inevitable challenge to her position from within.

On the other hand, does Labour’s success relative to low expectations suggest leaders do matter? Did voters actually come to like Corbyn? I am aware of no evidence that such was the case. I suspect he was still a drag on the party, but will leave it to other analysts to try to sort this out. It seems to me that any reasonably competent Labour leader could have won this election, which in turn would never have happened, because May would not have called it had the main opposition had a reasonably competent leader.

The numbers compared Seat Product expectations

On the quantitative indictors, the effective number of vote-earning parties (NV) was, by my calculations from data at BBC, 2.88. The last time it was that low in the UK was 1987, when the leading party (Conservative) won a vote share about the same as this time (42.3%), but it won 57.8% of the seats.

The effective number of seat-winning parties (NS) was 2.47. This is not so unusual, by UK standards, as the figure was 2.53 in the 2015 election and 2.57 in 2010, the last time no party won a majority. In fact, the UK has tended to have a less fragmented parliamentary party system than expected from the Seat Product Model, which would be NS=2.94. The maximum observed since 1945 was the just-reported 2.57 in 2010.

For NV, the Seat Product Model says to expect 3.32, based solely on the large assembly size. Although the post-WWII mean is much lower than that, the electoral party system was finally behaving in the 1992-2015 period, with all those elections seeing NV>3, and the last three (2005, 2010, 2015) all being at 3.6 or higher. Then came 2017, and the party system stopped behaving properly!

It should be emphasized that the Seat Product Model does not expect a majority party; with this large an assembly, even FPTP “should have” a largest party size of 44.5%. At 48.9%, the Conservatives are only a little higher than where they should be. But, of course, actual UK experience usually returns a majority in parliament, and this election was certainly expected to do so–where those expectations are based on political factors and the opinion polls, not the humble Seat Product Model.

Governance and policy

As for government-formation, clearly it is a Tory minority government. Claims by a few pundits that Corbyn could somehow assemble parliamentary support are pure fantasy. And there almost certainly won’t be a coalition. The most likely formula is backing from the Democratic Unionist Party (DUP), of Northern Ireland. The DUP’s 10 seats plus the Conservatives’ 318 combine for just over half the seats.

What will it mean for policy, especially Brexit? I can’t claim to know! But the DUP does not want a “hard border” with the Republic of Ireland, and that implies a “softer” Brexit. On the other hand, if the main motivation May had in calling the election was to boost her standing against restive members of her own caucus who want a harder Brexit, she failed. It will not be easy governance or policy-making for May or an intraparty successor.

Funny how elections don’t always turn out how we expect them to. Democracy! FPTP!

Appendix: Effective Number of Parties and the Seat Product Model

The effective number of parties is a size-weighted count, where each party’s share (of votes or seats) is weighted by itself through squaring. The squares are summed, and you take the reciprocal. See Michael Gallagher’s excellent website for details.

I am not going to explain here the logic behind the Seat Product Model. For that, see Taagepera (2007) or Li and Shugart (2016), or the forthcoming Shugart and Taagepera book, Votes from Seats (2017, due out in October). But the equations are as follows, where M is the mean magnitude (1, in the case of FPTP) and S is the assembly size (650 in recent UK elections).

NS=(MS)1/6;

NV=[(MS)1/4+1]2/3;

Seat share of the largest party: s1=(MS)-1/8.

The important thing to understand about these equations is that they are not post-hoc regression fits. They are logical models, derived without reference to the data. When tested against the data from hundreds of democratic elections under various electoral systems, they are astonishingly accurate.

Related earlier posts and comment threads:

UK 2017: Green Party won’t stand in Ealing constituency

So much for fixed terms

UK 2015 and Diverter’s Law

Does UK 2015 mean the death knell for Duverger’s Law?

UK 2017: Green Party won’t stand in Ealing constituency

Here is something we do not see in First-Past-the-Post elections* as much as the Duvergerianists seem to think we should: one party agreeing not to have a candidate in order to avoid vote-splitting in a district.

The Green Party has pulled out of a crucial election seat in a bid to help the Labour Party beat the Tories – the first tactical withdrawal of its kind ahead of the general election.

The decision is expected to allow more votes to go to Labour MP Rupa Huq, who beat the Conservatives with a majority of just 274 votes in 2015, when no other party managed to attract more than seven per cent of the vote.

Green Party members in Ealing — where the party won 1,841 votes in the 2015 election — voted not to field a candidate last week, after Ms Huq promised to campaign for voting reform and the environment.

____
* Except in India!

BC 2017: Non-Duvergerian watch

British Columbia votes for its provincial assembly on 9 May. Eric Grenier’s vote and seat projections at CBC suggest it could be a “non-Duvergerian” outcome, especially in the votes. The NDP looks likely to win a majority of seats, manufactured by the electoral system, as its votes are likeliest to stay below 45%. The Liberals are running well behind, but are also likely to be over-represented due to their more efficient votes distribution: Grenier’s average projection for the party would put them on just under 35% of votes but 39% of seats.

The non-Duvergerian aspect of the election is the Greens: average projection of 20% of the votes, which is on the high side for a third party under FPTP. The mechanical effect will crush them, however: average seats only 2 of 87. That, of course, is “typical” FPTP.

Where things get interesting is if the Greens rise even just a few percentage points. The “high” projection takes their votes up only to around 23% but their seats to 12 or more (around 14%)! Clearly, they are running second in many districts–particularly on Vancouver Island.

The reason this pattern is non-Duvergerian is that the so-called Duverger’s law says voters tend not to want to waste their votes on a third party. They should “coordinate” with the more-preferred of the two big parties. Further, this defection should be more likely after the third party has suffered from the mechanical effect of FPTP in past elections, as the Greens in BC have. Instead, their support is evidently growing.

But there is not much incentive to choose between the major parties when you have three-party competition throughout much of the province, and the third party could win a significant number of seats even if it stays well below 50% in the districts where it is most viable.

The BC NDP platform promises: “We’ll hold a referendum – and campaign for the “yes” side – on replacing our outdated voting system with a proportional one.”

Will they follow through if FPTP delivers them a majority?

Update: The tracker of where the party leaders are campaigning is interesting. Of course, there is much emphasis on swing districts, particularly for the NDP. However, this quotation from Richard Johnston (UBC political scientist) is spot on:
“You do have to campaign everywhere; you have to validate the existence of your candidates.” In Chapter 10 of Votes from Seats (forthcoming, Cambridge University Press), Rein Taagepera and I refer to this phenomenon as the “embeddedness” of districts in nationwide (here, provincewide) politics. We do not consider non-Duvergerian districts to be quite the puzzle that others do, because we expect voters to be more oriented towards the aggregate outcome than towards their own district, and parties to care about “showing the flag” even in districts they may not be very likely to win.

Uttar Pradesh, 2017

Election results have been released for the state assembly of Uttar Pradesh, India’s largest state. It was a big win for the federal ruling party, the BJP. The seat tally shows 312 for the BJP, with the second highest being the Samajwadi Party (SP) at 47. The SP, the ruling party since 2012, was in a pre-election coalition with the Indian National Congress (INC), which won just 7 seats. The Bahujan Samaj Party (BSP), which has been a significant party in the state in the past, won 19 seats.

Unlike 2012, when the SP majority in the assembly was achieved on not even 30% of the vote, this year’s BJP victory was a big win in votes, too. Not a majority, but a decisive plurality, at 39.7% of the vote. The SP-INC combine had 28.0% and the BSP 22.2%.

Note that the BJP managed a three-fourths majority (77.4%) of the 403 seats on not even 40% of the vote. The advantage ratio (%seats/%votes) was 1.95. That must be one of the biggest manufactured majorities under FPTP anywhere, at least in a large assembly.

Several other states have had recent elections as well. The news was better for the struggling INC in some, including Punjab, Goa, and Manipur, though its pluralities in these are short of majority status. The Aam Aadmi Party (which governs Delhi, but has had minimal success elsewhere) managed a distant second place in Punjab. See the results at the second link in this entry.

Republicans will likely keep their House majority – even if Clinton wins by a landslide – and it’s because of gerrymandering.

By Michael Latner

While the presidential race has tightened, the possibility of Donald Trump being defeated by a wide margin has some Republicans worried about their odds of retaining control of Congress. However, only a handful of Republican-controlled districts are vulnerable. Speaker Paul Ryan’s job security and continuing GOP control of the House is almost assured, even if Democrats win a majority of the national Congressional vote. How is it that the chamber supposedly responsive to “The People Alone” can be so insulated from popular sentiment? It is well known that the Republican Party has a competitive advantage in the House because they win more seats by narrow margins, and thus have more efficiently distributed voters. What is poorly understood is how the current level of observed bias favoring the GOP was the result of political choices made by those drawing district boundaries.

This is a controversial claim, one that is commonly challenged. However, in Gerrymandering in America: The House of Representatives, the Supreme Court, and the Future of Popular Sovereignty, a new book co-authored by Anthony J. McGann, Charles Anthony Smith, Alex Keena and myself, we test several alternative explanations of partisan bias and show that, contrary to much professional wisdom, the bias that insulates the GOP House Majority is not a “natural” result of demographic sorting or the creation of “majority-minority” districts in compliance with the Voting Rights Act of 1965. It is the result of unrestrained partisan gerrymandering that occurred after the 2010 Census, and in the wake of the Supreme Court’s 2004 decision in Vieth v Jubelirer, which removed legal disincentives for parties to maximize partisan advantage in the redistricting process.

Partisan bias tripled after congressional redistricting

We measure partisan bias using the symmetry standard, which asks: What if the two parties both received the same share of the vote under a given statewide districting plan? Would they get the same share of seats? If not, which party would have an advantage?

We calculate seats/votes functions on the assumption of uniform partisan swing – if a party gains 5% nationally, it gains 5% in every district, give or take an allowance for local factors (simulated through random effects that reduce our estimates of bias). Linear regressions provide an estimate of what level of support the Democrats expect to win in each district if they win 50% of the national vote, given the national level of support for the party in actual elections, and we generate a thousand simulated elections with hypothetical vote swings and different random local effects for each district, to create our seats/votes functions.

Figure 1: Seats/Votes function for Congress 2002-2010
SV2010//embedr.flickr.com/assets/client-code.js

Figure 2: Seats/Votes function for Congress 2012
sv2012

Figures 1 and 2 show the seats/votes functions under the 2002-2010 Congressional districts, and the 2012 post-redistricting districts, respectively. We observe a 3.4% asymmetry in favor of Republicans under the older districts. This is still statistically significant, but it is only about a third of the 9.39% asymmetry we observe in 2012. Graphically, the seats/votes function in Figure 1 comes far closer to the 50%votes/50%seats point. The bias at 50% of the vote is less than 2% under the older state districting plans, compared to 5% in 2012. That is, if the two parties win an equal number of votes, the Republicans will win 55% of House seats. Furthermore, the Democrats would have to win about 55% of the vote to have a 50/50 chance of winning control of the House in 2016. Thus it is not impossible that the Democrats will regain control of the House, but it would take a performance similar to or better than 2008, when multiple factors were favorably aligned for the Democrats.

Increased bias did not result from “The Big Sort” 

Perhaps the most popular explanation for increased partisan bias comes from the “Big Sort” hypothesis, which holds that liberals and conservatives have migrated to areas dominated by people with similar views. Specifically, because Democrats tend to be highly concentrated in urban areas, it is argued, Democratic candidates tend to win urban districts by large margins and “waste” their votes, leaving the Republicans to win more districts by lower margins.

The question we need to consider is whether the concentration of Democratic voters has changed relative to that of Republican voters since the previous districts were in place. In particular, if it is the case that urban concentration causes partisan bias, then we would expect to find relative Democratic concentration increasing in those states where partisan bias increases. In order to address this question, we measure the concentration of Democratic voters relative to that of Republicans with the Pearson moment coefficient of skewness, using county-level data from the 2004 and 2012 presidential elections. As shown in Table 1, in most states the level of skewness toward the Democrats actually decreased in 2012.

slide1_29768903842_o

In twenty-seven out of the thirty-eight states with at least three districts, the relative concentration of Democratic voters compared to Republican voters declines. Moreover, in those states where partisan bias increased between the 2000 and 2010 districting rounds, those with an increase in skewness are outnumbered by those where there was no increase in skewness, by more than two to one. We also find that there was reduced skewness in most of the states where there was statistically significant partisan bias in 2012.

Of course, we should not conclude that geographical concentration does not make it easier to produce partisan bias. North Carolina was able to produce a highly biased plan without the benefit of a skewed distribution of counties, but to achieve this, the state Generally Assembly had to draw some extremely oddly shaped districts. While the urban concentration of Democratic voters makes producing districting plans biased toward the Republicans slightly easier, it makes producing pro-Democratic gerrymanders very hard. In Illinois, the Democratic-controlled state legislature drew some extremely non-compact districts but still only managed to produce a plan that was approximately unbiased between the parties.

Increasing racial diversity does not require partisan bias

Another “natural” explanation for partisan bias, one that is especially popular among Southern GOP legislators, is that that it is impossible to draw districts that are unbiased while at the same time providing minority representation in compliance with the Voting Rights Act of 1965. The need to draw more majority-minority districts, it is argued, disadvantages the Democrats because it forces the inefficient concentration of overwhelmingly Democratic minority voters.

There are four states with four or more majority-minority districts – California, Texas, Florida, and New York – and they account for more than 60% of the total number of majority-minority districts. Of these, Texas and Florida have statistically significant partisan bias, but California, Illinois, and New York do not, so the need to draw majority-minority districts does not make it impossible to draw unbiased districting plans. Yet many of the states that saw partisan bias increase do have majority-minority districts – or rather a single majority-minority district in most cases. It is possible that packing more minority voters into existing majority-minority districts creates partisan bias.

To test this possibility, we subtract the average percentage of African-Americans and Latinos in districts where those races made up a majority of the population in the 110th Congressional districts from the 113th Congress averages. Figures 3 and 4 display the results for these states. For majority-Latino districts, we find no evidence that states with increased Latino density have more biased redistricting plans.

Figure 3: Majority-Latino District Density Change and Symmetry Change
latino_29847567976_o

Figure 4: Majority-Black District Density Change and Symmetry Change
black_29256018873_o

By contrast, states with increased majority-Black densities have clearly adopted more biased districting plans. Among the states with substantial reductions in partisan symmetry, only Louisiana (−1.9 %), Ohio (−1.0 %), and Pennsylvania (−0.9 %) had lower average percentages of African Americans in their majority-minority districts after redistricting. The three states with the largest increases in majority-Black district density, Tennessee (5.2 %), North Carolina (2.4 %), and Virginia (2.2 %), include some of the most biased plans in the country. This is not in any way required by the Voting Rights Act – indeed, it reduces the influence of African-American voters by using their votes inefficiently. However, it is consistent with a policy of state legislatures seeking partisan advantage by packing African-American voters, who overwhelmingly vote for the Democratic Party, into districts where the Democratic margin will be far higher than necessary.

Demography is not destiny

The bias we observe is not the inevitable effect of factors such as the urban concentration of Democratic voters or the need to draw majority-minority districts. It is for the most part possible to draw unbiased districting plans in spite of these constraints. Thus if state districting authorities draw districts that give a strong advantage to one party, this is a choice they have made – it was not forced on them by geography. The high level of partisan bias protecting GOP House control can only be explained in political terms. As we show in our book, Pro-Republican bias increased almost exclusively in states where the GOP controlled the districting process.

One of the immediate consequences of unrestrained partisan gerrymandering is that, short of a landslide Democratic victory resembling 2008, the Republicans are very likely to retain control of the House. But one of the more profound consequences is that redistricting has upended one of the bedrock constitutional principles, popular sovereignty. Without an intervention by the courts, political parties are free to manipulate House elections to their advantage without consequence.

 

“Votes that count” in different electoral systems

Throughout the discussion on electoral-system reform in Canada, I have seen various social-media posts that purport to compare the percentage of votes that “count” in Canada vs. in existing PR systems. Typically, these posts will cite a figure of 50% of votes counting in Canada and 95% or more in PR countries.

The numbers seemed fishy to me. I do not doubt that votes are substantially more likely to “count” towards election of a representative under PR than under FPTP or other majoritarian systems. But the 95% figure seemed too high.

As I happen to have a dataset of district-level electoral results in many countries at my disposal, it was not too hard to subject the claim to a test. The harder part is knowing how to operationalize “count”. I chose two methods, based on my understanding of the complaint that reform advocates have against FPTP. The problem, it will be seen, as I suspect both methods are being used by these advocates, but different methods for different systems.

The first method is the one that I believe they are applying to Canada, which is: “did my vote contribute to the election of someone in my district?” Guess what–when your district elects one, for many voters the answer is “no”. (And this may be sufficient reason to want to ditch the system!) The second method is one that I suspect the authors of these posts apply when looking at PR systems: “Did the party I voted for win representation?” That is, I suspect a district-level standard is being applied to FPTP, but a national-level one to PR countries. My calculations seem to bear that out.

If we use the first method–district-level count of votes for parties (or candidates) that did not win in the district, we almost nail the 50% figure for the most recent Canadian election for which I have data. In 2011, I get 50.4% of votes across the country “counting” in the sense that they were cast for the winner in the voter’s district.

Across the 25 FPTP elections for which I have complete data, the average figure is 55.8%. The lowest figures are around 47% in several UK and Indian elections, with the highest being 62% or more in the US and Barbados. (What do these latter two countries have in common? Very few votes going to parties other than the top two.)

I then apply that same standard to PR systems. A pause is needed here. Canada is very highly unlikely to adopt nationwide PR, either with a single national district (which I think we can say is politically and constitutionally impossible) or with districts but also nationwide compensation (as in Germany or Denmark, among others). Thus I consider the relevant metric to be those PR systems that employ districts, plural, and no nationwide compensation.

Using the same standard–votes “count” when cast for a party that wins a seat in the district–the mean for (districted) PR systems is 87.2%. That figure is a lot higher than 50%, I will grant. It is also a good deal lower than 95%.

If we look at nationwide (single-district) PR, guess what? 96.5%! A few specific elections come in at 99%, such as the Netherlands, 2002, and Israel, 1951. (Most Israeli and Dutch elections are over 97%, but Netherlands, 1952, was at a paltry 94.7%.) That’s great! However, most PR advocates, unless they are real purists (and not at all realists) do not advocate the adoption of the Dutch or Israeli type of PR.

What about the second standard? Under this one, your vote “counts” if it was cast for a party that won representation somewhere (at least one seat), even if it did not win in your district. As I noted above, I suspect this is the standard being applied, at least implicitly, to the PR countries in the social-media posts I have seen.

By this standard, districted PR systems’ average percentage of votes that count rises to 93.7%. We are almost to 95%! For nationwide PR, it obviously does not change (there’s only one district). What about for FPTP? 97.1%. Canada, 2011, comes in at 99.1%.

I do not actually know if a Canadian voter feels “represented” if she voted NDP but the NDP candidate in her district lost. I suspect many do feel so represented, or else the NDP would not get more than trivial vote shares in districts where it has no chance of winning. Yet it does. Greens can also get votes near or above their nationwide percentage even in many districts that are totally hopeless for them. Perhaps they feel represented by Elizabeth May (the one Green MP) even if they reside in a different district or province. Not as well represented as if their own MP was Green, presumably, but my point is that voters probably tend to think of national party systems when voting, even in districted systems. Yes, even in systems in which districts have one seat apiece.

There may be many reasons to prefer PR over FPTP. I can think of quite a few myself. But the idea of a vote “counting” towards representation may not be one of the more meaningful criteria to use. Or, if it is used, it might be OK not to exaggerate. The difference between 50% (Canada, 2011) and 87% (mean for districted PR) is impressive enough, using the first (district-based) criterion. We don’t need to pretend that twice as many votes “count” under PR as under FPTP. 1.74 times as many is still a lot more!