Spurious majorities in the US House in Comparative Perspective

In the week since the US elections, several sources have suggested that there was a spurious majority in the House, with the Democratic Party winning a majority–or more likely, a plurality–of the votes, despite the Republican Party having held its majority of the seats.

It is not the first time there has been a spurious majority in the US House, but it is quite likely that this one is getting more attention ((For instance, Think Progress.)) than those in the past, presumably because of the greater salience now of national partisan identities.

Ballot Access News lists three other cases over the past 100 years: 1914, 1942, and 1952. Sources disagree, but there may have been one other between 1952 and 2012. Data I compiled some years ago showed a spurious majority in 1996, if we go by The Clerk of the House. However, if we go by the Federal Election Commission, we had one in 2000, but not in 1996. And I understand that Vital Statistics on Congress shows no such event in either 1996 or 2000. A post at The Monkey Cage cites political scientist Matthew Green as including 1996 (but not 2000) among the cases.

Normally, in democracies, we more or less know how many votes each party gets. In fact, it’s all over the news media on election night and thereafter. But the USA is different. “Exceptional,” some say. In any case, I am going to go with the figure of five spurious majorities in the past century: 1914, 1942, 1952, 2012, plus 1996 (and we will assume 2000 was not one).

How does the rate of five (or, if you like, four) spurious majorities in 50 elections compare with the wider world of plurality elections? I certainly do not claim to have the universe of plurality elections at my fingertips. However, I did collect a dataset of 210 plurality elections–not including the USA–for a book chapter some years ago, ((Matthew Soberg Shugart, “Inherent and Contingent Factors in Reform Initiation in Plurality Systems,” in To Keep or Change First Past the Post, ed. By AndrĂ© Blais. Oxford: Oxford University Press, 2008.)) so we have a good basis of comparison.

Out of 210 elections, there are 10 cases of the second party in votes winning a majority of seats. There are another 9 cases of reversals of the leading parties, but where no one won over 50% of seats. So reversals leading to spurious majority are 4.8% of all these elections; including minority situations reversals are 9%. The US rate would be 10%, apparently.

But in theory, a reversal should be much less common with only two parties of any significance. Sure enough: the mean effective number (N) of seat-winning parties in the spurious majorities in my data is just under 2.5, with only one under 2.2 (Belize, 1993, N=2.003, in case you were wondering). So the incidence in the US is indeed high–given that N by seats has never been higher than 2.08 in US elections since 1914, ((The original version of this statement, that “N is almost never more than 2.2 here” rather exaggerated House fragmentation!)) and that even without this N restriction, the rate of spurious majorities in the US is still higher than in my dataset overall.

I might also note that a spurious majority should be rare with large assembly size (S). While the US assembly is small for the country’s population–well below what the cube-root law would suggest–it is still large in absolute sense. Indeed, no spurious majority in my dataset of national and subnational elections from parliamentary systems has happened with S>125!

So, put in comparative context, the US House exhibits an unusually high rate of spurious majorities! Yes, evidently the USA is exceptional. ((Spurious majorities are even more common in the Senate, where no Republican seat majority since at least 1952 has been based on a plurality of votes cast. But that is another story.))

As to why this would happen, some of the popular commentary is focusing on gerrymandering (the politically biased delimitation of districts). This is quite likely part of the story, particularly in some sates. ((For instance, see the map of Pennsylvania at the Think Progress link in the first footnote.))

However, one does not need gerrymandering to get a spurious majority. As political scientists Jowei Chen and Jonathan Rodden have pointed out (PDF), there can be an “unintentional gerrymander,” too, which results when one party has its votes less optimally distributed than the other. The plurality system, in single-seat districts, does not tote up party votes and then allocate seats in the aggregate. It only matters in how many of those districts you had the lead–of at least one vote. Thus a party that runs up big margins in some of its districts will tend to augment its total in its “votes” column at a faster rate than it augments its total in the “seats” column. This is quite likely the problem Democrats face, which would have contributed to its losing the seat majority despite its (apparent) plurality of the votes.

Consider the following graph, which shows the distribution (via kernel densities) of vote percentages for the winning candidates of each major party in 2008 and 2010.

Kernel density winning votes 2008-10
Click image for larger version

We see that in the 2008 concurrent election, the Democrats (solid blue curve) have a very long and higher tail of the distribution in the 70%-100% range. In other words, compared to Republicans the same year, they had more districts in which they “wasted” votes by accumulating many more in the district than needed to win it. Republicans, by contrast, tended that year to win more of their races by relatively tighter margins–though their peak is still around 60%, not 50%. I want to stress, the point here is not to suggest that 2008 saw a spurious majority. It did not. Rather, the point is that even in a year when Democrats won both the vote plurality and seat majority, they had a less-than optimal distribution, in the sense of being more likely to win by big margins than were Republicans.

Now, compare the 2010 midterm election, in which Republicans won a majority of seats (and at least a plurality of votes). Note how the Republican (dashed red) distribution becomes relatively bimodal. Their main peak shifts right (in more ways than one!) as they accumulate more votes in already safe seats, but they develop a secondary peak right around 50%, allowing them to pick up many seats narrowly. That the peak for winning Democrats’ votes moved so much closer to 50% suggests how much worse the “shellacking” could have been! Yet even in the 2010 election, the tail on the safe-seats side of the distribution still shows more Democratic votes wasted in ultra-safe seats than is the case for Republicans. ((It is interesting to note that 2010 was very rare in not having any districts uncontested by either major party.))

I look forward to producing a similar graph for the 2012 winners’ distribution, but will await more complete results. A lot of ballots remain to be counted and certified. The completed count is not likely to reverse the Democrats’ plurality of the vote, however.

Given higher Democratic turnout in the concurrent election of 2012 than in the 2010 midterm election, it is likely that the distributions will look more like 2008 than like 2010, except with the Republicans retaining enough of those relatively close wins to have held on to their seat majority.

Finally, a pet peeve, and a plea to my fellow political scientists: Let’s not pretend there are only two parties in America. Since 1990, it has become uncommon, actually, for one party to win more than half the House votes. Yet my colleagues who study US elections and Congress continue to speak of “majority”, by which they mean more than half the mythical “two-party vote”. In fact, in 1992 and every election from 1996 through at least 2004, neither major party won 50% of the House votes. I have not ever aggregated the 2006 vote. In 2008, Democrats won 54.2% of the House vote, Republicans 43.1%, and “others” 2.7%. I am not sure about 2010 or 2012. It is striking, however, that the last election of the Democratic House majority and all the 1995-2007 period of Republican majorities, except for the first election in that sequence (1994), saw third-party or independent votes high enough that neither party was winning half the votes.

Assuming spurious majorities are not a “good” thing, what could we do about it? Democrats, if they are developing a systematic tendency to be victims of the “unintentional gerrymander”, would have an objective interest in some sort of proportional representation system–perhaps even as much as that unrepresented “other” vote would have.

ESP champs 2008-2010

Just poking around a bit further in the Electoral Separation of Purpose data, as pictured and explained previously.

I wondered who the “ESP Champs” were of these cycles.

For 2008, I hereby crown Gene Taylor of Mississippi, who won 74.5% in his district on the same day that Obama managed 31.7%. Now that’s separation of purpose!

He still managed 47% even in 2010. Not bad, but not good enough.

In fact, that 2010 result makes Taylor one of only four Democrats to have won, at the midterm, more than 45% of the vote in a district in which Obama had won under 35%. But to be crowned champion for 2010, you should actually have won your race. So the 2010 title belongs to…

Dan Boren of Oklahoma, who won 56.5% in a district in which Obama had won 34.5%. This result still represented a massive adverse swing against Boren, who had 70.5% in 2008. But he held on.

Boren and Taylor, by the way, are both Blue Dogs.

With ESP numbers like these, we can see why some “blue” congressmen in deeply “red” districts were less than keen these past two years in coming to the support of Obama’s policy priorities. (This was a topic that generated considerable discussion in another thread earlier this month.)

The new House and polarization

Adam Bonica has posted some must-see graphs at Ideological Cartography. The graphs really drive home just how polarized the new US House of Representatives will be. The mean Democrat and mean Republican (and I suppose “mean” has both meanings here!) will be farther apart than any recent House, and the median of the entire House will be much more to the right than any in the past–notably more than the one elected in 1994. This follows the House elected in 2006, which was by far the most left-leaning House we have seen.

Another of Bonica’s graphs shows the extent to which entering Republicans are heavily skewed right. Exiting Democrats were less concentrated at any ideological position within their party, but the ranks of the moderates are going to be notably thinner.

Bonica concludes that “The polarization resulting from the 2010 Midterms is fundamentally different and more worrisome than what had preceded it.” Worrisome indeed.

As for the Senate, he has an animated view of polarization since 1967.

Obama and ESP

When Barack Obama was elected President in 2008, the election produced the second lowest value of “Electoral Separation of Purpose” of the preceding five decades.

Electoral Separation of Purpose (ESP) is a concept developed in David J. Samuels and Matthew S. Shugart, Presidents, Parties, and Prime Ministers (Cambridge, 2010). It starts with the difference between presidential and legislative votes, at the district level, for a given party. It then can be expressed in a summary indicator by the average of the absolute values of all these differences.

For Obama and the Democrats in 2008, ESP=10.45. In the book, we considered 42 observations for the USA (both parties in 21 elections through 2004); the only one lower than what we would see in 2008 was 8.79 for Democrats in 1996, when Bill Clinton won reelection.

That ESP would be relatively low in the Obama era is yet another window on the much talked-about “polarization” of US politics: votes for Congress now tend to be more similar to presidential votes at the (House) district level. In other words, the fates of members of the House are more tied to that of their co-partisan president (or presidential candidate) than used to be the case. Voters apparently do not “want different things” from congress and president as much as they once did (for instance, 1972 and 1974, ESPs of 20.4 and 25.8, respectively).

It is worth putting the 2008 election in comparative perspective, comparing both to other countries and to past US elections. When compared to other countries, a value of 10.45 is not especially low. Even when we eliminate all cases where presidential and legislative votes are “fused” (meaning ticket-splitting is impossible, so ESP=0), we still find that the 2008 Democratic ESP is at about the 60th percentile among 383 party-year observations from around the world. Even with polarization and tied fates, there is still a lot of room for divergence between presidential and congressional vote shares in the USA.

What is interesting is the pattern of this divergence. Below is the graph, where each data point is one of the House districts in 2008. Ignore the distinction between triangles and circles for now; we’ll get to that.

USA ESP Dems 2008

(Click the image for a larger view in a new window)

It is striking that in districts where the Democrat has over 50% of the legislative vote, Obama tends to run behind his co-partisan House candidate. That is, there are notably more points above the equality line for winning House Democratic districts than there are below the diagonal. Districts where he runs ahead of the Democratic House candidate tend to be where the party loses the congressional race. For instance, if Obama won about 60% of the vote in a given district, the Democrat tended to win around two thirds of the House vote. But if Obama won around 45% of the vote, the Democratic House candidate tended to get closer to 35% of the vote.

This pattern, which would be reflected by some sort of S-curve, had I bothered to try to plot it, seems to be a common feature of US elections. The graph for Republicans in 2004 (ESP=10.98) looks very similar (see p. 135 of the book). It is not a prevalent pattern in other countries. I suspect it has something to do with the “personal vote” of Representatives; incumbents run ahead of their party’s presidential candidate because some voters who vote for the presidential candidate of the other party nonetheless support the incumbent. However, I have not yet broken the data down by incumbency. In the losing districts, of course, much of it has to do with the Democrats’ not recruiting high-quality candidates in districts they were not likely to win anyway (but having a “high-quality” presidential candidate). Of course, this is a companion to the personal-vote story, whereby the Republican candidate was stronger and able to keep for the party voters who voted for Obama.

Does the graph shed any light on the electoral debacle suffered by Democrats this week? Not directly, although one can see at a glance the numerous districts in which the Democrat won despite the district having voted for McCain. Now here is where those triangles come in: they represent the districts that the Democrats lost in the 2010 midterm election. Not surprisingly, there are a lot of those in the part of the graph where Obama’s vote is less than 50%. In fact, over half the Democratic losses came in McCain 2008 districts. If that’s not a (mini-)realignment, it certainly is a readjustment.

However, the Democrats lost 29 districts in which Obama had won a majority in 2008. And here is where the pattern of 2008 Democratic House winners frequently having run ahead of Obama becomes so important. They had a “cushion” against an adverse swing against them, stemming from Obama’s unpopularity at the midterm, and they most certainly needed it!

ESP US Dems 2010

In this second graph we see that ESP actually declined further in 2010. At first, it may seem odd that one could go from unified to divided government, yet electoral separation of purpose decreased. But that is what happened. In 2010, ESP for Democrats dropped to 10.00. Note the near disappearance of winning Democrats who are more than about ten percentage points above where Obama was in their district in 2008. In fact, what really stands out here is the extent to which Democrats who won over 50% of their own district vote are concentrated very near, or slightly below, the equality line. That’s a good case of tied fates!

The S-curve pattern is gone, other than a continued bow in losing Democratic districts, where Obama’s 2008 vote is still higher (and often by a bigger margin) than the Democratic House candidate in 2010.

There are still some survivors in McCain districts, and they are about the only ones to still be running well ahead of Obama. If they could survive the great Democratic fall of 2010, they just might survive anything.

Now for the cross-time comparison. The following graph shows the ESP values for the president’s party for every US election since 1956, except for years following reapportionment and redistricting (and 1966, for mysterious reasons).

ESP in the USA since 1956

There is a clear trend in recent elections of declining ESP. No election for which we have data had ESP for the president’s party below 12.0 until 1996. The 1970s, and to a lesser extent the 1980s, were the days of high ESP, with Republicans often winning the presidency but Democrats keeping the House. Even in 1976, when Carter won, ESP was 14.55. Maybe this explains why Carter had so much trouble with his own party: they knew the president was less popular than they were. The graph from that election (not posted; I can’t post everything!) shows a huge bow of the S-curve above the equality line where practically all the Democratic House winners are found.

But note the almost steady downward trend after 1984, when Reagan was reelected. The 1994 midterm, when Democrats lost their House majority under Clinton, showed a downward trend. So 2010 is not unique in being an election that produces a transition to divided government yet sees ESP drop. However, in spite of the decline in ESP, it was still the case then that most Democratic winners in1994 were running ahead of where Clinton had been in 1992. Part of this is owed to the three-way presidential race in 1992. (All these graphs show actual vote percentages, not percentages of the “two-party vote.”) But then Clinton and the Democrats had tightly shared fates in 1996.

After a big upward blip in ESP in 1998, when Democrats had a rare seat gain in a midterm election, we enter the 2000s with ESP hovering in the 10-12 range.^

We really are in uncharted territory by US standards. We have not seen such closely tied presidential and legislative electoral fates at any other point in the last five decades or more.

What this might mean going forward is hard to say. I don’t have that kind of ESP! Or maybe it is not so hard. If Obama is reelected in 2012, it is unlikely to be with a broad personal victory like Nixon in 1972 and Reagan in 1984, which represent two of the three highest ESP concurrent elections. (The other is 1988, when the senior Bush effectively won Reagan’s “third term.”) But therein lies a ray of good news for Democrats–who are surely looking for such rays about now. Normally, if a President is reelected, he does so without much of a “pull” on the House races. However, we have already seen two incumbent presidents win a second term with a drop in ESP. In addition to Clinton, already mentioned as the lowest US ESP so far, the same happened with G.W. Bush (ESP=12.27 when he, uh, became president in 2000,* and a drop to 10.98 in 2004).

in such a low-ESP environment, with partisan fates so tied, it is entirely plausible that a reelected Obama would carry enough of that cluster of districts near 50% to regain a House majority. If he loses, of course, then so might several more Democratic House members. Such are the perils of governing and campaigning when electoral separation of purpose is tending to run so low, by historic US standards.


^ The 1998 plot shows a large number of Democratic winners well above where they had been in 1996, and thus also well above where Clinton ran in their districts in his low-ESP reelection in 1996. (This footnote was added a couple of days after initial planting.)

* ESP for Democrats in 2000 was a little higher (13.07), presumably because Gore ran well behind many Democratic incumbents. That the value would be so much higher than it had been for the Clinton-Gore team in 1996 really drives home how much Gore failed to cement the Democratic coalition that swung so tightly behind Clinton in 1996.