Obama and ESP

When Barack Obama was elected President in 2008, the election produced the second lowest value of “Electoral Separation of Purpose” of the preceding five decades.

Electoral Separation of Purpose (ESP) is a concept developed in David J. Samuels and Matthew S. Shugart, Presidents, Parties, and Prime Ministers (Cambridge, 2010). It starts with the difference between presidential and legislative votes, at the district level, for a given party. It then can be expressed in a summary indicator by the average of the absolute values of all these differences.

For Obama and the Democrats in 2008, ESP=10.45. In the book, we considered 42 observations for the USA (both parties in 21 elections through 2004); the only one lower than what we would see in 2008 was 8.79 for Democrats in 1996, when Bill Clinton won reelection.

That ESP would be relatively low in the Obama era is yet another window on the much talked-about “polarization” of US politics: votes for Congress now tend to be more similar to presidential votes at the (House) district level. In other words, the fates of members of the House are more tied to that of their co-partisan president (or presidential candidate) than used to be the case. Voters apparently do not “want different things” from congress and president as much as they once did (for instance, 1972 and 1974, ESPs of 20.4 and 25.8, respectively).

It is worth putting the 2008 election in comparative perspective, comparing both to other countries and to past US elections. When compared to other countries, a value of 10.45 is not especially low. Even when we eliminate all cases where presidential and legislative votes are “fused” (meaning ticket-splitting is impossible, so ESP=0), we still find that the 2008 Democratic ESP is at about the 60th percentile among 383 party-year observations from around the world. Even with polarization and tied fates, there is still a lot of room for divergence between presidential and congressional vote shares in the USA.

What is interesting is the pattern of this divergence. Below is the graph, where each data point is one of the House districts in 2008. Ignore the distinction between triangles and circles for now; we’ll get to that.

USA ESP Dems 2008

(Click the image for a larger view in a new window)

It is striking that in districts where the Democrat has over 50% of the legislative vote, Obama tends to run behind his co-partisan House candidate. That is, there are notably more points above the equality line for winning House Democratic districts than there are below the diagonal. Districts where he runs ahead of the Democratic House candidate tend to be where the party loses the congressional race. For instance, if Obama won about 60% of the vote in a given district, the Democrat tended to win around two thirds of the House vote. But if Obama won around 45% of the vote, the Democratic House candidate tended to get closer to 35% of the vote.

This pattern, which would be reflected by some sort of S-curve, had I bothered to try to plot it, seems to be a common feature of US elections. The graph for Republicans in 2004 (ESP=10.98) looks very similar (see p. 135 of the book). It is not a prevalent pattern in other countries. I suspect it has something to do with the “personal vote” of Representatives; incumbents run ahead of their party’s presidential candidate because some voters who vote for the presidential candidate of the other party nonetheless support the incumbent. However, I have not yet broken the data down by incumbency. In the losing districts, of course, much of it has to do with the Democrats’ not recruiting high-quality candidates in districts they were not likely to win anyway (but having a “high-quality” presidential candidate). Of course, this is a companion to the personal-vote story, whereby the Republican candidate was stronger and able to keep for the party voters who voted for Obama.

Does the graph shed any light on the electoral debacle suffered by Democrats this week? Not directly, although one can see at a glance the numerous districts in which the Democrat won despite the district having voted for McCain. Now here is where those triangles come in: they represent the districts that the Democrats lost in the 2010 midterm election. Not surprisingly, there are a lot of those in the part of the graph where Obama’s vote is less than 50%. In fact, over half the Democratic losses came in McCain 2008 districts. If that’s not a (mini-)realignment, it certainly is a readjustment.

However, the Democrats lost 29 districts in which Obama had won a majority in 2008. And here is where the pattern of 2008 Democratic House winners frequently having run ahead of Obama becomes so important. They had a “cushion” against an adverse swing against them, stemming from Obama’s unpopularity at the midterm, and they most certainly needed it!

ESP US Dems 2010

In this second graph we see that ESP actually declined further in 2010. At first, it may seem odd that one could go from unified to divided government, yet electoral separation of purpose decreased. But that is what happened. In 2010, ESP for Democrats dropped to 10.00. Note the near disappearance of winning Democrats who are more than about ten percentage points above where Obama was in their district in 2008. In fact, what really stands out here is the extent to which Democrats who won over 50% of their own district vote are concentrated very near, or slightly below, the equality line. That’s a good case of tied fates!

The S-curve pattern is gone, other than a continued bow in losing Democratic districts, where Obama’s 2008 vote is still higher (and often by a bigger margin) than the Democratic House candidate in 2010.

There are still some survivors in McCain districts, and they are about the only ones to still be running well ahead of Obama. If they could survive the great Democratic fall of 2010, they just might survive anything.

Now for the cross-time comparison. The following graph shows the ESP values for the president’s party for every US election since 1956, except for years following reapportionment and redistricting (and 1966, for mysterious reasons).

ESP in the USA since 1956

There is a clear trend in recent elections of declining ESP. No election for which we have data had ESP for the president’s party below 12.0 until 1996. The 1970s, and to a lesser extent the 1980s, were the days of high ESP, with Republicans often winning the presidency but Democrats keeping the House. Even in 1976, when Carter won, ESP was 14.55. Maybe this explains why Carter had so much trouble with his own party: they knew the president was less popular than they were. The graph from that election (not posted; I can’t post everything!) shows a huge bow of the S-curve above the equality line where practically all the Democratic House winners are found.

But note the almost steady downward trend after 1984, when Reagan was reelected. The 1994 midterm, when Democrats lost their House majority under Clinton, showed a downward trend. So 2010 is not unique in being an election that produces a transition to divided government yet sees ESP drop. However, in spite of the decline in ESP, it was still the case then that most Democratic winners in1994 were running ahead of where Clinton had been in 1992. Part of this is owed to the three-way presidential race in 1992. (All these graphs show actual vote percentages, not percentages of the “two-party vote.”) But then Clinton and the Democrats had tightly shared fates in 1996.

After a big upward blip in ESP in 1998, when Democrats had a rare seat gain in a midterm election, we enter the 2000s with ESP hovering in the 10-12 range.^

We really are in uncharted territory by US standards. We have not seen such closely tied presidential and legislative electoral fates at any other point in the last five decades or more.

What this might mean going forward is hard to say. I don’t have that kind of ESP! Or maybe it is not so hard. If Obama is reelected in 2012, it is unlikely to be with a broad personal victory like Nixon in 1972 and Reagan in 1984, which represent two of the three highest ESP concurrent elections. (The other is 1988, when the senior Bush effectively won Reagan’s “third term.”) But therein lies a ray of good news for Democrats–who are surely looking for such rays about now. Normally, if a President is reelected, he does so without much of a “pull” on the House races. However, we have already seen two incumbent presidents win a second term with a drop in ESP. In addition to Clinton, already mentioned as the lowest US ESP so far, the same happened with G.W. Bush (ESP=12.27 when he, uh, became president in 2000,* and a drop to 10.98 in 2004).

in such a low-ESP environment, with partisan fates so tied, it is entirely plausible that a reelected Obama would carry enough of that cluster of districts near 50% to regain a House majority. If he loses, of course, then so might several more Democratic House members. Such are the perils of governing and campaigning when electoral separation of purpose is tending to run so low, by historic US standards.

^ The 1998 plot shows a large number of Democratic winners well above where they had been in 1996, and thus also well above where Clinton ran in their districts in his low-ESP reelection in 1996. (This footnote was added a couple of days after initial planting.)

* ESP for Democrats in 2000 was a little higher (13.07), presumably because Gore ran well behind many Democratic incumbents. That the value would be so much higher than it had been for the Clinton-Gore team in 1996 really drives home how much Gore failed to cement the Democratic coalition that swung so tightly behind Clinton in 1996.

10 thoughts on “Obama and ESP

  1. This is really valuable. I have a few unrelated comments:

    1. The realignment caused in part by the 1960s civil rights legislation took awhile to work its way through the legislative level, and you had a substantial Southern/ rural wing of the Democratic Party for some time. This wing took big hits in 1994 and again in 2010, and its slow demise probably explains a good deal of the long term ESP drop.

    2. Another trend that seems clearer in retrospect than at the time is that during the Cold War, American voters liked re-electing Presidents (with the big exception of the two post Watergate elections). Eisenhower, Johnson, Nixon, and Reagan were all reelected with something around 60% of the vote and carried most of the states, and ran well ahead of the congressional candidates of their party. Plus in 1972 a substantial portion of the Democratic machine did not back its presidential nominee. This inflates the ESP value at least for 1956, 1972, and 1984. The trend also seems to have gone away with the end of the Cold War.

    3. American election law, by not regulating expenses, puts a huge cost on running a losing election campaign, both on the candidate personally and on the party and its backers. Campaigns essentially have to be two year events, meaning quitting your job and puttin the rest of your life on hold during that period, and it costs at least half a million dollars to run even a respectable losing campaign. Voters, as was confirmed again with many of the Tea Party candidates, will simply not vote for candidates they think are not “credible”, even if they dislike the incumbent. So in many Congressional districts the out party will make only a token effort, and its not unusual to not run any candidate at all. This inflates the “personal vote” for incumbent Congressmen.

  2. I think 1966 is excluded due to redistricting as well (imposed by the courts around that time).

    I agree with Ed that the time-lag between Republican breakthroughs in the South on the presidential and the congressional levels probably explains much of this. If the data was presented by region perhaps it would be even more interesting?

  3. I thought of Baker v. Carr as the reason we did not have 1966 data, but I thought that decision applied to state legislatures? Was the US House also malapportioned prior to that Supreme Court ruling?

    A state-level aggregation of the data is something I have thought about. Maybe one of these days,

  4. The U.S. House was definitely malapportioned within states, but to varying degrees of course. Some had not redistricted for decades, and even if they had standards could be lax. The courts had to intervene on a state-by-state basis, so like for the legislatures, districts were redrawn here and there throughout the 1960s.

  5. Unless one has handy an excellent resource such as the Historical Atlas of United States Congressional Districts by Kenneth C. Martis, it is actually a horrendous chore to find out exactly which areas were covered by the various districts in the various states at any one time. Partly I think this is one of the many drawbacks of having only numbered instead of geographically named districts.

    I think perhaps four or five regions would be enough to sort out major sectional particularities in the data. Maybe urban/suburban/rural would be interesting too, but that may be one of those horrendous chores.

  6. Matt

    Thanks for this posting. The book link to Cambridge provides prices in pounds–can you provide a US purchase link?

    I have a question about using this book or another book in an undergraduate class. Been using Cox’s “Making Votes Count” and Taagpera and Shugart in the first part of an elections course. For T&S I have students recalculate a few of your measures using up to date data and write a short memo.

    If I were to assign a chapter or two out of this new book, any recommendations?

  7. A bit off topic, but not entirely unrelated. There was a change of government in France, where the political fate of the president and party are very closely tied together. Or rather, there wasnt a change of Prime Minister despite the Presidents plans to change the direction of the govt in advance of the next presidential election in 2012.

    According to the French press, Sarkozy was obliged to keep his relatively conservative, but also relatively more popular PM by his parliamentary caucus, rather than appoint a new, more centrist head of govt-a situation that is regarded as unprecedented.

  8. Paul, thanks for the question. Probably chapters 5&7 are the ones most relevant to a course on elections. Of course, I think you should have your students buy the book and read all of it! (The link there goes to the US site.) Thanks also for using Taagepera and Shugart.

    DC, electoral separation of purpose has varied a lot over time, and depending on the party, in France. We have a table on the 1997 and 2002 elections on p. 136 of the book. In 2002, for the Rally for the Republic (Gaullists, who later would form the main part of the Union for a Popular Majority), ESP=16.45. Note that suggests fates a good deal less tied than that for US Democrats currently. In fact, 1990 was the last time it was that high for a US president and his party (then Republicans). I do not have 2007 district-level data for Sarkozy and the UMP in the current National Assembly.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.