Obama and ESP

When Barack Obama was elected President in 2008, the election produced the second lowest value of “Electoral Separation of Purpose” of the preceding five decades.

Electoral Separation of Purpose (ESP) is a concept developed in David J. Samuels and Matthew S. Shugart, Presidents, Parties, and Prime Ministers (Cambridge, 2010). It starts with the difference between presidential and legislative votes, at the district level, for a given party. It then can be expressed in a summary indicator by the average of the absolute values of all these differences.

For Obama and the Democrats in 2008, ESP=10.45. In the book, we considered 42 observations for the USA (both parties in 21 elections through 2004); the only one lower than what we would see in 2008 was 8.79 for Democrats in 1996, when Bill Clinton won reelection.

That ESP would be relatively low in the Obama era is yet another window on the much talked-about “polarization” of US politics: votes for Congress now tend to be more similar to presidential votes at the (House) district level. In other words, the fates of members of the House are more tied to that of their co-partisan president (or presidential candidate) than used to be the case. Voters apparently do not “want different things” from congress and president as much as they once did (for instance, 1972 and 1974, ESPs of 20.4 and 25.8, respectively).

It is worth putting the 2008 election in comparative perspective, comparing both to other countries and to past US elections. When compared to other countries, a value of 10.45 is not especially low. Even when we eliminate all cases where presidential and legislative votes are “fused” (meaning ticket-splitting is impossible, so ESP=0), we still find that the 2008 Democratic ESP is at about the 60th percentile among 383 party-year observations from around the world. Even with polarization and tied fates, there is still a lot of room for divergence between presidential and congressional vote shares in the USA.

What is interesting is the pattern of this divergence. Below is the graph, where each data point is one of the House districts in 2008. Ignore the distinction between triangles and circles for now; we’ll get to that.

USA ESP Dems 2008

(Click the image for a larger view in a new window)

It is striking that in districts where the Democrat has over 50% of the legislative vote, Obama tends to run behind his co-partisan House candidate. That is, there are notably more points above the equality line for winning House Democratic districts than there are below the diagonal. Districts where he runs ahead of the Democratic House candidate tend to be where the party loses the congressional race. For instance, if Obama won about 60% of the vote in a given district, the Democrat tended to win around two thirds of the House vote. But if Obama won around 45% of the vote, the Democratic House candidate tended to get closer to 35% of the vote.

This pattern, which would be reflected by some sort of S-curve, had I bothered to try to plot it, seems to be a common feature of US elections. The graph for Republicans in 2004 (ESP=10.98) looks very similar (see p. 135 of the book). It is not a prevalent pattern in other countries. I suspect it has something to do with the “personal vote” of Representatives; incumbents run ahead of their party’s presidential candidate because some voters who vote for the presidential candidate of the other party nonetheless support the incumbent. However, I have not yet broken the data down by incumbency. In the losing districts, of course, much of it has to do with the Democrats’ not recruiting high-quality candidates in districts they were not likely to win anyway (but having a “high-quality” presidential candidate). Of course, this is a companion to the personal-vote story, whereby the Republican candidate was stronger and able to keep for the party voters who voted for Obama.

Does the graph shed any light on the electoral debacle suffered by Democrats this week? Not directly, although one can see at a glance the numerous districts in which the Democrat won despite the district having voted for McCain. Now here is where those triangles come in: they represent the districts that the Democrats lost in the 2010 midterm election. Not surprisingly, there are a lot of those in the part of the graph where Obama’s vote is less than 50%. In fact, over half the Democratic losses came in McCain 2008 districts. If that’s not a (mini-)realignment, it certainly is a readjustment.

However, the Democrats lost 29 districts in which Obama had won a majority in 2008. And here is where the pattern of 2008 Democratic House winners frequently having run ahead of Obama becomes so important. They had a “cushion” against an adverse swing against them, stemming from Obama’s unpopularity at the midterm, and they most certainly needed it!

ESP US Dems 2010

In this second graph we see that ESP actually declined further in 2010. At first, it may seem odd that one could go from unified to divided government, yet electoral separation of purpose decreased. But that is what happened. In 2010, ESP for Democrats dropped to 10.00. Note the near disappearance of winning Democrats who are more than about ten percentage points above where Obama was in their district in 2008. In fact, what really stands out here is the extent to which Democrats who won over 50% of their own district vote are concentrated very near, or slightly below, the equality line. That’s a good case of tied fates!

The S-curve pattern is gone, other than a continued bow in losing Democratic districts, where Obama’s 2008 vote is still higher (and often by a bigger margin) than the Democratic House candidate in 2010.

There are still some survivors in McCain districts, and they are about the only ones to still be running well ahead of Obama. If they could survive the great Democratic fall of 2010, they just might survive anything.

Now for the cross-time comparison. The following graph shows the ESP values for the president’s party for every US election since 1956, except for years following reapportionment and redistricting (and 1966, for mysterious reasons).

ESP in the USA since 1956

There is a clear trend in recent elections of declining ESP. No election for which we have data had ESP for the president’s party below 12.0 until 1996. The 1970s, and to a lesser extent the 1980s, were the days of high ESP, with Republicans often winning the presidency but Democrats keeping the House. Even in 1976, when Carter won, ESP was 14.55. Maybe this explains why Carter had so much trouble with his own party: they knew the president was less popular than they were. The graph from that election (not posted; I can’t post everything!) shows a huge bow of the S-curve above the equality line where practically all the Democratic House winners are found.

But note the almost steady downward trend after 1984, when Reagan was reelected. The 1994 midterm, when Democrats lost their House majority under Clinton, showed a downward trend. So 2010 is not unique in being an election that produces a transition to divided government yet sees ESP drop. However, in spite of the decline in ESP, it was still the case then that most Democratic winners in1994 were running ahead of where Clinton had been in 1992. Part of this is owed to the three-way presidential race in 1992. (All these graphs show actual vote percentages, not percentages of the “two-party vote.”) But then Clinton and the Democrats had tightly shared fates in 1996.

After a big upward blip in ESP in 1998, when Democrats had a rare seat gain in a midterm election, we enter the 2000s with ESP hovering in the 10-12 range.^

We really are in uncharted territory by US standards. We have not seen such closely tied presidential and legislative electoral fates at any other point in the last five decades or more.

What this might mean going forward is hard to say. I don’t have that kind of ESP! Or maybe it is not so hard. If Obama is reelected in 2012, it is unlikely to be with a broad personal victory like Nixon in 1972 and Reagan in 1984, which represent two of the three highest ESP concurrent elections. (The other is 1988, when the senior Bush effectively won Reagan’s “third term.”) But therein lies a ray of good news for Democrats–who are surely looking for such rays about now. Normally, if a President is reelected, he does so without much of a “pull” on the House races. However, we have already seen two incumbent presidents win a second term with a drop in ESP. In addition to Clinton, already mentioned as the lowest US ESP so far, the same happened with G.W. Bush (ESP=12.27 when he, uh, became president in 2000,* and a drop to 10.98 in 2004).

in such a low-ESP environment, with partisan fates so tied, it is entirely plausible that a reelected Obama would carry enough of that cluster of districts near 50% to regain a House majority. If he loses, of course, then so might several more Democratic House members. Such are the perils of governing and campaigning when electoral separation of purpose is tending to run so low, by historic US standards.

^ The 1998 plot shows a large number of Democratic winners well above where they had been in 1996, and thus also well above where Clinton ran in their districts in his low-ESP reelection in 1996. (This footnote was added a couple of days after initial planting.)

* ESP for Democrats in 2000 was a little higher (13.07), presumably because Gore ran well behind many Democratic incumbents. That the value would be so much higher than it had been for the Clinton-Gore team in 1996 really drives home how much Gore failed to cement the Democratic coalition that swung so tightly behind Clinton in 1996.

Presidentialization in Toronto

A dimension of comparison that scholars of political parties have not paid enough attention to–and that means me, too–is the presence and impact of direct election of big-city mayors in countries that are parliamentary at higher levels of government (national, state/provincial, etc.).

This week’s election of the mayor of Toronto is a case in point. With direct election, we see some of the same dynamics of “presidentialization” at the local level that David Samuels and I find for national-level presidential politics in Presidents, Parties, and Prime Ministers.

The election results show a victory by Rob Ford, with 47% in a multi-candidate field.

Like many a presidential candidate, he won by emphasizing himself, personally, as an agent of change. And even though he has served on the city council, he is an “outsider” in the sense of not having allies, as the Globe and Mail commented the day after the election:

Before he ran for mayor, Mr. Ford was an isolated city councillor who often failed to understand the issues he was ranting about at city council. As a candidate, he ran on a series of simplistic slogans that say nothing about the real problems of a grown-up city.

Certainly, not the sort of leader who could become “PM” of the city, were the city to have a parliamentary system like the province of Ontario or Canada at the federal level.

The election represents “a wave of voter anger” and saw a record number of new faces elected to the city council. However, none of them are part of a party or team elected specifically to support Ford. Indeed, Ford comes into office with what the Globe and Mail calls “an aggressive agenda of cost-cutting but also a proposal to slice council in half.” One doubts the council will cheer that idea.

The election is by first-past-the-post, and it had many of the classic dynamics of FPTP in a multi-candidate field. The second-place candidate, George Smitherman, had 35.6%, and Joe Pantalone was third with 11.7%. The fourth candidate, Rocco Rossi, dropped out of the campaign, saying:

Despite my efforts to focus this race around issues and ideas that I feel matter, it has become clear that the majority of Torontonians have parked their support with one of two candidates: Mr. Smitherman or Mr. Ford.

There were various calls for Pantalone, a long-time city “insider,” to do the same. He was backed by the New Democratic Party, although evidently not with much effect.

All federal MPs from Toronto are currently Liberal or NDP. Yet the voters of the city have taken advantage of the direct election to choose a right-wing mayor.

With direct election, the process of selecting the mayor of Toronto could hardly be more different than the selection of the premier of the province or the PM of Canada. The federal parties may be “taking notes” on the Ford campaign, but the lessons will go only so far, given that the differences in executive type that structure their campaigns.

Some other parliamentary democracies also have directly elected mayors of large cities, including Japan and the U.K. There may well be a literature about this “presidentialism embedded in parliamentarism” that I have missed.

Brazil election 2010

The only real question in Sunday’s presidential election in Brazil is whether Dilma Rousseff, the candidate of the incumbent Workers Party (PT), will win in one round or two. Two-term Persident Luiz Inácio (Lula) da Silva, the founder and till now only presidential candidate of the PT, is barred constitutionally from running again.

The congressional elections will be interesting to watch. The PT takes advantage of “presidentialization” and “electoral separation of purpose” about as brilliantly as any party could. It has a now proven record of being able to build one coalition large enough to win national majorities to elect the president, while retaining its niche appeal as a cohesive leftist party in congressional elections. Its president then builds coalitions with other parties in order to govern. These coalitions and governing choices are not always to the congressional party’s liking, but it’s better to not like your president’s choices than not to have the presidency. A party so small could never dominate the executive branch in a parliamentary democracy.

The following graph shows the relationship between presidential and congressional votes of the PT in each congressional district (state) in the 1994-2006 period. Only in a few cases does the PT congressional vote come even close to the presidential vote in that state.

ESP in Brazil's PT

Consider that in 2006, Lula won 49% of the first round vote in his reelection bid, and 61% in the runoff. Yet in the Chamber of Deputies elected the same day as the first round, the PT won 15% of the vote and 83 of the 513 seats. In 2002 it had been similar: Lula won 46% in the first round and 61% in the second, with the PT winning 18% of the Deputies vote and 92 seats.

In fact, the votes of the presidential and congressional “branches” of the PT were negatively correlated in 2006! That is, where Lula gained votes, his party lost votes. He and his party really have learned how to fish in different ponds.

The graph and the references to “presidentialization” and “electoral separation of purpose” come from David J. Samuels and Matthew S. Shugart, Presidents, Parties, and Prime Ministers: How the Separation of Powers Affects Party Organization and Behavior. New York: Cambridge University Press, 2010.

Do parties dump their own PM less often in Westminster systems?

Building on a discussion in one of the Australia post-election threads, I checked the data for Presidents, Parties, and Prime Ministers to see if PMs in “Westminster” systems were any less frequently dismissed due to “intra-party” politics than PMs in parliamentary systems overall. To make a long story short, they are (except, apparently, in Australia).

The question came up because Julia Gillard, the current PM of Australia, replaced a co-partisan PM between elections, and Alan noted that her predecessor was the third out of the last ten so replaced. At the time I pointed out that 3 out of 10 was precisely the rate at which PMs in parliamentary systems overall are dismissed by their own parties. More precisely, we found that 30.2% of 354 PMs surveyed left office on account of internal party politics. (The remainder was about evenly split between election defeats and coalition breakdown.) So, Australia seemed “normal,” not a case where PM termination between elections was overly frequent by the comparative yardstick.

But what about compared to similar parliamentary democracies, where majority government is the norm (or at least used to be!)? That is, let’s leave out parliamentary systems where coalitions are the norm, and where perhaps some cases of intra-party conflict are really generated by the tensions of governing in coalition, and not by purely intra-party matters.

My first look at the data suggested that the rate was not much different in this subset than overall (I found 22 of 78). However, quite a few of these were coded in the detailed data as something like “Intraparty — left office voluntarily.” Now, that is clearly an intra-party replacement, so I am not second-guessing our own data coding! However, there is quite a lot of difference between a party undertaking an inter-electoral leadership change because the former PM decided on his or her own accord to leave, and a challenge to a sitting leader who goes involuntarily.

So let’s look at things again, with cases of “voluntary” departure relegated to a residual category.

Now things look rather different. There are only nine of 78 PMs in Westminster systems who leave for reasons that might be termed intra-party conflict. That’s 11.5%, which I have to agree is a good deal less than 3 of 10!

What about in the non-Westminster cases? We should also remove the “voluntary” departures from this subset. When we do, we are left with about 24% leaving due to intra-party conflict in the entire parliamentary data. In the non-Westminster subset, it’s 28%.

So there you have it. Roughly one eighth of Westminster PMs leave due to intra-party conflict, compared to well over a quarter of non-Westminster PMs. That seems pretty significant. And I thank Alan for prompting me to look deeper at the data!


The countries taken as “Westminster” for the purposes of this exercise, and their own rate of dismissals for intra-party reasons, are:

Australia (2/9)*
Bangladesh (0/3)
Canada (1/12)
India (1/14)**
Jamaica (0/6)
New Zealand (1/12)***
Sri Lanka (0/10 during its parliamentary regime)
U.K. (4/12)

The real question, of course, would be to break down majority vs. coalition (and minority) governments, not Westminster vs. non-Westminster. But the two categories are quite closely aligned.

Obviously, another big difference across parliamentary subtypes is in the percentage of PMs who leave office due to electoral defeat. Elections account for about a third of all parliamentary PM terminations, but 48.1% (38 of 79) in these Westminster systems. Electoral defeat is what we might expect to be the most common mechanism of PM termination in a Westminster system, of course. Just as obviously, inter-party conflict is almost never a cause of PM termination in Westminster systems: just five PMs in India and one in Bangladesh left for such reasons. (Perhaps we could add Whitlam, in Australia, who is coded as an “other” due to his dismissal by the Governor General, but the underlying reason for conflict was his lack of support in the second chamber.)


* This was before Rudd’s forced resignation.

** The Indian case is Desai, who headed the unwieldy (and not very Westminster) anti-Congress coalition that collapsed in 1979.

*** I excluded PMs in the PR era, since 1996. Since then, our dataset has only Shipley, who served out her term. We could add Clark, who also survived to see electoral defeat. The two of them would leave NZ, through 2008, at 1/14.

Sri Lanka’s presidential election

Votes are now being counted in Sri Lanka’s presidential election. Various news reports have indicated that citizens from the Tamil minority are expected to side mostly with opposition candidate Sarath Fonseka against incumbent Mahinda Rajapaksa. Unlike in past, war-time, elections, Tamils appear to have turned out in large numbers.

This contest is classic presidentialization in action, but it seems somewhat less classic as an example of the “instant runoff” rules that are used.

Presidentialization is manifest in the main opposition party, the United People’s Freedom Party (UNP), having endorsed as its presidential candidate a person with no links to the party whatsoever. In fact, Fonseka was the head of the army, and thus an agent of President Rajapaksa, as the Sri Lankan state crushed the Tamil Tigers (TTLE) guerrilla forces last year. The Independent describes the dilemma faced by the UNP and others seeking the defeat of Rajapaksa:

The former army chief was quickly recruited by an unlikely coalition, made up of the UNP, Muslims, Tamils and some strident nationalists who believed that in the martial, militaristic atmosphere following the crushing of the LTTE, Mr Fonseka with his chest full of medals represented their only chance of defeating the president. “You have to make the best of what there is,” admitted a senior UNP leader, Ravi Karunanayake

If Fonseka wins, what loyalty can the UNP count on? He is an army man, who launched his candidacy outside the party organization, and then was endorsed by the UNP. The UNP surely needs him far more than the reverse. Classic presidentialization.

Not so classic is the way Sri Lanka’s version of “instant runoff” for presidential elections is working in this campaign. Although there are twenty-two candidates, and voters may give a second-preference ranking as well as a first, the contest is very much a two-man race. Most strikingly, the Tamil minority does not have its own candidate, but rather is being courted by the two leading candidates. That seems much more like a plurality dynamic than an instant-runoff dynamic.

It must be noted that the electoral rule in Sri Lanka is not the alternative vote (i.e. STV with one being elected), in which the lowest-ranking candidates are sequentially eliminated until transferred preferences push one of the remaining candidates over the majority threshold. Rather, in Sri Lanka, if no candidate has a majority of first preferences, all but the top two are eliminated, and their ballots are examined for lower-ranked preferences among the remaining two. This is a more literal definition of “instant runoff” but not (I hope) what US-based IRV advocates mean to see adopted. (What to call this variant that Sri Lanka uses is not entirely clear; see Bob Richard’s comment.)

I am curious to know how IRV advocates would explain the absence of a significant Tamil candidate (or candidates of the Muslim and other minorities) in an IRV contest, but rather a plurality-like contest in which minority voters choose the “lesser evil” among the majority’s two candidates. Is it because of the war (too dangerous for a Tamil to come forward, too politically volatile for a Sinhala candidate to appeal for second preferences)? Is it because the rules in use are not the alternative vote? Or what?

(Rajapaksa was elected in 2005 by majority of first-preference votes as candidate of the United Peoples Freedom Alliance.)

In any case, this campaign does not match the “IRV” dynamic, but it most certainly does match the “presidentialization” dynamic.

New paper: Duverger’s contribution and the ‘semi-presidential’ model

I have just uploaded a working paper, “The ‘Semi-presidential’ model and its subtypes: Party presidentialization and the selection and de-selection of prime ministers,” For French Political Science Association symposium on the contributions of Maurice Duverger to political science, September 2009.

Although it is a “working paper,” it is more or less complete, as it needs to be translated for publication in France.

The paper is co-authored with David Samuels, and based on part of our nearly finished book (the older chapter drafts of which may also be accessed from the same link).

Presidents, Parties, Prime Ministers

Please visit the site for the forthcoming Samuels and Shugart book. There you may download the “final” drafts, chapter by chapter. Comments very much welcome, whether here or at our university e-mail addresses (linked at the site).

A Framework for Analysis

David J. Samuels
Matthew S. Shugart

Forthcoming, Cambridge University Press

The arrangement of the terms in the title summarizes the main conclusions of the book: Presidents above parties; parties above prime ministers.

In other words, where there is a directly elected president, the political parties become “presidentialized,” meaning that the way they organize and behave in elections and governance is shaped by the dynamics of the separation of powers. On the other hand, in parliamentary democracy, prime ministers are clearly subordinate to parties. These different authority lines have several observable implications, and the book is devoted to systematic worldwide empirical testing of those implications.

A majority of the world’s democracies since around the year 2000 have had directly elected presidencies (Figure 1.1, which is also posted at the book’s website), yet our theories of political parties remain largely grounded in either the West European parliamentary experience, or American-centric theories that often implicitly assume away the impact of the presidency. Thus “catching up” theory with the real world is one of the underlying purposes of the book. So is subsuming the US case within the broader comparative politics enterprise.

An important further finding of the book is that when there is both an elected president and a parliamentary-dependent prime minister–a semi-presidential system–the president remains “above all”; in fact, the prime minister becomes subordinate to the president, as long as the presidentialized party has effective control of parliament. In our analysis of semi-presidential systems, we confirm that the premier-presidential subtype (where the PM’s formal accountability is exclusively to the parliamentary majority) results in effective subordination of the cabinet to the parliamentary majority, when the latter is controlled by the president’s opposition. Yet even these “cohabitation” phases remain presidentialized in important respects; they also account for barely over one fifth of all presidential tenure in premier-presidential systems. (They account for only about 2% in the other suybtype, president-parliamentary; we confirm that this latter subtype is almost totally presidentialized.)

The book is oriented theoretically by the neo-Madisonian framework. The empirical scope is worldwide, including every country that has met basic democratic criteria for at least five consecutive years since 1946 (see Table 2.1). We analyze the impact of different executive-legislative structures on the selection and de-selection of presidents and prime ministers, the impact of institutional variation on electoral “fusion of purpose” (constituency overlap), and the propensity for electoral mandates to be violated (by a “policy switch”). Two chapters contain paired case studies: one demonstrates the immediate presidentialization of parties that resulted from the adoption of direct executive elections in France and Israel (and the subsequent reversal when parliamentarism was re-adopted in the latter country); the other demonstrates the impact of presidentialization on the electoral and governing experiences of two long-time opposition parties in presidential systems, the (leftist) Workers Party in Brazil and the (rightist) National Action in Mexico.

The conclusion notes several broader implications, and also calls attention to a trend: while abolition of an existing directly elected executive almost never occurs in democracies (in addition to Israel, the only other case is Moldova), some countries have adopted reforms intended to limit the president’s formal powers over the cabinet. These reforms may indicate dissatisfaction with party presidentialization, though our overarching conclusion would be that continued presidentialization is likely to prevail, even if it can be attenuated somewhat (for instance by moving to premier-presidentialism).