Case studies and case selection Gary Goertz Kroc Institute for - - PowerPoint PPT Presentation

Case studies and case selection

Gary Goertz Kroc Institute for International Peace Studies University of Notre Dame ggoertz@nd.edu Spring 2018

One uses case studies based on Z to answer the question: “How generalizable is the case study?”

Large-N qualitative testing

There are no cross-case comparisons used for causal inference.

Large-N qualitative testing: some examples

Target authors: Acemoglu, D., and J. Robinson. 2006. Economic

• rigins of dictatorship and democracy. Cambridge: Cambridge

University Press. Critics: Haggard, S. and Kaufman, R. 2012. Inequality and regime change: democratic transitions and the stability of democratic rule. American Political Science Review 106:1–22. Target author: Fearon, J. 1994. Domestic political audiences and the escalation of international disputes. American Political Science Review 88:577–92. Critic: Trachtenberg, M. 2012. Audience costs: an historical analysis. Security Studies 21:3–42. Target authors: Mansfield, E., and J. Synder. 2005. Electing to fight: why emerging democracies go to war. Cambridge: MIT Press. Critics: Narang, V. and Nelson, R. 2009. Who are these belligerent democratizers? Reassessing the impact of democratization on war. IInternational Organization 63:357–79.

Why large-N qualitative testing is possible: democratization and war

Not Demo*Weak Inst. War 111 6 Not war 227 142

Recent work by Carles Boix and Daron Acemoglu and James Robinson has focused on the role of inequality and distributive conflict in transitions to and from democratic rule. We assess these claims through causal process observation, using an

• riginal qualitative dataset on democratic transitions and

reversions during the “third wave” from 1980 to 2000. (Haggard and Kaufman 2012, 1, complete abstract).

Large-N qualitative testing: basic procedure

• 1. The target authors are quite clearly identified.
• 2. The target works are game theoretic or statistical.
• 3. The methodology of testing is looking for or at the causal

mechanism and process tracing in individual cases.

• 4. Each case is coded as showing the causal mechanism or

not.

• 5. Critics look at “all” cases.
• 6. They base their conclusions on the percentage of case

studies where the causal mechanism is present. Typically the percentage of cases where the causal mechanism is present is low, and ideally close to zero.

Defining scope and populations: audience costs

I will be looking at a set of crises – episodes in which there was a significant perceived risk of war – involving great powers, at least one of which was a democracy, and that were settled without war. These criteria were chosen for the following

• reasons. The cases are all crises because the Fearon theory

explicitly deals with crises, but I will be looking only at great power crises for essentially practical reasons. . . . The focus here, moreover, is on crises in which at least one of the contending parties is a democracy, since much of the debate

• n this issue has to do with whether the audience costs

mechanism gives democracies an advantage over non-democratic regimes. This means that the crises to be examined all took place after 1867. . . . Finally, only those crises that did not terminate in war will be examined here. . . . That set

• f criteria generates a list of about a dozen crises.

(Trachtenberg 2012, 5–6)

X-centric strategy: choose on X = 1 column, e.g., where audience costs should be present and working.

Results of large-N qualitative testing: audience costs

So what conclusion is to be drawn from the discussion in this whole section of the great power crises won by democratic states [(1,1) cases]? The basic finding is quite simple. There is little evidence that the audience costs mechanism played a “crucial” role in any of them. Indeed, it is hard to identify any case in which that mechanism played much of a role at all. There are all kinds of ways in which new information is generated in the course of a crisis, and that new information, for the reasons Fearon outlined, plays a fundamental role in determining how that crisis runs its course. Audience costs, however, were not a major factor in any of the crises examined

• here. (Trachtenberg 2012, 32)

The Y-centric strategy: exploring equifinality

We assess these claims through causal process observation, using an original qualitative dataset on democratic transitions and reversions during the “third wave” from 1980 to 2000. Haggard and Kaufman 2012, abstract)

The Y-centric strategy: exploring equifinality

We assess these claims through causal process observation, using an original qualitative dataset on democratic transitions and reversions during the “third wave” from 1980 to 2000. Haggard and Kaufman 2012, abstract) Against theoretical expectations, a substantial number of these transitions occur in countries with high levels of inequality. Less than a third of all reversions are driven by distributive conflicts between elites and masses. We suggest a variety of alternative causal pathways to both transitions and reversions. (Haggard and Kaufman 2012, abstract).

The convergence of X-centric and Y-centric strategies: the (0,0) cell is not used.

Comparative case studies mimics statistics

The comparative [case study] method can now be defined as the method of testing hypothesized empirical relationships among variables on the basis of the same logic that guides the statistical method, but in which the cases are selected in such a way as to maximize the variance of the independent variables and to minimize the variance of the control variables. (Lijphart 1975, 164, emphasis in original) [A] good case (or set of cases) for purposes of causal analysis is generally one that exemplifies quasi-experimental properties, that is, it replicates the virtues of a true experiment even while lacking a manipulated treatment. (Gerring and Cojocaru 2017, 397)

Case study: definition

A case study, for present purposes, is an intensive study of a single case or a small number of cases that promises to shed light on a larger population of cases. (Gerring and Cojocaru 2016, 394)

Typologies of case studies

Lijphart (1971:691) proposes six case study types: a-theoretical, interpretative, hypothesis-generating, theory-confirming, theory-infirming, and deviant. Eckstein (1975) identifies five species: configurative-idiographic, disciplined-configurative, heuristic, plausibility probes, and crucial-case. Skocpol and Somers (1980) identify three logics

• f comparative history: macrocausal analysis, parallel

demonstration of theory, and contrast of contexts. Gerring (2007a) and Seawright and Gerring (2008) identify nine techniques: typical, diverse, extreme, deviant, influential, crucial, pathway, most-similar, and most-different. Levy (2008) identifies five case study research designs: comparable, most-likely, least-likely, deviant, and process tracing . . . .

Typologies, Gerring

Table 1. Case Selection Strategies. Goals/Strategies n Factors Criteria for Cases

• I. Descriptive (to describe)

Typical 1þ D Mean, mode, or median of D Diverse 2þ D Typical subtypes

• II. Causal (to explain Y)
• 1. Exploratory (to identify Hx)

Outcome 1þ Y Maximize variation in Y Index 1þ Y First instance of DY Deviant 1þ Z Y Poorly explained by Z Most-similar 2þ Z Y Similar on Z, different on Y Most-different 2þ Z Y Different on Z, similar on Y Diverse 2þ Z Y AllpossibleconfigurationsofZ(assumption:X2Z)

• 2. Estimating (to estimate Hx)

Longitudinal 1þ X Z X changes, Z constant or biased against Hx Most-similar 2þ X Z Similar on Z, different on X

• 3. Diagnostic (to assess Hx)

Influential 1þ X Z Y Greatest impact on P(Hx) Pathway 1þ X Z Y X!Y strong, Z constant or biased against Hx Most-similar 2þ X Z Y Similar on Z, different on X and Y

Note: D ¼ descriptive features (other than those to be described in a case study); Hx ¼ causal hypothesis of interest; P(Hx) ¼ the probability of Hx; X ¼ causal factor(s) of theoretical interest; X!Y ¼ apparent or estimated causal effect, which may be strong (high in magnitude) or weak; Y ¼ outcome of interest; Z ¼ vector of background factors that may affect X and/or Y.

Exploratory case studies

• Outcome. An outcome case maximizes variation on the
• utcome of interest. This may be achieved by a case that

exhibits extreme values on Y (or ∆Y). (Gerring and Cojocaru 2016, 398)

Deviant Case

A deviant case deviates from an expected causal pattern, as suggested by theories or common sense, registering a surprising result. (Gerring and Cojocaru 2016, 399)

Most-different type, method of agreement?

Most-different cases (aka the method of agreement) vary widely in background factors regarded as potential causes (Z), while sharing a common outcome (Y). The assumption is that background factors that differ across the cases are unlikely to be causes of Y since that outcome is constant across the

• cases. The hope is that if a factor (X) can be identified that is

constant across the cases it may be the cause of Y. (Gerring and Cojocaru 2016, 399–400)

Influential type

An influential case is one whose status has a profound effect on the probability of a hypothesis being true, P(Hx) . . . . In social science settings, the most influential cases are usually those that falsify, or threaten to falsify, a hypothesis. Decisively corroborating cases are rare. (Gerring and Cojocaru 2016, 403)

Influential type

An influential case is one whose status has a profound effect on the probability of a hypothesis being true, P(Hx) . . . . In social science settings, the most influential cases are usually those that falsify, or threaten to falsify, a hypothesis. Decisively corroborating cases are rare. (Gerring and Cojocaru 2016, 403) “Influential cases may take the form of crucial cases, if certain background conditions hold (Eckstein 1975). If the goal is to prove a hypothesis, the crucial case is known as a least-likely case.” (Gerring and Cojocaru 2016, 404)

Crucial, least-likely, and extrapolation types

Influential cases may take the form of crucial cases, if certain background conditions hold (Eckstein 1975). If the goal is to prove a hypothesis, the crucial case is known as a least-likely

• case. (Gerring and Cojocaru 2016, 404)

Pathway type

A pathway case is one where the apparent impact of X on Y conforms to theoretical expectations and is strongest (in magnitude), while background conditions (Z) are held constant

• r exert a “conservative” bias. This might also be called a

conforming or typical case, since it conforms to or typifies a causal relationship of interest.

Algorithmic

Algorithmic case selection follows a set of rules executed in a sequence of steps, which we envision as follows.

• 1. Define the research question and the population of theoretical

interest.

• 2. Identify a sample of potential cases. Ideally, this sampling frame

should be representative of the population of interest.

• 3. Measure relevant features of the cases—for example, D, X, Y,

and/or Z—across the sample.

• 4. Combine diverse indicators of D, X, Y, and/or Z into indices, if

necessary.

• 5. Construct a causal model, if required.

(Gerring and Cojocaru 2016, 411, emphasis is mine)

Exercise: coding versus causal inference

An example of standardized coding across cases is found in a recent study by Haggard and Kaufman (2012) who examine over 100 regime transitions in order to determine the role of distributional conflict in these events. The case profiles are housed in a lengthy online

• document. The published study presents the data derived from this

extensive analysis, condensed into tabular formats. The role of the in-depth qualitative investigation is thus to arrive at a binary coding of each case—as “distributive” or “nondistributive.” It is an ingenious study, and evidence of extraordinary labor on the part of a coordinated research team. However, it is hardly a case study in the sense in which we have defined the term. Indeed, it seems no different from any data collection project in which the authors conduct careful, nose-to-the-grindstone coding and preserve their notes in a codebook.

Representative cases

When selecting cases, one aims for cases that are representative of a larger population. . . . If the chosen case(s) is representative of the population – in whatever ways are relevant for the hypothesis at hand – then one has jumped the first hurdle to external validity. (Gerring 2016, 144)

Representative cases

When selecting cases, one aims for cases that are representative of a larger population. . . . If the chosen case(s) is representative of the population – in whatever ways are relevant for the hypothesis at hand – then one has jumped the first hurdle to external validity. (Gerring 2016, 144) Once again, though, we run into a problem of representativeness. If

• ne is selecting a few cases from a larger set, why this one and not

another? Why shouldn’t the reader be suspicious about selection of “good cases” if no explanation is given for the choice? If an explanation is given and it amounts to convenience sampling, don’t we still need to worry about representativeness? (Fearon and Laitin 2008, 762–63)

Representative cases

When selecting cases, one aims for cases that are representative of a larger population. . . . If the chosen case(s) is representative of the population – in whatever ways are relevant for the hypothesis at hand – then one has jumped the first hurdle to external validity. (Gerring 2016, 144) Once again, though, we run into a problem of representativeness. If

• ne is selecting a few cases from a larger set, why this one and not

another? Why shouldn’t the reader be suspicious about selection of “good cases” if no explanation is given for the choice? If an explanation is given and it amounts to convenience sampling, don’t we still need to worry about representativeness? (Fearon and Laitin 2008, 762–63) The second principle for gaining external validity is to capture representative variation. Such empirical works are most likely to

• generate. . . . externally valid findings when the variation in the sample

broadly mirrors variation in some broader and explicitly defined population of cases. (Slater and Ziblatt 2013, 1311–12)