A question of size? A heuristics for stepwise comparative research design

There are some areas where size doesn't not matter. Social Science is not one of them since size "Large N" is widely perceived as a necessary condition for the corroboration and falsification of generalizations. Valid generalizations, it is habitually asserted, are those that have been examined against a large number of cases. Despite a recognition of the importance of case-oriented research, the prevailing view is that when one faces the choice between small-N and large-′, the larger is the better. This chapter offers some caveats regarding this conventional wisdom; it qualifies the fetishism attached to size through an ontological inquiry into the issues of inference, and proposes a practical solution for the problems of generalization in case-oriented analysis. This solution involves a stepwise comparative design, balancing internal and external validity, and a distinction among four different strategies of inference and case selection. The chapter discusses the challenges of comparative designs intended to increase the number of cases, and then offers a heuristic of comparative research design for comparative politics and policy analysis that allows a controlled, reasoned increase in the number of cases without compromising the strength of case-oriented analysis. At the centre of the discussion are issues of generalization, control, and inference in case-oriented designs. These are basic issues as old as the social sciences, and are often framed in the social science textbooks through notions such as the quantitative-qualitative divide. Unsurprisingly, considering the importance of these issues, the literature on the "divide" is voluminous (certainly too large to cite in full here). Rather than advocating the advantages of the qualitative over the quantitative, or vice versa, this chapter aims to bridge the divide, a goal that is often considered as gracious and important. Gracious since it may help to create a more genial atmosphere in an environment that is often competitive in unconstructive ways. Important, since these divisions tend somewhat to impede mutually beneficial discourse across methodological barriers. So not surprisingly, considerable efforts are periodically made in this direction (Lijphart 1971; 1975; King, Keohane, and Verba 1994). Still, gracious and important as these efforts indeed are, they may lead us astray, if, say, bridging the divide compromises the meta-goal of all sciences, namely the production and dissemination of knowledge through adversarial or critical modes of inquiry. To suggest that there is only one logic for all social science research is one example of unproductive ways of trying to bridge the divide. Not only does the advocacy of one logic' ignore the plurality of goals in social science research (seven according to Ragin 1994), it ignores the plurality of heuristics, which connects ideas and evidence in the social sciences. To argue for a monistic view of science is one problem, but even more problematic is to argue that the logic of social science research is best articulated in quantitative research, and this should serve as the model for qualitative research (Brady and Collier 2004). Quantitative research is grounded in several heuristics of analysis that are as revealing as they are constraining. If bridging the divide means less pluralism in scientific inference, and if this implies that one logic of research is to be enforced, we might best keep the barriers and the divide intact. This is not to suggest that we should maintain our methodological walls; the best option is certainly in the direction of bridging the divide. In many respects I follow in the footsteps of Charles Ragin (1987, 1994, 2000; Ragin and Becker 1992) whose treatises on methodological problems of the social science have opened the way to the development of several heuristics of comparative analysis. Charles Ragin's own advocacy of Qualitative Comparative Analysis (QCA) and Fuzzy-Set inquiry stands independent of his contributions to our understanding of the limits and strength of case-and variable-oriented research. My discussion in this chapter, and the heuristic offered, are framed in Ragin's approach for social science inquiry, yet they are closer in spirit to the simpler and more easily applied small-N design, which does not involve Computer Mediated Methodology (CMM). The drive to formalize comparative techniques of inference follows Smelser's Comparative Methods in the Social Science (1976), where the explicit and implicit techniques of some great comparativists are discussed in detail. The heuristic this chapter advances is grounded in four principals. First, a distinction between different cases according to their iiiferential roles in the research design. This justifies varied degrees of in-depth analysis of the cases under study, hence broadens the generalization without compromising the indepth study of the primary cases. Second, a stepwise, research design with two major components. The first, based on a most-similar research design. aiming to enhance internal validity (via an in-depth study of primary cases). The second builds on a most-different research design, aiming to increase external validity (via the study of secondary and tertiary cases). Third, the application of four comparative inferential strategies in various stages of the research. Finally, a formalization of the analysis, in order to improve the consistency and transparency of case selection and of the inferential process. While I discuss and advocate here a particular heuristic I also suggest that that it may trigger the development of more heuristics that will bridge the divide between case-and variable-oriented analyses in creative ways. This heuristic sets out to deal with six interrelated problems that I have encountered in my effort to enhance the validity of my own conclusions in the context of comparative political and policy-oriented research. These six problems are the subject of the first part of the chapter. The second part discusses the two major studies that advance various techniques of increasing the number of cases in the context of case-oriented designs (Lijphart 1971; King, Keohane, and Verba 1994). My discussion suggests that these techniques pay inadequate attention to the case-oriented logic of research. As an alternative, I outline the ontology of case-oriented research and discuss the criterion of consilience as it affects case selection and theory choice. The chapter then distinguishes four inferential techniques, which are framed by the interaction of Mill's methods of agreement and difference and Przeworski and Teune's distinction between a Most-Similar and a Most-Different System Design. The stepwise use of these different techniques to balance internal and external validity is then discussed. The chapter concludes with a formal presentation of a heuristic that allows an increase the number of cases in a comparative manner without compromising the strength of case-oriented analysis.