Abstract
This paper examines how panelists serving on interdisciplinary funding panels produce evaluations they perceive as fair, drawing on 81 interviews with panelists serving on multidisciplinary fellowship competitions. We identify how peer reviewers define "good" interdisciplinary proposals and the rules they follow: respect for disciplinary sovereignty, deference to expertise and methodological pluralism. These rules ensure the preponderance of the voices of experts over non-experts. Panelists also adopt strategies to make other reviewers who lack expertise trust that their judgments are disinterested and unbiased, while reviewers who lack expertise are not afraid to make decisions based on idiosyncratic tastes rather than substantive quality.
| Original language | English |
|---|---|
| Pages (from-to) | 43-55 |
| Number of pages | 13 |
| Journal | Research Evaluation |
| Volume | 15 |
| Issue number | 1 |
| DOIs | |
| State | Published - Apr 2006 |
| Externally published | Yes |
Bibliographical note
Funding Information:Michèle Lamont acknowledges a generous grant from the National Science Foundation (grant no. SES-0096880), which made this research possible, as well as a fellowship from the Center for Advanced Study in the Behavioral Sciences, with the support of the Andrew W. Mellon Foundation (grant no. 29800639). Grégoire Mallard acknowledges the support of a graduate research fellowship from the Lurcy Foundation, which funded his work at Princeton University in 2002–2003. Joshua Guetzkow acknowledges the support of a fellowship from the Robert Wood Johnson Scholars in Health Policy Research Program.
Funding Information:
The competitions we studied were not “interdisciplinary” in the full sense of the term; all of the panelists were trained in a discipline, and all of the proposals were rooted in a discipline, even though they almost all drew on methods, materials and theories from other disciplines. In practice, this is true of most research that goes under the rubric of “interdisciplinary” (Klein, 1990). We delve into ‘procedural fairness’ (the fairness of the decision-making process as opposed to the fairness of the outcome) in Mallard et al (under review). That paper analyzes how panelists respond to epistemological diversity (the presence of diverse epistemological styles in interdisciplinary panels), yet maintain faith in the evaluation process. The present paper builds on our previous argument to address the broader topic of interdisciplinarity. For a complete discussion of all these aspects, see Lamont (forthcoming). Hoping to provide a direct empirical comparison between interdisciplinary and disciplinary panels in the social sciences and the humanities, we had secured permission to study disciplinary panels from the division of the Social and Behavioral Sciences at the National Science Foundation. However, invoking the Privacy Act, access was ultimately denied by NSF’s General Counsel’s office. The privileging of interdisciplinary proposals on these panels is underscored by a historian who explains that in order to be successful, applicants have to appeal to scholars with a range of interests and intellectual horizons. They have to “be able to hit a basic threshold of significance … and that has to do with ‘how is somebody outside of that field going to read this?’ If you can reach people outside your field, you’re interdisciplinary, you know, you’ve reached across the disciplinary divide.” The specific competitions studied were the International Dissertation Field Research Fellowship program of the Social Science Research Council and the American Council of Learned Societies; the Women’s Studies Dissertation Grant Program at the Woodrow Wilson National Fellowship Foundation; and the Fellowship Program in the Humanities of the