Block Randomized Trials at Places: Rethinking the Limitations of Small N Experiments

David Weisburd*, Charlotte Gill

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

41 Scopus citations

Abstract

Objectives: Place-based policing experiments have led to encouraging findings regarding the ability of the police to prevent crime, but sample sizes in many of the key studies in this area are small. Farrington and colleagues argue that experiments with fewer than 50 cases per group are not likely to achieve realistic pre-test balance and have excluded such studies from their influential systematic reviews of experimental research. A related criticism of such studies is that their statistical power under traditional assumptions is also likely to be low. In this paper, we show that block randomization can overcome these design limitations. Methods: Using data from the Jersey City Drug Market Analysis Experiment (N = 28 per group) we conduct simulations on three key outcome measures. Simulations of simple randomization with 28 and 50 cases per group are compared to simulations of block randomization with 28 cases. We illustrate the statistical modeling benefits of the block randomization approach through examination of sums of squares in GLM models and by estimating minimum detectable effects in a power analysis. Results: The block randomization simulation is found to produce many fewer significantly unbalanced samples than the naïve randomization approaches both with 28 and 50 cases per group. Block randomization also produced similar or smaller absolute mean differences across the simulations. Illustrations using sums of squares show that error variance in the block randomization model is reduced for each of the three outcomes. Power estimates are comparable or higher using block randomization with 28 cases per group as opposed to naïve randomization with 50 cases per group. Conclusions: Block randomization provides a solution to the small N problem in place-based experiments that addresses concerns about both equivalence and statistical power. The authors also argue that a 50 case rule should not be applied to block randomized place-based trials for inclusion in key reviews.

Original languageEnglish
Pages (from-to)97-112
Number of pages16
JournalJournal of Quantitative Criminology
Volume30
Issue number1
DOIs
StatePublished - Mar 2014

Keywords

  • Block randomization
  • Crime and place
  • Equivalence
  • Policing
  • Randomized controlled trials
  • Statistical power

Fingerprint

Dive into the research topics of 'Block Randomized Trials at Places: Rethinking the Limitations of Small N Experiments'. Together they form a unique fingerprint.

Cite this