Are criminologists describing randomized controlled trials in ways that allow us to assess them? Findings from a sample of crime and justice trials

Descriptive validity is an important factor in assessing the transparent reporting of randomized controlled trials (RCTs). Measures of validity in crime and justice have reported on this issue but there has been a lack of standardization in comparison to other discipline areas (e.g., healthcare) where tools such as the Consolidated Standards of Reporting Trials (CONSORT) Statement have improved reporting standards. In this study, we evaluate crime and justice trials from five different settings (community, prevention, policing, correctional, and court) and assess the extent to which they transparently report information using the CONSORT Statement as a guide. Overall, the findings suggest that crime and justice studies have low descriptive validity. Reporting was poor on methods of randomization, outcome measures, statistical analyses, and study findings, though much better in regard to reporting of background and participant details. We found little evidence of improvement in reporting over time and no significant relationship between the number of CONSORT items reported and size of the trial sample. In conclusion, we argue that the state of descriptive validity in crime and justice is inadequate, and must change if we are to develop higher-quality studies that can be assessed systematically. We suggest the adoption of a modified CONSORT Statement for crime and justice research.