U.S. flag

An official website of the United States government, Department of Justice.

NCJRS Virtual Library

The Virtual Library houses over 235,000 criminal justice resources, including all known OJP works.
Click here to search the NCJRS Virtual Library

Testing for Analysts' Bias in Crime Prevention Experiments: Can We Accept Eisner's One-Tailed Test?

NCJ Number
227271
Journal
Journal of Experimental Criminology Volume: 5 Issue: 2 Dated: June 2009 Pages: 185-200
Author(s)
Lawrence W. Sherman; Heather Strang
Date Published
June 2009
Length
16 pages
Annotation
This article attempts to evaluate Eisner's proposed remedy in testing for developer-led evaluations and independent evaluations bias in crime prevention.
Abstract
Eisner (2009) suggests that developer-led evaluations often make programs look better than independent evaluations do because the former suffer systematic biases in favor of prevention success. However, his proposed remedies suffer their own systematic bias, constituting a "one-tailed" test of bias in only one direction. This response suggests that a more objective assessment of analysts' effects requires a "two-tailed" test of bias, in which reviewers would measure indications of bias both for and against success in evaluations reported by both developers and independent evaluators. Specifically, the article explores the complex nuances of key concepts that both Eisner and other commentators fail to define adequately, it presents a fifth case for Eisner documenting the "negative developer effect" it proposes a set of indicators of analysts' biases against finding crime prevention effects, and it suggests that, if any assessment of analysts' bias is added to systematic reviews, it should be conducted in the proposed two-tailed framework. Table, figures, and references