U.S. flag

An official website of the United States government, Department of Justice.

NCJRS Virtual Library

The Virtual Library houses over 235,000 criminal justice resources, including all known OJP works.
Click here to search the NCJRS Virtual Library

Beyond Random Assignment - Factors Affecting Evaluation Integrity

NCJ Number
75886
Journal
Evaluation Review Volume: 5 Issue: 1 Dated: (February 1981) Pages: 51-67
Author(s)
E L Rezmovic; T J Cook; L D Dobson
Date Published
1981
Length
17 pages
Annotation
Experience gained from an experimental evaluation of a Chicago program serving ex-offenders is reported and used as the basis for a discussion of practical problems in maintaining the integrity of evaluation studies.
Abstract
These problems revolved around budgeting, randomization, the data collectors, interview payments, and evaluation of black box treatments. The program dealt with ex-offenders' employment and social/personal needs. The program evaluation focused on the program's social reintegration component, in which about 1,000 parolees per year were each assigned to 1 of 3 treatment models. The models included a staff associate model, in which private citizens were matched with individual ex-offenders for a 1-year period; a VISTA model, in which VISTA volunteers who were ex-offenders were assigned caseloads of 15 to 20 clients for 3-month intervals; and a paraprofessional model, in which employed paraprofessionals handled caseloads of 25 to 30 clients for 6-month periods. Clients were randomly assigned to one of the treatment models in order to facilitate the evaluation. The inclusion of evaluation funds in the program's overall operating budget was designed to strengthen both the program and the evaluation but created a sense of competition for scarce resources. The random assignment process was not always followed, due to staff errors and lack of understanding of the assignment process. Use of data collectors not closely allied to the research effort resulted in some fraudulent reporting of interviews because of data collectors' frustration with the task level, personal problems, and concrete benefits of cheating. Compensation of research subjects had both benefits and problems. Through monitoring of treatment implementation was impossible, resulting in a 'black box' effect. Overall, a strong experimental design cannot stand alone, and nontechnical aspects of evaluation can significantly affect the validity of research results. Both research experience and research methodology should be communicated when evaluation results are reported. Fifteen references are listed.

Downloads

No download available

Availability