skip navigation

PUBLICATIONS

Register for Latest Research

Stay Informed
Register with NCJRS to receive NCJRS's biweekly e-newsletter JUSTINFO and additional periodic emails from NCJRS and the NCJRS federal sponsors that highlight the latest research published or sponsored by the Office of Justice Programs.

NCJRS Abstract

The document referenced below is part of the NCJRS Virtual Library collection. To conduct further searches of the collection, visit the Virtual Library. See the Obtain Documents page for direction on how to access resources online, via mail, through interlibrary loans, or in a local library.

 

NCJ Number: 65334 Add to Shopping cart Find in a Library
Title: SENSITIVITY, BIAS, AND THEORY IN IMPACT EVALUATION (FROM EVALUATION STUDIES REVIEW ANNUAL, VOL 4, 1979, BY LEE SECHREST ET AL - SEE NCJ-65332)
Author(s): R F BORUCH; H GOMEZ
Corporate Author: Sage Publications, Inc
United States of America
Date Published: 1979
Page Count: 26
Sponsoring Agency: Sage Publications, Inc
Thousand Oaks, CA 91320
US Dept of Health, Education, and Welfare
Washington, DC 20203
Contract Number: NIE-C-74-0115
Type: Program/Project Evaluation
Format: Document
Language: English
Country: United States of America
Annotation: TO DEVELOP MORE RIGOROUS AND SOCIALLY BENEFICIAL PROGRAM EVALUATION, THIS ARTICLE IDENTIFIES AND ATTEMPTS TO SOLVE MEASUREMENT PROBLEMS WHICH, IN USUAL EXPERIMENTAL DESIGN, INVITE UNDERPOWERED (INSENSITIVE) EXPERIMENTS.
Abstract: MEASUREMENT SHOULD BE CONCERNED NOT ONLY WITH THE RELIABILITY OF THE DEPENDENT VARIABLE BUT, MORE IMPORTANTLY, WITH VALIDITY OF MEASUREMENT AND WITH MEASUREMENT, THROUGH SYSTEMATIC OBSERVATION, OF THE TREATMENT VARIABLE. IN AN IMPACT EVALUATION, WHICH ENTAILS THE MEASUREMENT OF THE RELATIVE EFFECT OF A PROGRAM ON ITS TARGET GROUP, EVALUATORS SHOULD BE ABLE TO ANTICIPATE THE POWER OF AN EVALUATION DESIGN, ENHANCE THAT POWER, AND UNDERSTAND THE EVALUATION RESULTS. WHEN A PROGRAM IS TRANSFERRED FROM THE LABORATORY TO THE FIELD, THE PRECIPITOUS DROP IN STATISTICAL POWER RESULTS FROM THE MULTIPLICATIVE EFFECTS OF INVALIDITY OF RESPONSE AND INFIDELITY OF TREATMENT. TO DEAL WITH THIS PROBLEM, ORTHODOX MEASUREMENT THEORY SHOULD BE AUGMENTED BY INPUT FROM PROGRAM MANAGERS, DEVELOPERS, AND THE SUBSTANTIVE THEORIST. THE RESPONSE VARIABLE MUST BE MEASURED USING CRITERIA RELEVANT TO THE TREATMENT VARIABLE. CRITERIA FOR DETERMINING RELEVANCE INCLUDE SUBSTANTIVE THEORY, PRIOR DATA, PILOT TESTS, SIDE THEORIES, AND OTHER MODELS FOR UNDERSTANDING THE RESPONSE VARIABLES' RELATIONSHIP TO THE ACTUAL STATUS OF THE INDIVIDUAL AND TO PROGRAM CONTENT. THE TREATMENT VARIABLE MUST ALSO BE EXAMINED, RATHER THAN ASSUMING THAT THE PRESCRIBED LEVEL OF TREATMENT IS EITHER PRESENT OR ABSENT. THE THREE MAIN PROBLEMS IN IDENTIFYING AND MEASURING TREATMENT IN FIELD STUDIES ARE (1) POLICY PACKAGED AS PROGRAM, (2) THE STRUCTURALLY INCOMPLETE PROGRAM, AND (3) INCOMPLETE PROGRAM RECEPTION OR DELIVERY AT THE INDIVIDUAL LEVEL. PILOT TESTS OR SIDE STUDIES, MOUNTED PRIOR TO A FULL-SCALE EVALUATION, SEEM WELL JUSTIFIED HERE. IT IS CONCLUDED THAT ORTHODOX EXPERIMENTAL DESIGN AND QUASI-EXPERIMENTAL APPROACHES NEED TO BE AUGMENTED FOR EVALUATING SOCIAL PROGRAMS AND THAT AN INTERDISCIPLINARY TEAM APPROACH SHOULD BE USED IN EVALUATION. REFERENCE NOTES, A REFERENCE LIST, AND AN APPENDIX PRESENTING THE ALGEBRAIC FORMULATION OF THE MEASUREMENT THEORY, ARE INCLUDED. (CFW)
Index Term(s): Evaluation; Evaluation techniques; Evaluative research; Program evaluation; Research methods; Testing and measurement; Theory
Note: REPRINTED FROM PROFESSIONAL PSYCHOLOGY, (NOVEMBER 1977), P 411-434
To cite this abstract, use the following link:
http://www.ncjrs.gov/App/publications/abstract.aspx?ID=65334

*A link to the full-text document is provided whenever possible. For documents not available online, a link to the publisher's website is provided. Tell us how you use the NCJRS Library and Abstracts Database - send us your feedback.