U.S. flag

An official website of the United States government, Department of Justice.

NCJRS Virtual Library

The Virtual Library houses over 235,000 criminal justice resources, including all known OJP works.
Click here to search the NCJRS Virtual Library

POWER OF SOCIAL INTERVENTION EXPERIMENTS TO DISCRIMINATE DIFFERENCES BETWEEN EXPERIMENTAL AND CONTROL GROUPS (FROM EVALUATION STUDIES - REVIEW ANNUAL, V 3, 1978 BY THOMAS D COOK ET AL - SEE NCJ-51598)

NCJ Number
51604
Author(s)
J A CRANE
Date Published
1978
Length
19 pages
Annotation
THE DISCRIMINATING CAPACITY OF EVALUATIVE RESEARCH ON CASE SERVICES IN SOCIAL WORK IS EVALUATED, AND THE USE OF STATISTICAL POWER ANALYSIS IN SOCIAL EXPERIMENTATION IS DESCRIBED.
Abstract
THE DISCRIMINATING POWER OF A SOCIAL EXPERIMENT IS DEFINED AS THE PROBABILITY THAT IT WILL DETECT A SPECIFIED DIFFERENCE IN THE POPULATION SAMPLED. BECAUSE THE POWER OF AN EXPERIMENT VARIES MARKEDLY WITH THE MAGNITUDE OF DIFFERENCES TO BE DETECTED, POWER ANALYSIS IS A CRITICALLY IMPORTANT ALTHOUGH NEGLECTED TOPIC IN RESEARCH ON THE EFFECTS OF EXPERIMENTAL SOCIAL INTERVENTION. IF AN EXPERIMENT AIMS TO IDENTIFY DIFFERENCES IN THE PROPORTION OF SUCCESS AMONG DIFFERENT EXPERIMENTAL GROUPS, AN UPPER LIMIT TO ITS POWER TO DETECT ANY SPECIFIED DIFFERENCE MAY READILY BE CALCULATED FROM KNOWLEDGE OF THE SAMPLE SIZE, ESTIMATED VARIATION IN STUDY POPULATIONS OF THE CHARACTERISTIC BEING EXAMINED, AND THE LEVEL OF STATISTICAL SIGNIFICANCE CHOSEN BY THE EXPERIMENTER. THE EXPERIMENT REPORTED IN THE ARTICLE WAS CONCERNED PRIMARILY WITH ASSESSING THE UPPER LIMITS OF POWER AS DETERMINED BY VARIABLES OTHER THAN MEASUREMENT ERROR. OUTCOME CRITERIA WERE CLASSIFIED AS ECONOMIC FUNCTIONING OF THE FAMILY, ADEQUACY OF PARENTING BEHAVIOR, CHILDREN'S PROBLEMS, AND PROBLEMS OR PATTERNS OF ADAPTIVE BEHAVIOR IN FAMILY RELATIONSHIPS. THE METHOD OF POWER ANALYSIS WAS BASED ON THE DEPENDENCE OF THE POWER OF THE EXPERIMENT ON SAMPLE SIZE, MAGNITUDE OF DIFFERENCES TO BE DETECTED, AND LEVEL OF STATISTICAL SIGNIFICANCE CHOSEN BY THE EXPERIMENTER. ANALYSIS OF 25 EXPERIMENTAL TESTS OF THE EFFECT OF CASE SERVICES IN SOCIAL WORK SHOWS THAT 18 HAVE A 0.95 PROBABILITY OF DETECTING DIFFERENCES ON THE ORDER OF 25 PERCENT OR GREATER. IN MOST CASES, THIS MUST BE INCREASED BY AT LEAST ONE-HALF AS A RESULT OF MEASUREMENT ERROR. DATA FROM THE EXPERIMENT ARE TABULATED; THEY SHOW THAT RESEARCH EVIDENCE ON THE EFFECTIVENESS OF CASE SERVICES IN SOCIAL WORK IS MUCH LESS CLEARCUT THAN HAS OFTEN BEEN ASSUMED. THERE IS A NEED FOR CAUTION IN GENERALIZING ABOUT POLICY IMPLICATIONS IN AREAS SUCH AS THE SELECTION OF ONE OR ANOTHER MODE OF INTERVENTION. A TECHNICAL NOTE ON CALCULATION METHODS USED IN THE EXPERIMENT IS CONTAINED IN AN APPENDIX, AND NOTES ARE INCLUDED. (DEP)