U.S. flag

An official website of the United States government, Department of Justice.

NCJRS Virtual Library

The Virtual Library houses over 235,000 criminal justice resources, including all known OJP works.
Click here to search the NCJRS Virtual Library

POST-EVALUATION DETERMINATION OF A PROGRAM'S GENERALIZABILITY

NCJ Number
67442
Journal
Evaluation and Program Planning Volume: 2 Issue: 4 Dated: (1979) Pages: 297-302
Author(s)
N L SMITH; D N CAULEY
Date Published
1979
Length
6 pages
Annotation
THIS PAPER SUGGESTS SECONDARY ANALYSIS PROCEDURES WHICH CAN BE EMPLOYED USING EXISTING EVALUATION DATA TO ESTIMATE A PROGRAM'S GENERALIZABILITY WHEN FOLLOWUP FIELD STUDIES ARE NOT FEASIBLE.
Abstract
POSTEVALUATION DETERMINATIONS OF A PROGRAM'S GENERALIZABILITY ARE POSSIBLE, WHETHER THE EVALUATION WAS PREPLANNED OR POSTPLANNED. REGARDLESS OF THE ADEQUACY OF THE EVALUATION PLANNING AND DESIGN, UNEXPECTED QUESTIONS OF PROGRAM GENERALIZATION OFTEN ARISE DUE TO FACTORS SUCH AS THE CHANGES IN THE NATURE OF THE PROGRAM ACTIVITIES FROM PLANNING TO COMPLETION, POLITICAL PRESSURE TO EXPAND A SUCCESSFUL PROGRAM, UNANTICIPATED EFFECTS MAKING THE PROGRAM BENEFICIAL TO NEW CLIENT GROUPS, REDEFINITION PROGRAM THRUST, AND THE AVAILABILITY OF ADDITIONAL FUNDS FOR PROGRAM EXPANSION INTO NEW AREAS. EXTERNAL VALIDITY MUST BE TESTED. EXTERNAL VALIDITY CAN BE CLASSIFIED ACCORDING TO GENERAL TYPES--POPULATION VALIDITY WHICH REFERS TO GENERALIZATIONS ACROSS GROUPS OF PEOPLE AND ECOLOGICAL INITIAL EVALUATION DESIGNS, THEREBY MAKING POST HOC ANALYSES EASIER AND MORE USEFUL. A SAMPLING PLAN USED IN THE EVALUATION STUDY CAN DETERMINE THE SCOPE OF PERMISSIBLE GENERALIZATION. THE MOST IMPORTANT POPULATION VARIABLES MUST BE IDENTIFIED IN DOING A POST HOC CONSIDERATION A PROGRAM GENERALIZABILITY. ONE'S ABILITY TO MAKE A GENERALIZATION IS THUS DEPENDENT ON THE CONTEXT (AS DEFINED BY THE SIGNIFICANT POPULATION AND ECOLOGICAL VARIABLES) IN WHICH IT IS TO BE APPLIED. FOR QUESTIONS OF ECOLOGICAL VALIDITY, IT IS INCREASINGLY RECOGNIZED THAT SUCCESSFUL INNOVATIONS ARE CHARACTERIZED BY LOCAL ADAPTATION OF THE INNOVATIVE PROGRAM. EACH PROGRAM IS CUSTOMIZED TO MEET DIFFERENT NEEDS AND SETTINGS. POSTEVALUATION PROCEDURES PROVIDE POTENTIALLY USEFUL TECHNIQUES FOR EVALUATORS TRYING TO RESPOND TO LAST MINUTE QUESTIONS OF PROGRAM GENERALIZABILITY. REFERENCES ARE CITED. (MJW)