U.S. flag

An official website of the United States government, Department of Justice.

NCJRS Virtual Library

The Virtual Library houses over 235,000 criminal justice resources, including all known OJP works.
Click here to search the NCJRS Virtual Library

CORRECTING COVARIATES FOR UNRELIABILITY - DOES IT LEAD TO DIFFERENCES IN EVALUATOR'S CONCLUSIONS?

NCJ Number
50181
Journal
EVALUATION QUARTERLY Volume: 2 Issue: 3 Dated: (AUGUST 1978) Pages: 401-420
Author(s)
R G ST PIERRE
Date Published
1978
Length
20 pages
Annotation
THE EFFECTS OF DIFFERENT ASSUMPTIONS ABOUT TEST RELIABILITY ON THE FINDINGS AND CONCLUSIONS OF PROGRAM EVALUATIONS ARE EXPLORED IN AN ANALYSIS OF DATA FROM AN EXPERIMENTAL EDUCATIONAL PROGRAM.
Abstract
EVALUATIONS OF EDUCATIONAL PROGRAMS TYPICALLY ARE QUASI-EXPERIMENTAL. USUALLY SOME FORM OF ANALYSIS OF CONVARIANCE IS EMPLOYED TO EQUATE TREATMENT AND COMPARISON GROUPS ON PRETREATMENT CONDITIONS. THE APPLICATION OF ANALYSIS OF COVARIANCE TO QUASI-EXPERIMENTAL DATA HAS BEEN CRITICIZED ON THE GROUNDS THAT THE PROCEDURE LEADS TO BIAS IN THE ADJUSTMENT OF OUTCOME SCORES. VARIOUS PROCEDURES, MOST DEALING WITH THE CORRECTION OF COVARIATES FOR UNRELIABILITY, HAVE BEEN DESIGNED TO YIELD UNBIASED ADJUSTMENTS. ONE SUCH PROCEDURE WAS APPLIED TO DATA COLLECTED IN THE NATIONAL EVALUATION OF PROJECT FOLLOW THROUGH, A LARGE-SCALE QUASI-EXPERIMENTAL EDUCATIONAL PROGRAM FOR DISADVANTAGED CHILDREN. PROJECT EVALUATORS SOUGHT TO DETERMINE WHETHER THE APPLICATION OF RELIABILITY CORRECTIONS ALTERS RESULTS OBTAINED WITH UNCORRECTED COVARIATES SO AS TO CHANGE APPRECIABLY THE CONCLUSIONS DRAWN FROM THE RESULTS. THE RELIABILITY CORRECTIONS LED TO SOME CHANGES IN CONCLUSIONS ABOUT THE PERFORMANCE OF VARIOUS EDUCATIONAL MODELS FUNDED UNDER PROJECT FOLLOW THROUGH. THE RESULTS OF BOTH ANALYSES WERE PRESENTED IN THE EVALUATORS' REPORT, BUT CONCLUSIONS WERE BASED ON THE ANALYSIS EMPLOYING THE UNCORRECTED DATA DUE TO CONTROVERSY SURROUNDING METHODS OF CORRECTING FOR UNRELIABILITY IN COVARIATES. DETAILS OF THE ANALYSIS, SUPPORTING DATA, AND A LIST OF REFERENCES ARE INCLUDED. (AUTHOR ABSTRACT MODIFIED--LKM)