U.S. flag

An official website of the United States government, Department of Justice.

NCJRS Virtual Library

The Virtual Library houses over 235,000 criminal justice resources, including all known OJP works.
Click here to search the NCJRS Virtual Library

Masked Intervention Effects: Analytic Methods for Addressing Low Dosage of Intervention

NCJ Number
215608
Journal
New Directions for Evaluation Issue: 110 Dated: Summer 2006 Pages: 19-32
Author(s)
John E. Lochman; Caroline Boxmeyer; Nicole Powell; David L. Roth; Michael Windle
Date Published
2006
Length
14 pages
Annotation
This article examines how a particular strategy for analyzing evaluation data, intent-to-treat analyses, may underestimate the true effects of interventions.
Abstract
Intent-to-treat (ITT) analyses for program evaluations with randomized experimental designs provide rigorous tests of intervention effects. Once cases are randomized to conditions, all cases are included in analyses. Although this is a widely accepted analytic strategy, estimates of intervention effect sizes can be biased if any participants fail to comply with the assigned treatment condition. Using the example of the Coping Power Program--an evidence-based program for aggressive children--bias would result if attendance at parent sessions varied among parents. ITT compares average outcomes between randomized groups, but ignores compliance. An ITT analysis provides evidence of whether the intervention and control groups differ, but if significant intervention effects are not detected, it is unclear whether the intervention was ineffective or whether noncompliance with the intervention program undermined its effectiveness. Although progress is being made in developing and using new methods of addressing variables in participant attendance in intervention trials, these methods have rarely been compared and have not been applied to preventive interventions designed to reduce children's problem behaviors. This article compares propensity-score analyses and three types of complier-average-causal-effect (CACE) analyses with ITT analyses and as-treated analyses in evaluating the Coping Power Project. Both the propensity-score approach and the CACE approach addressed participants attendance and compliance. These differences in findings when compliance thresholds were set at different levels suggest that intervention researchers should work with program designers to estimate the minimally important level of participation necessary for program success. 26 references