Introduction

The Office of Juvenile Justice and Delinquency Prevention (OJJDP) is dedicated to preventing and reversing trends of increased delinquency and violence among adolescents. These trends have alarmed the public during the past decade and challenged the juvenile justice system. It is widely accepted that increases in delinquency and violence over the past decade are rooted in a number of interrelated social problems—child abuse and neglect, alcohol and drug abuse, youth conflict and aggression, and early sexual involvement—that may originate within the family structure. The focus of OJJDP's Family Strengthening Series is to provide assistance to ongoing efforts across the country to strengthen the family unit by discussing the effectiveness of family intervention programs and providing resources to families and communities.

Research points out that aggression in children is escalating—and at younger ages (Campbell, 1990; Webster-Stratton, 1991). These studies also indicate that anywhere from 7 to 25 percent of preschool and early school-age children meet the diagnostic criteria for oppositional defiant disorder (ODD) and/or conduct disorder (CD), referred to generically in this paper as "conduct problems" (defined as high rates of aggression, noncompliance, and defiance). These trends have disturbing implications because the early onset of these problems in young children, in the form of high rates of oppositional defiant aggression and noncompliance, is predictive of substance abuse, depression, juvenile delinquency, antisocial behavior, and violence in adolescence and adulthood (Loeber, 1985). People with histories of chronic aggression beginning in childhood are more likely than others to commit murder, rape, robbery, arson, and driving under the influence (DUI) offenses and to engage in substance abuse (Kazdin, 1995). The problem of escalating aggression in young children is thus a universal concern.

The Incredible Years Parents, Teachers, and Children Training Series, developed by Dr. Carolyn Webster-Stratton, uses group discussion, videotape modeling, and rehearsal intervention techniques to assist adults living and working with children ages 2 to 10. The series is designed to prevent, reduce, and treat conduct problems among these children and to increase their social competence. This Bulletin presents an overview of the series' program designs, goals, and target populations; an extended discussion of each program's rationale, content, process, and methods; the findings of several research evaluations of the programs; and a summary of the series' effectiveness. The Incredible Years Training Series was selected as the 1997 winner of the United States Leila Rowland National Mental Health Award for outstanding prevention programs and has been identified as an exemplary "best practices" program by OJJDP's Family Strengthening Project.

OJJDP's Family Strengthening Project's Evaluation Criteria

The Incredible Years Parents, Teachers, and Children Training Series was identified as an exemplary "best practices" program by OJJDP's Family Strengthening Project. A panel of national experts selected the OJJDP-identified "best practices" programs based primarily on the programs' focus on family, strength of theoretical foundation, program content, capacity for dissemination (e.g., availability of curriculum manuals, etc.), and the training capacity of the program developer. Critical to selection was the quality of the program evaluation, research design, and outcome results in relation to family and youth change. There were three categories of programs: exemplary, model, and promising approaches. The exemplary category indicates the program has an evaluation of the highest quality (experimental with random assignment, quasi-experimental design with matched controls, etc.), excellent effectiveness results, and overall high ratings by the national review committee for its category. Reviewers independently rated the 14 dimensions listed and then were required to come to consensus regarding the quality of the program in question. Programs rated as "exemplary" were those that were well implemented, were rigorously evaluated, and had consistently positive findings (integrity ratings of "A4" or "A5" on the assessment scale):

  • Theory. The degree to which the project findings are based in clear and well-articulated theory, clearly stated hypotheses, and clear operational relevance.

  • Fidelity of interventions. The degree to which there is clear evidence of high fidelity implementation, which may include dosage data.

  • Sampling strategy and implementation. The quality of sampling design and implementation.

  • Attrition. Evidence of sample quality based on information about attrition.

  • Measures. The operational relevance and psychometric quality of measures used in the evaluation and the quality of supporting evidence.

  • Missing data. The quality of implementation of data collection (e.g., amount of missing data).

  • Data collection. The method(s) by which data were collected (in terms of bias or demand characteristics and haphazard manner).

  • Analysis. The appropriateness and technical adequacy of techniques of analysis, primarily statistical.

  • Other plausible threats to validity (excluding attrition). The degree to which the evaluation design and implementation address and eliminate plausible alternative hypotheses concerning program effects. The degree to which the study design and implementation warrant strong causal attributions concerning program effects.

  • Replications. The exact or conceptual reproduction of both the intervention implementation and evaluation.

  • Dissemination capability. Program materials developed including training in program implementation, technical assistance, standardized curriculum and evaluation materials, manuals, fidelity instrumentation, videos, recruitment forms, etc.

  • Cultural and age appropriateness. The degree to which the project is culturally and age appropriate.

  • Integrity. The overall level of confidence that the reviewer can place in project findings based on research design and implementation.

  • Utility. The overall usefulness of project findings for informing prevention theory and practice.



Previous Contents Next

Line
The Incredible Years Training Series Juvenile Justice Bulletin June 2000