TITLE: National Evaluation of G.R.E.A.T. AUTHOR: Finn-Aage Esbensen and D. Wayne Osgood SUBJECT: gangs, juvenile delinquency prevention, conflict resolution SERIES: Research in Brief DATE: November 1997 PAGES: 15 BYTES: 31,000 Figures, charts, forms, and tables are not included in this ASCII plain-text file. To view this document in its entirety, download the Adobe Acrobat graphic file available from this Web site or order a print copy from NCJRS at 800-851-3420. ------------------------------------------------------- National Institute of Justice Research in Brief Jeremy Travis, Director National Evaluation of G.R.E.A.T. by Finn-Aage Esbensen and D. Wayne Osgood ------------------------------------------------------- ISSUES AND FINDINGS Discussed in this Brief: Early findings and research methods pertaining to a national evaluation of the effectiveness of the Gang Resistance Education and Training (G.R.E.A.T.) program, a school-based gang prevention strategy taught to middle school students. Key issues: During the 1980s and 1990s, gang affiliation by youths and their involvement in criminal activity became a major concern for law enforcement and the public. The G.R.E.A.T. program was developed to reduce adolescent involvement in criminal behavior and gangs. The national evaluation of the program consists of a two-pronged research approach: (1) a preliminary study comparing students who completed G.R.E.A.T. with others who either had not participated or had enrolled but failed to finish, and (2) a longitudinal quasi-experimental design assessing both the short- and long-term effectiveness of G.R.E.A.T. Key findings: Early findings from the national evaluation are based on a cross-sectional survey of 5,935 eighth graders from 42 schools in 11 locales where G.R.E.A.T. is taught. Researchers are also assessing the training of police officers who teach the program. Preliminary results indicate that students who completed the G.R.E.A.T. lessons reported more prosocial behaviors and attitudes than their peers who did not finish the program or failed to participate in the first place. Among other findings: o Lower rates of self-reported delinquency and gang membership. o More communication and attachment with parents. o Greater commitment to school and lower levels of perceived obstacles to academic achievement. The questionnaire administered to the eighth graders used five background characteristics--sex, race, age, family status, and parental education--to determine whether significant differences existed between students who completed the G.R.E.A.T. program and students who comprised the comparison group. Differences between the groups were small, and initial findings of the program's positive impact are not a product of preexisting differences between the G.R.E.A.T. participants and comparison students. Target audience: Gang, delinquency prevention, and juvenile justice specialists and researchers; law enforcement agencies; school administrators; and State and local policymakers. ------------------------------------------------------- Youth delinquent gangs continue to generate concern among criminal justice professionals and the general public. Gang membership and related criminal activity increased in the late 1980s and early 1990s, and the availability of firearms has led to more gang-related homicides. One way to address these problems is to find ways to prevent youths from joining gangs. In 1991 police officers from the Phoenix Police Department and from Mesa, Glendale, and Tempe, Arizona, and special agents of the Bureau of Alcohol, Tobacco and Firearms developed Gang Resistance Education and Training (G.R.E.A.T.) to reduce adolescent involvement in criminal behavior and gangs. G.R.E.A.T. is a national, school-based gang prevention program in which uniformed law enforcement officers teach a 9-week curriculum to middle school students. As of June 1997, more than 2,400 officers from 47 States and the District of Columbia had completed G.R.E.A.T. training. Given this rapid program expansion, the National Institute of Justice, in cooperation with the Treasury Department's Bureau of Alcohol, Tobacco and Firearms, sponsored a comprehensive, multisite evaluation to assess G.R.E.A.T.'s effectiveness. Initial findings indicate the program is having a positive effect on student attitudes and behaviors and is deterring them from involvement in gangs. This Research in Brief discusses the evaluation's design and methodology, G.R.E.A.T.'s program and officer training, and preliminary findings of a cross-sectional study. EVALUATION DESIGN Context. The research design for the national evaluation considered previous research and public policy on gangs. Consensus is lacking about the magnitude of the gang problem, the extent and level of gang organization, and the action needed to address the issue. Some of the epidemiological and etiological issues can be traced to different methodologies and theoretical perspectives. Policy differences can be attributed to competing government priorities and to the limited number of evaluations of programs undertaken to address the gang phenomenon. However, a number of suppression, intervention, and prevention programs with evaluative components have been implemented in the past few years at local and national levels.[1] Knowledge about gangs traditionally has come from one of three sources: observational or case studies,[2] law enforcement records,[3] and surveys.[4] On one point there is considerable consensus among researchers: the high rate of criminal offending among gang members. Two objectives and two strategies. The national evaluation of G.R.E.A.T. has two primary objectives: (1) to perform an outcome analysis examining G.R.E.A.T.'s short- and long-term effects on students and (2) to conduct a process evaluation assessing the quality and effectiveness of officer training (see "G.R.E.A.T. Officer Training"). Two strategies were developed to determine program effectiveness. The first is a cross-sectional study of students in 11 locales where G.R.E.A.T. is taught; group questionnaires were administered to a sample of eighth-grade students. The second strategy, which recognizes the limitations of retrospective, cross-sectional designs, is a prospective longitudinal study initiated at six sites.[5] A quasi-experimental research design guided the assignment of classrooms to experimental and comparison groups. Students in both groups completed pretests and posttests during the first half of the 1995-96 school year and will be administered questionnaires annually through fall 1999. CROSS-SECTIONAL SURVEY For the first study, a cross-sectional survey of 5,935 eighth-grade students was completed in spring 1995. Survey results were used to create a treatment group and a comparison group to assess G.R.E.A.T.'s effectiveness in the 11 cities where the survey was administered. These cities had delivered the G.R.E.A.T. program during the 1993-94 school year, when the targeted students were seventh graders. Surveying these students as eighth graders permitted a 1-year followup to their program participation and also guaranteed that none of the survey sample were currently enrolled in the program. Site selection. In selecting the 11 sites, consideration was given to geographic location, population characteristics, and population size. The cities selected were Phoenix, Arizona; Torrance, California; Orlando, Florida; Pocatello, Idaho; Will County, Illinois; Kansas City, Missouri; Omaha, Nebraska; Las Cruces, New Mexico; Philadelphia, Pennsylvania; Providence, Rhode Island; and Milwaukee, Wisconsin. In those cities, questionnaires were administered to all eighth graders in attendance on the specified day at schools that had offered G.R.E.A.T. during the previous 2 years. This resulted in a final sample of 5,935 eighth-grade students from 315 classrooms in 42 schools. Measures. The questionnaire was designed to assess the G.R.E.A.T. curriculum. The goal was to include questions that would assess specific aspects of the G.R.E.A.T. program while also measuring dominant criminological theories.[6] Several measures also were developed to reflect the curriculum's cognitive aspects. For example, lesson 3 of the program introduces students to six steps and five personal prerequisites for conflict resolution. A sample measure for this lesson was to ask students to respond to the following statement: "Violence interferes with a person's basic right to feel safe and secure." (See "The G.R.E.A.T. Curriculum.") Another key measure concerns gang membership and involvement in gang activity. Questions were designed to elicit self-reports of illegal activity. This technique has been used widely during the past 30 years and has provided a good measure of actual behavior rather than a measure of police response to behavior.[7] (See "Measuring Gang Affiliation.") Comparison group. A primary question was whether students who completed the G.R.E.A.T. program were comparable to those who did not complete it--either because they never participated or dropped out of the program. The treatment group and comparison group were defined through answers to the question, "Did you complete the G.R.E.A.T. program?" Of the 5,836 respondents who answered the question (99 students did not respond), 2,629 (45 percent) reported they had completed the program and thus were the treatment group. The 3,207 who had not became the comparison group. The schools varied substantially, however, in the number of students who had completed and who had not completed the G.R.E.A.T. program. Since the precision with which program impact can be established at each school depends on the number of students in both treatment and comparison groups, schools with few students in one of the groups could contribute relatively little to the evaluation. Therefore, analysis of the treatment and comparison groups was replicated in a restricted sample of 28 schools where at least 15 students comprised each group. Controlling for other differences. Because data were gathered on one occasion only, a year after students had completed the program, the researchers had to compare the treatment and comparison groups using statistical controls to rule out the possibility that differences between them were attributable to various background characteristics. BACKGROUND CHARACTERISTICS Questions were asked in the survey to determine five background characteristics that could be associated with the outcome measures.[8] The analysis controlled for the following: o Sex. o Race/ethnicity (white, African-American, Hispanic, Asian- American, and other). o Age (there was little variation in age, because only eighth- grade students participated in the evaluation). o Family status (as reflected in the adults with whom the youths resided). o Parental education (defined as the highest level attained by either parent). Not surprisingly, differences surfaced among the 42 schools in racial composition and socioeconomic status (as reflected by family status and parental education).[9] The analysis, which controlled for differences between schools, found a few small but statistically significant differences in background characteristics between treatment and comparison groups. Ideally, the treatment and comparison groups would have been matched, but this could not be expected in a post hoc evaluation such as this study. The pattern of group differences in background characteristics is ambiguous but does not appear especially problematic to determining the impact of the G.R.E.A.T. program. Comparisons of treatment and nontreatment groups revealed no systematic bias. Both groups had demographic characteristics indicating high or low risk for delinquency, gang membership, or both. In the comparison group, 15-year-old students were overrepresented; in the treatment group African-American youths were overrepresented. The comparison group had fewer females but more youths from single-parent homes. Given this inconsistent pattern and the small size of group differences, it was concluded that the outcome measures were not a product of preexisting differences between the G.R.E.A.T. and comparison students.[10] Program impact was thus determined through a model that controlled for school and the five background characteristics. Although the results are consistent, restricting the analysis to the 28 schools tends to strengthen the magnitude of the program's effect.[11] INITIAL RESULTS Early findings indicate that G.R.E.A.T. appears to be meeting its objective--to reduce gang affiliation and delinquent activity. The students completing G.R.E.A.T. reported lower levels of gang affiliation and delinquency than did comparison students. These differences are small but statistically significant. (See "Statistical and Substantive Differences.") Not only is the aggregate measure of delinquency lower for the G.R.E.A.T. group but so are most of the subscales, i.e., drug use, minor offending, property crimes, and crimes against persons. No differences between the groups were found for rates of victimization or selling drugs. A number of differences also were found for attitudinal measures. G.R.E.A.T. lessons are aimed at reducing impulsive behavior, improving communication with parents and other adults, enhancing self-esteem, and encouraging students to make better choices. The cross-sectional survey results (see exhibit 1) reveal that 1 year after completing the program, G.R.E.A.T. students (in contrast to the comparison group) reported the following: o Lower rates of delinquency. o Lower rates of gang affiliation. o More positive attitudes toward the police. o More negative attitudes about gangs. o More friends involved in prosocial activities. o Greater commitment to peers promoting prosocial behavior. o Higher levels of perceived guilt at committing deviant acts. o More commitment to school. o Higher levels of attachment to both mothers and fathers. o More communication with parents about their activities. o Fewer friends involved in delinquent activity. o Less likelihood of acting impulsively. o Lower likelihood of engaging in risky behavior. o Lower levels of perceived blocks to academic success. The cross-sectional survey also yielded findings about gang membership that are contrary to popular perceptions and other research results. For example, white youths comprised a larger share of the gang population (25 percent), in contrast to previous studies that found that gangs were predominantly composed of minorities. (See "Differences Between Gang Members and Nonmembers" and exhibit 2.) CONCLUSIONS AND POLICY IMPLICATIONS G.R.E.A.T. is one of myriad gang prevention efforts employed to reduce adolescent involvement in crime and gangs. The preliminary findings of this study support continuation of G.R.E.A.T.; other prevention programs await evaluation results. Results from the 1995 cross-sectional survey suggest that students who participated in G.R.E.A.T. reported significantly more prosocial behaviors and attitudes than students who did not take part in the program. This 1-year followup survey supports the idea that trained law enforcement personnel can serve as prevention agents as well as enforcers of the law. These cross-sectional results need to be viewed with caution, however. Some differences existed between treatment and comparison groups prior to the introduction of the program. Although most of these differences were controlled through available statistical techniques, a quasi-experimental design such as that being implemented in the longitudinal phase of this evaluation will provide a better assessment of program effectiveness. This longitudinal design also will allow for examination of long-term effects. ------------------------------------------------------- G.R.E.A.T. OFFICER TRAINING Currently, the Bureau of Alcohol, Tobacco and Firearms, the Federal Law Enforcement Training Center, and the Phoenix Police Department coordinate officer training and, with the Orange County, Florida, Sheriff's Office, share management responsibility for the G.R.E.A.T. program. Evaluators attended five officer training sessions as well as one G.R.E.A.T. management training session during fiscal year 1995. Despite some shortcomings, G.R.E.A.T. officer training has many strengths that prepare officers to become successful classroom instructors. Primary among them is the supportive learning environment the training staff creates for the officers. Instructors deal with officer students in an enthusiastic, engaging, and encouraging manner. The instructional format provides a spirit of camaraderie and cooperation, and a repeated emphasis on professionalism creates a context of mutual respect. Further, the strategy of modeling each lesson of the curriculum and requiring officers to present a lesson is the keystone to the training process, which repeatedly exposes officer students to material they themselves will soon be teaching in their own classrooms. Other strengths of the training program include its use of occasional role-play techniques and group exercises. In addition, the curriculum focuses on several important skills--including meeting basic needs, resolving conflict, taking responsibility, and setting goals--that can be taught to middle school students and may be instrumental in achieving the goal of crime-free adolescents. Its graduation event serves as a motivator and culminating activity. ------------------------------------------------------- THE G.R.E.A.T. CURRICULUM To meet its objectives of reducing gang activity and teaching the consequences of gang involvement, the curriculum consists of nine lesson plans to be offered once a week to middle school students, primarily seventh graders. Each detailed lesson plan contains clearly stated purposes and objectives. In addition to the nine lesson plans, the curriculum calls for the teaching officers to discuss gangs and how they affect the quality of people's lives. The nine G.R.E.A.T. lessons are: 1. Introduction. Students get acquainted with the G.R.E.A.T. program and the presenting officer. 2. Crime, Victims, and Your Rights. Students learn about crimes, their victims, and their impact on school and neighborhood. 3. Cultural Sensitivity and Prejudice. Students explore how cultural differences affect their school and neighborhood. 4.,5. Conflict Resolution (two lessons). Students are taught how to create an atmosphere of understanding that enables all parties to better address problems and work on solutions together. 6. Meeting Basic Needs. Students learn how to meet their basic needs without joining a gang. 7. Drugs and Neighborhoods. Students are educated about how drugs affect their school and neighborhood. 8. Responsibility. Students examine the diverse responsibilities of people in their school and neighborhood. 9. Goal Setting. Students learn the need for goal setting and how to establish short- and long-term goals. For information about G.R.E.A.T., contact the Bureau of Alcohol, Tobacco and Firearms at 800-726-7070. ------------------------------------------------------- MEASURING GANG AFFILIATION What constitutes a gang member? In the current research, two filter questions introduce the gang-specific section of the questionnaire: "Have you ever been a gang member?" and "Are you now in a gang?" Of the total sample, 994 youths, or 17 percent, indicated they had belonged to a gang at some point in their lives. In contrast, slightly more than half of these youths--522, or 9 percent of the sample--indicated they were currently gang members. In an attempt to limit the sample of gang members to "delinquent gangs," two different measures were employed. First, a restrictive or conservative definition limited gang status to those respondents who stated they were current gang members and that their gangs engaged in at least one type of delinquent behavior (fighting other gangs, stealing cars, stealing in general, or robbing people). This resulted in identification of 451 gang members, or 8 percent of the sample. Second, a more liberal, yet still somewhat restrictive, definition included youths who indicated they "had ever been a gang member" and whose gang had been involved in at least one of the four illegal activities. This more liberal definition produced 623 gang members, representing 10.6 percent of the sample. The latter, more liberal, definition was used for this research. ------------------------------------------------------- STATISTICAL AND SUBSTANTIVE DIFFERENCES An important distinction exists between statistically significant differences and substantively important differences. Sample sizes and statistical approaches can affect the level of statistical significance, sometimes exaggerating an effect and other times underestimating an effect. One alternative to relying solely on statistical significance is to examine relative effect sizes. Effect size (ES) can be defined as "a measure of change due to the treatment as a proportion of the standard deviation for each measure employed."* Thus, an ES of -1 indicates that the treatment group performed one standard deviation lower than the comparison group; an ES of +1 indicates that the treatment group performed one standard deviation unit higher than the comparison group. The larger the ES, the greater the measurable impact of the program. In one review of delinquency treatment and prevention programs, the author found average effect sizes of .17 and argued that even a small ES of .10 may have practical value when dealing with criminal activity.** One way of interpreting an effect size is to convert it to a percentage. This can be done by dividing the effect size by two. For example, an effect size of .10 represents a 5 percent difference. In the current research, effect sizes were in the .10 range, indicating modest program effects. *Gottfredson, Denise C., "School-Based Crime Prevention," in Larry W. Sherman, et al., Preventing Crime: What Works, What Doesn't, What's Promising: A Report to the United States Congress, Washington, D.C.: U.S. Department of Justice, Office of Justice Programs, 1997. **Lipsey, Mark W., "Juvenile Delinquency Treatment: A Meta- Analytic Inquiry into the Variability of Effects," in Meta- Analysis for Explanation, ed. T.D. Cook, et al., Beverly Hills, California: Sage, 1992. ------------------------------------------------------- DIFFERENCES BETWEEN GANG MEMBERS AND NONMEMBERS Contrary to much of the prevailing literature about the male- dominated nature of gangs, 38 percent of gang members in the sample were females. Although this figure still indicates that females are underrepresented among gang members, it is to a far lesser extent than is commonly assumed.* The racial composition of gang members in this sample reveals that white youths were proportionately less involved in gangs than African-American and Hispanic youths, but not to the extent that prior research (often based on case studies of minority populations) has suggested: 25 percent of the gang members in this study are white. In fact, if some of the "other" category--which comprises white youths who identified themselves as American, Italian, German, Portuguese, and the like--is included, the proportionate difference is reduced even further. Consistent with earlier assessments of the demographic characteristics of gangs, this sample reveals that younger youths are underrepresented in gangs, and gang members are more likely to live with a single parent and have parents with lower levels of educational attainment. Even within this limited age sample, the youths who were 13 and younger accounted for only 17 percent of gang members, although they represented 31 percent of the nongang sample. At the other extreme, 23 percent of gang members were 15 years old or older, although only 9 percent of nongang members were in this age bracket. A minority of youths lived in single parent homes, but gang members reported living in single parent homes more frequently (40 percent) than nongang youths (30 percent). Gang members' mothers, fathers, or both were more likely not to have finished high school (20 percent for gang members, 11 percent for nongang youths). These demographic characteristics suggest there may be qualitative differences in the living situations between gang and nongang youths. *This discrepancy in rates of female participation in gangs may be due to a combination of methodological issues. First, relatively few studies have sampled youths as young as 12 and 13. Second, few studies have used general surveys of adolescent populations. ------------------------------------------------------- Finn-Aage Esbensen is a professor in the Department of Criminal Justice, University of Nebraska at Omaha; D. Wayne Osgood is a professor in the Department of Sociology, Pennsylvania State University. ------------------------------------------------------- NOTES 1. For a review see Howell, James C., Youth Gang Violence Prevention and Intervention: What Works, Washington, D.C.: U.S. Department of Justice, 1996. 2. See, for example, the works of Campbell, Anne, The Girls in the Gang, 2d ed., Cambridge, Massachusetts: Basil Blackwell, 1991; Hagedorn, John M., People and Folks: Gangs, Crime and the Underclass in a Rustbelt City, Chicago, Illinois: Lakeview Press, 1988; Miller, Walter B., "Lower Class Culture as a Generating Milieu for Gang Delinquency," Journal of Social Issues, 14(1958):5-19; Thrasher, Frederick M., The Gang: A Study of One Thousand Three Hundred Thirteen Gangs in Chicago, Chicago: University of Chicago Press, 1927; Vigil, James D., Barrio Gangs: Street Life and Identity in Southern California, Austin, Texas: University of Texas Press, 1988. 3. Maxson, Cheryl L., and Malcolm Klein, "Street Gang Violence: Twice as Great or Half as Great?" in Gangs in America, ed. C. Ronald Huff, Newbury Park, California: Sage Publications, Inc., 1990; Spergel, Irving A., "Youth Gangs: Continuity and Change," in Crime and Justice: An Annual Review of Research, ed. Norval Morris and Michael Tonry, Chicago: University of Chicago Press, 1990. 4. Esbensen, Finn-Aage, and David Huizinga, "Gangs, Drugs, and Delinquency in a Survey of Urban Youth," Criminology, 31(1993):565-589; Esbensen, Finn-Aage, David Huizinga, and Anne W. Weiher, "Gang and Non-gang Youth: Differences in Explanatory Variables," Journal of Contemporary Criminal Justice, 9(1993):94-116; Fagan, Jeffrey, "The Social Organization of Drug Use and Drug Dealing Among Urban Gangs," Criminology, 27(1989):633-669; Thornberry, Terence P., Marvin D. Krohn, Alan J. Lizotte, and Deborah Chard- Wierschem, "The Role of Juvenile Gangs in Facilitating Delinquent Behavior," Journal of Research in Crime and Delinquency, 30(1993):55-87; Winfree, L. Thomas, Teresa V. Backstrom, and G. Larry Mays, "Social Learning Theory, Self- Reported Delinquency, and Youth Gangs: A New Twist on a General Theory of Crime and Delinquency," Youth and Society, 26(1994):147-177. 5. The six sites are Las Cruces, New Mexico; Lincoln, Nebraska; Omaha, Nebraska; Philadelphia, Pennsylvania; Phoenix, Arizona; and Portland, Oregon. 6. Winfree, L. Thomas, Finn-Aage Esbensen, and D. Wayne Osgood, "Evaluating a School-Based Gang Prevention Program: A Theoretical Perspective," Evaluation Review, 20(1996):181-203. 7. Hindelang, Michael J., Travis Hirschi, and Joseph G. Weis, Measuring Delinquency, Beverly Hills, California: Sage, 1981; Huizinga, David, and Delbert S. Elliott, "Reassessing the Reliability and Validity of Self-Report Delinquency Measures," Journal of Quantitative Criminology, 2(1986):293-327; Huizinga, David, "Assessing Violent Behavior with Self-Reports," in Neuropsychology of Aggression, ed. Joel Milner, New York: Kluwer Academic Publishers, 1991. 8. For a discussion of the correlates of delinquency and gang membership, consult Covey, Herbert C., Scott Menard, and Robert J. Franzese, Juvenile Gangs, Springfield, Illinois: Charles Thomas Publisher, 1992; and Esbensen, Finn-Aage, David Huizinga, and Anne W. Weiher, "Gang and Nongang Youth: Differences in Explanatory Variables," Journal of Contemporary Criminal Justice, 1993. 9. "Dummy" variables were created to control for the mean differences between schools so that the assessment of program impact is an average of the within-school differences between G.R.E.A.T. students and comparison group (weighted according to sample size), adjusted for any differences in the five background characteristics. 10. An additional analysis examined only students from schools at which no preexisting differences were found between the treatment and comparison groups. While some of the findings differ from those reported here, the same pattern of program effect was found. These findings are reported in Esbensen, Finn- Aage, and D. Wayne Osgood, "Promising Results from a Gang Prevention Program," Annual Meeting of the Academy of Criminal Justice Sciences, Louisville, Kentucky, 1997. 11. A copy of the technical report of this study is available from Finn Esbensen, Department of Criminal Justice, University of Nebraska at Omaha, Lincoln, Nebraska 68588. ------------------------------------------------------- This research is supported under grant number 94-IJ-CX-0058 from the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice. Points of view in this document are those of the authors and do not necessarily represent the official position of the U.S. Department of Justice. ------------------------------------------------------- The National Institute of Justice is a component of the Office of Justice Programs, which also includes the Bureau of Justice Assistance, the Bureau of Justice Statistics, the Office of Juvenile Justice and Delinquency Prevention, and the Office for Victims of Crime. ------------------------------------------------------- NCJ 167264 -------------------------------------------------------