Title: Improving the Nation's Criminal Justice System: Findings and Results From State and Local Program Evaluations Effective programs Monograph No. 1. Series: Monograph Author: Bureau of Justice Assistance Subject: Program Evaluations 56 pages 126,000 bytes Figures, charts, forms and tables are not included in this ASCII plain-text file. To view this document in its entirety, download the Adobe Acrobat graphic file available from this Web site or order a print copy from BJA at 800-688-4252. U.S. Department of Justice Office of Justice Programs Bureau of Justice Assistance Improving the Nation's Criminal Justice System: Findings and Results From State and Local Program Evaluations Effective Programs Monograph No. 1 ----------------------------- U.S. Department of Justice Office of Justice Programs 810 Seventh Street NW. Washington, DC 20531 Janet Reno Attorney General U.S. Department of Justice Raymond C. Fisher Associate Attorney General Laurie Robinson Assistant Attorney General Nancy E. Gist Director, Bureau of Justice Assistance U.S. Department of Justice Response Center 1-800-421-6770 Bureau of Justice Assistance Clearinghouse 1-800-688-4252 Bureau of Justice Assistance World Wide Web Home Page http://www.ojp.usdoj.gov/BJA ------------------------ This document was prepared by Justice Research and Statistics Association, under grant number 95-DD-BX-K011, awarded by the Bureau of Justice Assistance, Office of Justice Programs, U.S. Department of Justice. The opinions, findings, and conclusions or recommendations expressed in this document are those of the authors and do not necessarily represent the official position or policies of the U.S. Department of Justice. ------------------------ The Bureau of Justice Assistance is a component of the Office of Justice Programs, which also includes the Bureau of Justice Statistics, the National Institute of Justice, the Office of Juvenile Justice and Delinquency Prevention, and the Office for Victims of Crime. ------------------------ Improving the Nation's Criminal Justice System: Findings and Results From State and Local Program Evaluations Effective Programs Monograph No. 1 December 1997 NCJ 166821 ------------------------ Foreword Improving the Nation's Criminal Justice System is the central mission of the Bureau of Justice Assistance (BJA). Building safer, less violent communities is a major challenge being faced by all States and local communities. Real progress can be achieved only if we demonstrate and confirm "what works," so we can all profit from the impact of over 10 years of Federal funding through the Edward Byrne Memorial Law Enforcement Assistance Formula Grant Program. As we approach the year 2000, our knowledge of effective programs will continue to be critical as we solve one of the Nation's toughest problems. Underlying our ability to gain reliable knowledge and disseminate information of effective programs and activities is our commitment to evaluating and publicizing those programs and activities as we implement our plans and strategies. Evaluation done right permits program managers to find out what is and is not working, and why. BJA carries the responsibility of building the capacity of State and local governments to design and conduct their own assessments and evaluations of the effectiveness of criminal justice programs. BJA's handbooks and technical assistance seek to create a systematic, disciplined framework focusing on performance and results, both quantitative and qualitative, to be measured and evaluated. This initiative also represents an area of close participation with the National Institute of Justice to build Federal, State, and local evaluation systems over the past decade. BJA is pleased to present this monograph, Improving the Nation's Criminal Justice System: Findings and Results from State and Local Evaluation. It is the first in a series of reports to highlight and document the approaches and results of evaluations funded at State and local levels. The six programs that were the focus of the evaluations presented in the monograph depict demonstration projects affecting many components of the criminal justice system. Having been evaluated and identified as effective programs, they become models for other States and localities to replicate. This monograph has been produced to make effective State and local programs more accessible to planners and practitioners alike. Future reports in the series will continue to communicate the results of the strong Federal, State and local partnerships to enhance the role of evaluation, build excellent evaluation systems, and disclose on the impact of efforts to combat crime in America. BJA is proud of its work identifying effective drug abuse and violent crime programs. Identifying such programs will demonstrate which ones work most effectively and will communicate lessons learned at the State and local levels that can be shared nationally to ensure that the most promising approaches can have a broad impact. Nancy E. Gist Director ------------------------ Acknowledgments This monograph on effective State and local program evaluations is a product of a cooperative effort by the States and the Bureau of Justice Assistance as part of the State Evaluation Development Program, which is coordinated by the Justice Research and Statistics Association (JRSA). The State Evaluation Development Program relies on the expertise of the State Planning Agencies to ensure the success of its publications. The following State Planning Agencies contributed their knowledge and time to make this monograph a success: California Office of Criminal Justice Planning; Colorado Division of Criminal Justice; Florida Bureau of Community Assistance; Illinois Criminal Justice Information Authority; Iowa Governor's Alliance on Substance Abuse; and New York State Division of Criminal Justice Services. The Justice Research and Statistics Association prepared this document under the direction of Joan C. Weiss, Executive Director. The JRSA staff working on this publication, under the supervision of Deputy Director, Kellie J. Dressler, were Marylinda Stawasz and Nancy Michel. ------------------------ Contents State and Local Evaluations and the Effective Programs Initiative California Donovan Prison Therapeutic Community Program (Continuity of Care Project) Colorado Intensive Supervision Probation Program Florida Drug Treatment and Rehabilitation Programs for Offenders Illinois Gang Violence Reduction Project Iowa Batterer's Education Program New York Drug Treatment Alternative to Prison Program Appendix A Identifying Effective Criminal Justice Programs: Guidelines and Criteria for the Nomination of Effective Programs Appendix B Sources for Further Information ------------------------ Chapter 1 State and Local Evaluations and the Effective Programs Initiative The Bureau of Justice Assistance's (BJA's) evaluation strategy is designed to determine the effectiveness and impact of BJA's grant programs. The goal is to confirm whether performance objectives established by the States are being achieved and, if they are, what critical elements were responsible for success. Hence, the overall goal of the evaluation program is to identify programs of proven effectiveness so that they can be publicized and replicated in other jurisdictions. By building strong assessment and evaluation foundations in the 56 States and territories, BJA can account for the efforts of programs funded by its grants and add to comprehensive knowledge of what works and does not work throughout the criminal justice system. BJA's dependence on data and the results of research and evaluation underlies both program development under the discretionary program and BJA's ability to provide guidance and model programs under the formula and block grant programs. BJA relies on evaluation results to guide the formulation of policy and programs within Federal, State, and local criminal justice agencies and to ensure that policies and funded programs are based on proven results. The required State annual reports are useful in working toward this goal, but BJA intends to go further to capture both quantitative and qualitative measures of program performance. As part of a continuing effort to provide the criminal justice community with improved access to information on successful programs dealing with problems of drug abuse and/or violent crime, BJA is publishing this first volume of Improving the Nation's Criminal Justice System: Findings and Results From State and Local Program Evaluations. The evaluations, as well as the programs being evaluated, reflect the results of program development and implementation activities funded under BJA's Formula Grant Program to State and local governments and organizations. Lessons learned at the State and local levels can be shared nationally to ensure that the most promising approaches can have a broad impact. In addition to wanting to know about successful initiatives, justice system planners and managers need to understand the scope and level of effort required for innovative approaches. Readers of this report will observe that the state and local agencies are actively funding, implementing, and evaluating a broad range of programs to deal with drug abuse and violent crime. Effective Programs Initiative Enhancing State and local assessment and evaluation capabilities is a very high priority. The success of the Federal, State, and local partnership yielded many results within the States. This national initiative is designed to develop and/or enhance drug control and system improvement strategy performance monitoring, measurement, and evaluation capacities in the States and territories. BJA believes that setting standards for evaluation will have major benefits, while better informing the public about how Federal dollars were spent. Evaluation requires the institutionalization of new partnerships among funding agencies, program managers, and evaluators. No longer the sole domain of academic researchers, evaluation is making real differences in the ability of the criminal justice community to base policy and decisions on accurate and useful information. This is the result of the strong relationships formed between evaluators and practitioners. BJA has begun this new initiative to enhance State and local evaluation capacity building directly supporting evaluation activities, as well as the utilization of outcome results. The program creates a mechanism for enhancing the design, implementation, measurement, evaluation, and dissemination of information on programs in high-priority program areas. BJA has created this initiative to respond to the Attorney General's charge to "find out what works and spread the word." The objectives of the Effective Programs Initiative are to: o Enhance the ability of State and local agencies to generate and use evaluation results for strategy development, program improvement, and effective program identification. o Identify and document useful approaches for designing and conducting evaluations at State and local levels. BJA needs to identify effective State and local criminal justice programs, practices, and products as part of broader efforts at the national level to improve the criminal justice system by disseminating useful program information to policymakers and practitioners. Through this approach, which might be called "leading by example," information on successful programs will be disseminated to the field in a credible and timely fashion. The results from the 56 laboratories (States and territories) put in place under the Byrne Formula Grant Program are the products of this initiative. Program effectiveness guidelines and criteria presented in Identifying Effective Criminal Justice Programs: Guidelines and Criteria for the Nomination of Effective Programs (see Appendix A) are available for use in submitting potential programs. Once the effective programs have been approved, additional BJA monographs and bulletins will be published. Identifying and promoting sound programs is essential to developing effective strategies at Federal, State, and local levels. BJA wants to enhance the criminal justice system in general, while recognizing the many exceptional State and local advances in combating violent crime and drug abuse through the use of Federal funds that are based on sound programming. The National Institute of Justice (NIJ) is an active participant in BJA's evaluation program. BJA and NIJ jointly develop evaluation guidelines and conduct comprehensive evaluations of selected programs receiving Byrne discretionary and formula grant funds. BJA worked closely with NIJ to develop the guidelines and criteria for documenting the six program evaluations presented in this monograph. The overall effectiveness of Federal funding is best reached through a program development model that tests innovative State and local ideas in action and then shares the information gained with the broadest audience. Promising steps have been taken, and much has been learned at the State and local levels regarding what works and what does not work. ------------------------ Chapter 2 CALIFORNIA Donovan Prison Therapeutic Community Program (Continuity of Care Project) The evaluation research presented in this report was conducted by Harry K. Wexler, Ph.D., Senior Principal Investigator, National Development & Research Institutes, Inc., and Lois Lowe, Ph.D., Office of Substance Abuse Programs, California Department of Corrections. The Continuity of Care Project, which was established using Byrne grant funds, with the assistance of the California Office of Criminal Justice Planning, enables graduates of institutional treatment to continue intensive substance abuse treatment, while on parole, in a community setting in Vista, California. Program Overview In 1989, following increasing evidence that substance abuse is a driving factor in both prison overcrowding and parole revocation, the California Department of Corrections (CDC) formed the Office of Substance Abuse Programs (OSAP). In 1990, OSAP, in partnership with Amity, Inc., a nationally recognized community-based treatment organization, initiated California's first intensive substance abuse treatment program at the R.J. Donovan Correctional Facility near San Diego. The treatment program that was developed allows selected drug-abusing, medium-security inmates to serve their last year of incarceration in a therapeutic community (TC) setting. The 200 inmates assigned to the project participate in group meetings, seminars, group and individual counseling, video feedback, relapse prevention, and urine testing. Preparole planning is also provided to assist inmates in returning to the community. Upon successful completion of the in-prison program, a limited number of inmates paroled to San Diego County are able to enter the community residential facility in Vista, California. This facility initially offered 6 beds but quickly expanded to 40; full treatment services are available; and participants may stay until they are ready to make the transition to independent living, up to 1 year. The average stay is 5.3 months. The TC at R.J. Donovan is housed in a unit isolated from the general population, but participants are integrated with other prisoners for certain activities. Seminars, ancillary groups, and workshops are provided for family, friends, and other parolees. Goals and Objectives Established with the assistance of the California Office of Criminal Justice Planning in 1991, the Continuity of Care Project enables institutional treatment graduates to continue intensive substance abuse treatment in a community setting in Vista, California, while on parole. The goals of the Continuity of Care Project are to provide substance abuse treatment services for offenders graduating from the in-prison component, to provide preparole planning for the in-prison participants to enhance the likelihood of success on parole, and to assure continued community safety. The objectives to achieve these goals are to: o Provide ancillary groups, workshops, or seminars at the facility for a minimum of 400 nontransition parolees, family members, and friends. o Graduate at least 190 inmates, 70 of whom will enter the transition facility for an average of 120 days. o Complete transition plans for in-prison participants. o Provide contacts and conduct preparole interviews. o Provide supervision by the project's parole agent staff, which will administer urine tests, conduct initial interviews, and make case contacts at the facility. o Monitor the number of incidents requiring an activity report, positive urine samples and types of drugs detected, participants returned to custody, successful parole discharges, hours of community service donated by participants, and participants employed. Program Activities/Components The Continuity of Care Project includes three distinct program areas: preparole transition planning, substance abuse treatment services, and parole supervision. Preparole Transition Planning. At the R.J. Donovan Correctional Facility, contracted treatment personnel assess inmates and develop program plans. The transition plan identifies individuals' specific shortcomings and establishes goals that must be achieved for successful parole. A transition plan may include residential treatment; outpatient treatment; participation in outreach groups or seminars and in Alcoholics Anonymous and Narcotics Anonymous programs; and monitoring of the parolee's relationships with past associates. Involving the project's parole agent is critical to preparole planning. Often this agent is the offender's only positive parole contact. Clearly establishing the offender's responsibility regarding parole supervision offers the offender the maximum opportunity for successful reintegration; coordinating and integrating parole supervision and treatment creates a considerable force to motivate the offender and reinforce positive behavior. Substance Abuse Treatment Services. Project treatment components offered at the continuation facility are consistent with those in the institution and include individual counseling, ongoing assessment, group counseling, psychodrama, problem-solving skills development, self- examination, and educational seminars. These activities are supplemented with regular workshops in substance abuse education, relationships, recreation, nutrition, support networks, family dynamics, personal chronologies, and communication skills. All the activities are intended to provide participants with the skills necessary for successful adjustment. Parole Supervision. The parole agent assigned to the project as a liaison, with treatment and custody staff at the institution, provides ongoing contact with other community programs and communicates with the County Alcohol and Drug Administration staff to facilitate long-term parole planning. The parole agent works closely with the continuance facility staff and its residents while monitoring urine tests and obtaining collateral information to validate the progress of offenders through the program. The parole agent also monitors offenders' behavior in the community to ensure public safety. Performance Measures and Evaluation Methods A. The National Development & Research Institutes, Inc. (NDRI) completed a study entitled "Return to Custody Follow-up of Participants Paroling and in Treatment, November 1991-February 1996." The purpose of the study was to answer the following questions: o What percentage of community residential program participants returned to State prison custody following community treatment at 12 and 24 months postparole? Is there a difference in returns by program completion status, completed versus dropout? o What is the average number of days program participants were on parole prior to a first return to custody? o What is the average number of days participants were in prison for a first return? o What was the reason for a first return to State prison? Study subjects were identified and classified as to community residential program completion status by the community residential program director. Personal characteristics and movement transactions of the participants were abstracted from CDC's computerized database, the Offender Based Information System. The data included these variables: length of time on parole, return-to-custody status, days back in custody, and days to first return to custody. No participants were contacted. The information was then entered into a computerized database for analysis using statistical software. The data were analyzed in terms of community residential program completion status, completed (remained in an Amity community residential facility 90 days or longer) versus dropout (did not remain in community treatment 90 days), and length of time on parole, 12 and 24 months postparole. Study participants consisted of all Donovan Prison program completers who were paroled into San Diego County and entered an Amity community residential facility from November 1991 through February 1996, and were on parole 12 months or longer. Of these, 126 were on parole at least 24 months. The number of persons not returned/returned after 12 months was 142 and 46, respectively. At 12 months, the age range of the participants was 21 to 67 years, the average age was 35.5 years, the number not returned to custody was 36.0, and the average age of the participants who were returned in 12 months was 33.9. The race/ethnicity was Caucasian, 50.5 percent; African-American, 27.7 percent; Hispanic/Latino, 20.2 percent; and other, 1.6 percent. The percentages of not returned and returned by race/ethnicity were Caucasian, 50.7 percent/50.0 percent; African-American, 30.3 percent/19.6 percent; Hispanic/Latino, 16.9 percent/30.4 percent; and other, 2.1 percent/0.0 percent. Analysis of variance was used to determine whether continuous variables (days returned to custody, days to first return, age) were statistically equivalent for the completed and dropout groups. The nonparametric chi- square test was used to compare the categorical frequency data. B. NDRI, under a grant from the National Institute on Drug Abuse, conducted a comprehensive followup study on a random sample of in- prison treatment participants, in-prison plus community treatment participants, program dropouts, and nontreatment (control) subjects. The focus of the study included: o Profiles--to describe the profiles of prison inmates with histories of substance abuse in terms of background, level of functioning, social and psychological adjustment, criminality, and substance abuse. o Effectiveness--to evaluate, in terms of both process and outcome, the effectiveness of a modified prison-based TC for inmates with histories of substance abuse. o Effectiveness--to assess the incremental value of extending the TC experience from prison into the community. The total sample, randomly selected, was 715 persons categorized as follows: controls, 290; prison TC dropouts, 95; prison TC completers, 193; aftercare TC dropouts, 45; and aftercare completers, 92. Data collection was effected at intake, 6 and 12 months into treatment, and 12 months posttreatment. Instruments included a Baseline Interview Protocol, which provided personal history and background information; the Tennessee Self Concept, Beck Depression Inventory, etc., which indicated psychological status; a Diagnostic Interview Survey, which indicated psychiatric status; the TC Clinical Progress Scale, the Correctional Institutional Environment Scale, etc., which provided insight into client perceptions and progress; and a review of criminal justice histories. Characteristics of the study population were as follows: o The random selection procedure was successful because few differences were found between the treatment and nontreatment groups. o Average age was 31, and the population was racially balanced. o Psychiatric diagnoses included 51 percent antisocial personality, 33 percent attention deficit hyperactivity disorder; and 17 percent depression/dysthymia. o The men were primarily stimulant, intravenous (IV) users. o Of the participants who were at high risk for HIV, 58 percent were IV users, 63 percent injected with dirty needles, 25 percent shared needles with strangers, and 97 percent practiced sex. o Of the study population, 75 percent committed violent crimes, such as assault, kidnaping, rape, and murder. C. Amity, Inc., has contracted with Lois Lowe, Ph.D., OSAP, CDC, to do a process evaluation of the entire Donovan Prison TC program. Program Evaluation Findings and Results A. NDRI: "Return to Custody Follow-up of Participants Paroling and in Treatment, November 1991-February 1996." 12 Months Postparole (N=188). Of the 132 participants who completed treatment, 22 (16.7 percent) had been returned to State prison custody 1 or more days. In contrast, 24 (42.9 percent) of the 56 participants in the dropout group were returned. The difference in the return rates between the two groups is statistically significant. The average number of days on parole prior to a first return to custody was 236.3 days for the completed and 210.9 days for the dropout group. The difference in days between the two groups is not statistically significant. The average number of days in custody following a first return was calculated for both groups. The average for the completed group was 86.0 compared to 106.2 days for the dropout group. The difference in days between the two groups is not statistically significant. 24 Months Postparole (N=126). Of the 87 participants in the completed group, 27.6 percent had been returned to custody. The return rate for the 39 participants in the dropout group was twice as high--61.5 percent. The difference between the two groups is statistically significant. The average number of days on parole prior to a first return to custody was 363.7 days for the completed and 324.5 days for the dropout group. The difference in days between the two groups is not statistically significant. The average number of days in custody following a first return was calculated for both groups. The average for the completed group was 279.8 days, compared to 263.9 days for the dropout Group. The difference in days between the two groups is not statistically significant. Reason for First Return to Custody. This variable is of interest to correctional management because the more serious the reason for return, the longer the inmate will remain in prison custody. The current average cost (July 1996) to house an inmate in prison for a year is $21,631, whereas the annual average cost to supervise a parolee within the community is $2,100. In no instance studied were the differences between the two groups statistically significant. B. The NDRI comprehensive study has been completed, but the results have not yet been released. However, some outcome findings were presented in November 1996. Reincarceration. The prison TC was effective in reducing reincarceration rates 12 and 24 months after release for completers. The cumulative effects of the prison TC and community aftercare TC were greater than the effects of the prison TC alone in reducing reincarceration rates at 12 and 24 months after release. Twelve months after release, 40 percent of the prison TC completers were reincarcerated, and 7 percent of those who completed prison TC and community aftercare were reincarcerated. Almost 50 percent of the control subjects were reincarcerated after 12 months. Of those completing the prison TC, 53 percent were reincarcerated 24 months after release, and of those who completed the prison TC and the community aftercare, 16 percent were reincarcerated 24 months after release. Sixty-five percent of the control subjects were reincarcerated 24 months after release. Time Until Reincarceration. A significant positive relationship was found between the amount of treatment and "time until reincarceration" at 12 and 24 months after release from prison. Comparison of Donovan Prison TC and Other Studies. The recidivism results are similar to the results of evaluation studies of prison and aftercare TCs in Delaware and Texas, but the participants in this program had more severe criminal backgrounds than those in the other programs. The next analytical steps include determining the impact of treatment on other outcomes, such as HIV risk behaviors and social functioning, inmate characteristics related to success and failure, and the relationship of treatment and cost avoidance. C. The preliminary working group met in April 1997. No results are available at this time. ------------------------ Chapter 3 COLORADO Intensive Supervision Probation Program The evaluation research presented in this report was conducted by Kim English, Susan M. Chadwick, and Suzanne K. Pullen of the Office of Research and Statistics, Colorado Division of Criminal Justice. The Intensive Supervision Program (ISP) has received financial support from Byrne grant funds. Receipt of these funds by the State is accompanied by a mandate to evaluate the impact of local projects supported by the grants. Program Overview ISP was designed as an intermediate sanction intended to divert prison- bound offenders away from prison and into a community alternative. ISP began in the mid-1980s in Colorado. This program emphasizes close monitoring and clearly defined treatment plans for offenders. ISP is a modified version of probation, with smaller caseloads and more stringent program requirements, including frequent monitoring (both scheduled and random) and mandatory treatment participation for specific needs. Goals and Objectives The goal of ISP is to divert prison-bound offenders away from prison and into a community alternative while maintaining public safety. The objectives of ISP are to provide close monitoring of clients and to develop clearly defined treatment plans for offenders. Program Activities/Components Typical components of ISP include the following: o A minimum of two scheduled contacts per week with an ISP officer. o Random contacts with an ISP officer. o Prohibition of drug and alcohol use. o Participation in treatment, as necessary. o Payments of $20 in supervision fees each month. Performance Measures and Evaluation Methods This evaluation was conducted to answer the following three questions: o Does ISP divert offenders from prison? o Does ISP protect the public? o What are the successful treatment/surveillance components of ISP? To analyze these issues, ISP offenders were compared with probationers, community corrections clients, and prisoners. For the question about offender diversion, sentencing decisions from nine Colorado judicial districts contained in the Division of Criminal Justice's (DCJ) annually collected court data file were analyzed. The sampling unit was cases reaching disposition in 1990. The sample comprised 100 percent of offenders sentenced to prison and community corrections and 50 percent of offenders sentenced to probation, including sentences to ISP. To answer the questions about protecting the public and successful treatment/surveillance components of ISP, data were obtained from three sources: the State Judicial Branch's ISP database, DCJ's 1990 court database, and the database of all offenders terminated from diversion community corrections in FY 1990-91, maintained by DCJ's Office of Community Corrections. For all samples, cases whose sentences were terminated by June 30, 1991, were selected to allow for a 12-month followup. To avoid sampling and generalization problems, the samples were stratified by conviction offense: theft, burglary, drug offense, assault, sexual assault, and robbery. Because ISP was the smallest study group, in order to have comparable sample sizes, the number of cases in ISP that met the sampling criteria set the sample size for the other two study groups. In ISP, 263 cases met the sampling criteria. For probation and community corrections, the stratified samples were reduced by computerized random sampling. Additionally, to answer the public safety question, each offender was tracked for 12 months following program completion. Arrest data maintained by the Colorado Bureau of Investigation were used to determine whether an offender was arrested in Colorado for a new misdemeanor or a felony during this 12-month period. Arrest for either a new crime (misdemeanor or felony) or a technical violation served as the measure of public safety. For each offender, data on all arrests during the 12-month period were collected, and the most serious arrest was coded as the outcome variable. Does ISP Divert Offenders From Prison? To answer this question, the characteristics of probationers, ISP clients, and prisoners were compared. If ISP participants are prison-bound offenders, as planned by the General Assembly, then ISP offenders and prisoners should look similar on the profile analysis. Current offense, demographics, criminal history, and the various needs that are present among these offenders were examined. Does ISP Protect the Public? The evaluation compared program completion rates of offenders in three community placements: probation, ISP, and community corrections. On the factors studied, the profiles of ISP and community corrections offenders did not differ statistically from each other; however, these two groups were quite different from probation clients. Factors examined included gender, mean age, education (last grade completed), employment at arrest, offender needs (mental health treatment, alcohol treatment, drug treatment, financial counseling), criminal history score, and mean age at first arrest. In addition, two measures were used to answer the question of how safe the public is: (1) program termination and, for those who terminated the program successfully, (2) rearrest during the 12 months following program termination. What Are the Successful Treatment/Surveillance Components of ISP? To determine the components of ISP that might relate to successful program completion, the evaluation analyzed the following factors: o Psychological counseling. o Alcohol abuse counseling. o Alcoholics Anonymous (AA) meetings. o Substance abuse educational sessions. o Vocational training. o Drug and alcohol detection tests. o Electronic monitoring. o Employment. o Frequency and type of contacts by an ISP officer. o Risk/needs of the offender. o Type of current offense. o Length of stay in ISP. Program Evaluation Findings and Results Does ISP Divert the Offender? Based on criminal history, the ISP program in Colorado appears to be diverting offenders who would otherwise be prison bound. Differences in the criminal history profiles of offenders placed on ISP and those sentenced to probation were clearly illustrated in the analysis of the data. All criminal history factors (except age at first arrest) rated lower for probationers compared with ISP clients. A number of similarities exist between offenders sentenced to community corrections and those sentenced to ISP. This is somewhat expected because community corrections is also designed to divert offenders from prison. Of particular interest is the similarity of ISP and community corrections offenders with prior adult arrests for both violent and nonviolent offenses. The ISP sample actually had a slightly higher ratio of offenders with a history of adult violent and nonviolent arrests compared to the community corrections sample, but the difference is not statistically significant. An important distinction between ISP clients and offenders in prison, however, is that the prisoners had, on average, only one more arrest on their rap sheet than offenders who were placed on ISP. It might be inferred then, that ISP offenders are "one arrest away" from prison. Given that the profile of probationers clearly looks different than the profiles of ISP clients, community corrections clients, and prisoners, it appears that ISP is, on average, not widening the probation net, but is providing a placement alternative for would-be prison-bound offenders. How Safe Is the Public? Probation had the highest proportion of normal program completions (68.5 percent), compared with community corrections (43 percent) and ISP (46.5 percent). Conversely, probationers had the highest failure rate for the commission of new crimes (10.8 percent), ISP was next (7.5 percent), and community corrections had the lowest rate (3.4 percent). While offender behavior may vary by virtue of program selection procedures, it also seems probable that the closer offenders are supervised, the more likely officers are to catch clients violating program rules. Thus, in programs with contact standards that are more stringent than traditional probation (e.g., residential community corrections and ISP), an offender is more likely to be caught breaking the rules before he is caught breaking the law. Conversely, on probation, where surveillance is less stringent, an offender has a greater opportunity not only to break the rules, but also to commit new crimes. Empirical findings underscore the differences: Terminations due to technical violations are nearly 2 1/2 times greater in ISP and three times greater in community corrections than they are on probation (45.5 percent, 46.7 percent, and 16.9 percent, respectively). The three community sanctions studied in this evaluation appear to provide a reasonable degree of safety to the community. The probation group, despite a sample designed to minimize variation among the study groups, consisted of considerably fewer serious offenders compared with those placed in ISP and community corrections and would logically warrant fewer supervision resources. This explained the relatively low cost of $2.36 per day, per offender. However, just over 10 percent of the probation group committed a new crime while on probation. Nearly 8 percent of the ISP clients (at a cost of $6.07 per day) reoffended with a new crime, and community corrections--the most restrictive and expensive ($33.33 per day) of the programs studied--logged a new crime rate of less than 4 percent. Within 12 months of release, 99.3 percent of the former probationers were not rearrested; 91.3 percent of the prior ISP clients did not lapse into recidivism; and 77.6 percent of the "graduated" community corrections clients remained arrest free. While community corrections clients did significantly less well than the probation groups, it is nevertheless important to compare the recidivism rate of these three community programs with the success rate of the overall prisoner population. According to Department of Corrections data, 60 percent of the entire group of parolees remained arrest free during the first 3 years of parole or discharge. Comparing the three groups for violent felony arrests, of those offenders released from probation and ISP, none were arrested for violent felonies in the 12 months after successful release from prison. What Program Components Relate to Completion? Two important resources--officer contact and sentence duration--did not affect overall success rates in this study. Three program components statistically improved an offender's probability of successfully completing ISP: full- time employment, individual and/or group counseling, and participation in AA. Nearly three-fourths (73 percent) of offenders who held a full-time job while participating in ISP completed the program without a revocable incident. This is in considerable contrast to the 28.6 percent of unemployed clients who successfully completed ISP. Further, fewer than half (46 percent) of offenders who worked part time or sporadically completed ISP successfully. Offenders who participated in individual and/or group psychological counseling were significantly more likely to succeed in ISP. Of the offenders who received individual counseling, 70 percent went on to successfully complete ISP. This compares with 30 percent for those who did not complete the ISP program. Additionally, for those who received group counseling, nearly 68 percent completed the program, compared with 32.6 percent who failed to successfully complete the program. For this study group, participation in AA meetings significantly increased an offender's probability of successfully completing ISP. A significantly higher proportion of ISP clients who attended AA (73.4 percent) succeeded in ISP, compared with 26.1 percent of offenders who failed to complete the ISP program. This evaluation of the Colorado ISP program recognizes that it is statistically impossible to separate the effect of the continuity-of-care (ISP clients' progress to regular probation) component from any ISP program- ming factor. Analyzing the impact of continuity of care would require an experimental research design in which half of a matched sample received ISP in its current form and the other half received ISP with no probation followup. Any findings reported here must be considered in the context of the continuity-of-care provisions. It is possible, for example, that the progression from very strict monitoring and programming to regular probation--a fairly unobtrusive sanction--is the driving factor for program success and 12-month outcome success. Experimental designs are very costly and very difficult to implement in a real-life setting such as probation.[1] ------------------------ Chapter 4 FLORIDA Drug Treatment and Rehabilitation Programs for Offenders The evaluation research presented in this report was conducted by C. Aaron McNeece, Ph.D., Principal Investigator, and Charles M. Daly, Ph.D., Project Manager of the Florida State University, Institute for Health and Human Services Research. Byrne grant funds, administered by the Florida Department of Community Affairs (DCA), are distributed annually, on an equity basis, to local units of government, which then decide how to best use these funds to address substance-abuse-related criminal activities in their communities. Program Overview The Drug Control and System Improvement (DCSI) formula grant funds are administered by the Florida DCA. Funding is distributed annually, on an equity basis, to local units of government that then decide how best to use these funds to address substance-abuse-related criminal activities in their communities. Many of the local units recognize the importance of including substance abuse prevention and treatment programs for offenders as part of their overall strategy to reduce drug-related crime. In 1992, the Florida State University (FSU) Institute for Health and Human Services Research entered into a contract with DCA to conduct an evaluation of 15 local substance abuse prevention and treatment projects funded through DCA. The projects were located in five counties and one city throughout the State. Goals and Objectives The goal is to break the cycle of drug use and related crime by reducing the demand for drugs. The objectives of the program are: o To retrain the offender's physical and psychological needs through treatment services. o To substitute viable academic and vocational skills that offset the financial incentives for drug involvement. A composite definition of the ideal client outcome is an adult offender who successfully completes the program and acquires living skills to enable him or her to become a productive member of society and to remain out of the criminal justice system. Program Activities/Components The DCSI grant is administered by DCA. DCA staff: o Conduct fiscal and program monitoring of all grant-funded projects, at least annually, using an automated grants processing, monitoring, and management information system (GPMMIS), which tracks program expenditures and performance. o Require subgrantees to submit quarterly, annual, and final program performance reports. Data from these reports are entered into GPMMIS to assess the efficiency and effectiveness of services. o Direct subgrantees to submit and receive approval on numerous fiscal documents such as budgets, claims for reimbursement, budget amendments, etc. These reports are also entered into GPMMIS for tracking. o Contract with FSU to evaluate selected purpose areas. Subgrantees are awarded funds on a formula basis that takes into account the county's size and problems with substance abuse. At least 51 percent of the units of government within the county, representing at least 51 percent of the county's population, must sign off on a project to receive funds. This approach, unique to Florida, has resulted in programs that are based more on community and social services and medical treatment than on law enforcement. Treatment and rehabilitation services are furnished either by the State or by local providers to drug offenders who are incarcerated, on probation, or in pretrial status. The settings in which the services are delivered vary, but are primarily prisons, jails, or in the community. In spite of differences, treatment programs are standardized in Florida. They are licensed by the Department of Health and Rehabilitative Services (DHRS) and must comply with minimum State standards. Services available are defined and codified in the Florida Administrative Code. GPMMIS offers local-level project managers the ability to make program adjustments to meet their performance objectives, because current performance on objectives is instantly available to them. Continuing education efforts by DCA are convincing more project managers to develop and use performance-based objectives. Performance Measures and Evaluation Methods FSU and DCA have organized the data differently. FSU categorizes offenders into the type of program service in some instances or the type of offender status in others, and divides the population into adults and juveniles. The DCA database aggregates offenders of all ages and separates results by provider/venue. DCA staff established uniform objectives to measure program activities for each purpose area. Within the treatment and rehabilitation purpose area, these objectives vary by treatment modality, but generally, they measure the number of offenders receiving a specific set of services, such as intake and screening, diagnostic services, case management, or short- term counseling. Project staff establish the level of performance they expect to achieve and communicate this expectation to DCA when applying for grant funds. The services provided most frequently are procedural (intake/screening, case management, tracking, and court liaison). This service array matches the program definition for successful completion of treatment/rehabilitation: completion of the steps outlined in an individualized treatment plan (as opposed to assessing long-term followup data). Fifteen treatment programs were evaluated by FSU under a contract with DCA. The programs were located in Broward, Escambia, Indian River, Polk, and Seminole Counties and the City of Hollywood. While these programs are not new or different from a program design standpoint, they are unique in emphasizing the importance of substance abuse prevention and treatment as part of broad local antidrug strategies. An essentially descriptive evaluation methodology was prepared by the evaluation team and adapted to the unique characteristics of each program. New data were collected and analyzed when possible; existing data were evaluated; and project staff and community stakeholders were interviewed for their perceptions of program strengths and weaknesses. The evaluation framework provided for the review of five basic aspects of program performance: o Program Design: The evaluation explored the initial and ongoing soundness of each program's design. Special emphasis was placed on assessing the ability of program managers and staff to adapt to changing needs and circumstances in a timely and efficient manner. o Program Management: The evaluation assessed the strengths and weaknesses of each program's organizational structure, procedures, and oversight, as well as its integration into larger substance abuse treatment and human services delivery systems. o Service Area Coverage Accountability: This section investigated how well each program was reaching its target population and geographical service area. o Service Delivery Accountability: The volume and types of services actually delivered by each program were compared with the level of service delivery originally projected in the grant proposal. o Program Outcomes: This was the most important and most difficult component of the evaluation. Within the limitations of this evaluation post hoc design, the team attempted to identify and measure reasonable indicators of the impact of the program on compliance with treatment and the future behavior of target populations. As an additional criterion, the evaluation team examined adult and juvenile posttreatment arrest histories. This analysis was based on the assumption that while a low posttreatment arrest rate could not be conclusively attributed to the effects of participation in the program, it could provide one important indication of movement toward the major goal of reducing offender recidivism. The evaluation methodology called for obtaining arrest histories on a representative sample of clients served by each project. Arrest histories were compiled for the period beginning 2 years prior to beginning treatment and extending up to 2 years after exiting treatment. Adult arrest histories were obtained through the Florida Department of Law Enforcement (FDLE) and juvenile arrest data from the DHRS database. In theory, if treatment makes a difference, offenders exposed to education and/or treatment should have less posttreatment involvement with the juvenile or criminal justice system than the substance-involved offender population as a whole. Program Evaluation Findings and Results In instances in which the findings appeared critical of program efforts, adjustments were offered with the intention of effecting improvements in the delivery of services and enhancing success rates. It was suggested that some project managers rethink the design of their programs to obtain referrals or keep offenders in treatment. Flexibility and proactive management are key elements of change. Generally, first-line managers and project staff were found to be well trained and highly competent. With a few significant exceptions, staff turnover was low and morale exceptionally high. The programs were also found to be adequately integrated into community substance abuse treatment delivery systems. Potential referral sources and other treatment providers were well aware of the programs, and administrators were well informed about most communication and coordination issues, although they were not always sufficiently assertive in making needed changes. Performance objectives submitted in the grant proposals should be viewed as dynamic management tools and used more frequently. They should be communicated to delivery staff, who need to know what they are supposed to accomplish and whether they are making progress in achieving objectives. The evaluation team found that the majority of projects had adhered to their priority target groups. This was measured in two ways: (1) a demographic profile of clients served by each project was compared with the population of the county served and with FDLE arrest data for the county; and (2) the FDLE and children, youth, and family arrest histories of adults and juveniles were examined to verify offender status for each project's client population. No arrest records were found for 24.5 percent of the adult Caucasian males, for 16.3 percent of the adult African- American males, and for 40 percent of the juveniles. The relatively high proportion of offender admissions in most counties is a strong, positive indicator that the majority of treatment providers have made their programs readily accessible to substance-involved offenders. The volume and types of services actually provided were compared with volume and types of services originally planned. The primary sources of information were project case records and the quarterly and annual reports submitted to DCA. Definitions need to be uniform from program to program so that their reports can be used for analyses. Program Outcomes. One frequently used measure of program success is the completion of the planned course of treatment through active participation on the part of the client. The combined sample data for 164 juveniles revealed that 26.8 percent successfully completed treatment. Females were more likely to complete treatment than were males, 34.0 percent versus 24.3 percent, and Caucasian males were significantly more likely to complete treatment than were African-American males, despite the fact that two of the three juvenile programs were designed to target young African-Americans with culturally sensitive education and/or counseling needs. Overall, 42.9 percent of the adult sample successfully completed treatment. Caucasian males had a completion rate of 56.7 percent and African-American males 49.5 percent; adult females had rates of 31.4 percent and 22.6 percent for Caucasians and African-Americans, respectively. Completion rates varied from program to program, as did toleration of deviation from program rules and/or court orders. There was no "standard." In the absence of controls for client and program characteristics, the direct effect of education or treatment upon client arrest rates cannot be estimated. Conversely, it cannot be definitively concluded that treatment did not influence the rates. The posttreatment findings do provide a useful indicator of program outcomes. Of 487 adult offenders, randomly selected, who had been out of treatment for at least 6 months, 22.1 percent had at least one new arrest; 6.0 percent of all new arrests were for drug offenses. Twelve months after treatment, 353 offenders had completed treatment, and 40.7 percent had been rearrested at least once since leaving treatment; 12.5 percent of new arrests were drug related. By 18 months after treatment, the percentage of new arrests rose to 66.1 percent, with 28.9 percent of those drug related. Adult after-treatment arrests were also compared using the completion status of the offender. Six months after treatment, the rearrest rate for those who had successfully completed treatment was less than half that of their cohorts who had not, i.e., 13.0 percent versus 29.8 percent. Treatment completers who had been out of treatment at least 1 year had a 30.2 percent rearrest rate, compared with 47.1 percent for noncompleters. For this group of offenders, it can be concluded that completing the prescribed course of treatment is significantly more likely to result in positive client outcomes. Juvenile arrests at 6 months after treatment were 28.6 percent, with 3.7 percent drug related, all of which were for noncompleters. More than 55 percent of juveniles had at least one new arrest after 12 months, with 9.5 percent drug related. After 18 months, the percentages were 84.0 and 28.0, respectively. After 6 months the percentages for rearrest of completers and noncompleters was 20.5 and 31.8, respectively. The most positive finding was that even after 12 months, none of the treatment completers had been arrested on a new drug-related charge. ------------------------ Chapter 5 ILLINOIS Gang Violence Reduction Project The evaluation research presented in this report was conducted by Irving Spergel, Ph.D., and Susan F. Grossman, Ph.D., of the School of Social Service Administration, University of Chicago. An evaluation of the Gang Violence Reduction Project was conducted by the University of Chicago, School of Social Service Administration, using Byrne grant funds. Program Overview The Gang Violence Reduction Project was initiated in July 1992 in Little Village, a mainly Hispanic/Latino lower income and working-class community. Little Village has one of the highest gang-violence rates of the Hispanic/Latino and African-American communities in Chicago. The area is the home base for two of the largest Hispanic/Latino gangs in the Midwest, the Latin Kings and the Two Six. The project was funded by the U.S. Department of Justice (DOJ), Bureau of Justice Assistance, Federal Violence Reduction in Urban Areas Program. Funds were appropriated through the Illinois Criminal Justice Information Authority to the Chicago Police Department, which administers the project. The project assumes that a relatively small number of youth, 17 to 25 years old, is responsible for most of the serious gang violence in the area. It also addresses the fragmented efforts of police, other justice agencies, youth agencies, and local organizations. The community and justice system must direct special attention to issues of control and social development, including providing jobs, training, and education for these hardcore older gang youth who are responsible for most of the violent gang crime and are models for young recruits to the gang. Goals and Objectives The chief goal of the Gang Violence Reduction Project is to lower the level of gang violence or reduce the rate of increase among older hardcore targeted youth, as well as in the Little Village area as a whole, compared with other similar nontargeted youth in the same area or in other areas of the city with a high rate of gang violence. Highly violent or highly at-risk gang members, 17 to 24 years of age, were targeted in the first 4 years of the project; an additional group of violent and highly at-risk gang youth, 14 to 16 years of age, was added in the fourth year of the project. The first objective of the project was to reduce, absolutely or relatively, the levels of serious gang violence (particularly gang homicides, aggravated gang batteries, and aggravated gang assaults) among 200 youth, mainly from 2 large gangs, targeted in the program. The second objective was to reduce the aggregate level of gang violence in the target beats of the 10th District of the Chicago Police Department, which included Little Village, in comparison with preselected similar high-gang- crime beats in the Pilsen area of the 12th District and other similar, mainly Hispanic/Latino, beats in five other districts of the city (8, 9, 13, 14, and 25). A third objective was to lower the level of gang crime as perceived and experienced by residents and local organizations in Little Village compared with Pilsen. Program Activities/Components The strategy of the program is to target selected gang youth; coordinate the justice system's response; mobilize local citizens' and organizations' interest, resources, and efforts; and plan and implement relevant research evaluation. The strategy evolved from a Youth Gang Violence Suppression and Intervention Program conducted between 1987 and 1991 and funded by the Office of Juvenile Justice and Delinquency Prevention of DOJ. The Gang Violence Reduction Project team consists of a 10th District Police Unit of two full-time tactical and two part-time neighborhood relations officers, including a sergeant; a Cook County Adult Probation Unit of three full-time probation officers and a supervisor; and a University of Chicago School of Social Service Administration Unit of three community youth workers and a supervisor. A field supervisor of the Pre-Trial Services Unit of the Cook County Circuit Court is also involved in the project. Weekly unit staff meetings, biweekly interagency field meetings, and periodic administrative meetings across agencies provide direction for field staff operations. Gang leaders, influential members, and shooters (gunshooters) are identified, and positive relationships are established to assist gang youth in rethinking their values and the consequences of antisocial behavior, encouraging them to pursue legitimate careers. Extra constraints for gang youth continuing to engage in violent behavior include additional contacts, surveillance, arrest, and special pretrial services. Fieldworkers across units conduct contacts at least three times per week to deal with gang crises and related problems. Overlapping, flexible fieldwork schedules cover prime gang activity times. Backup police coverage is arranged for times when assigned tactical officers are not on duty. Visits to gang leaders and influential members in jail and prison facilitate truces and peace arrangements between warring gangs or factions, although arranging truces is not a principal objective of the project. A local community organization, Neighbors Against Gang Violence (NAGV), has been developed. It comprises representatives of local Catholic and Protestant churches, the alderman's office, youth agencies, an employment agency, block clubs, and neighborhood residents whose purpose it is to support the project team. Increasing communication and interaction with gang youth and opening up community resources for them is a major objective of NAGV. The Gang Violence Reduction Project has assisted NAGV in the development of retreats, parent support meetings, and church membership gatherings that address the gang problem. The Gang Violence Reduction Project and NAGV periodically hold meetings with the District Commander, the Probation Department, and Research and Development Unit representatives to orient the Commander about the purpose and nature of the project; provide guidance on how to strengthen the structure of the Gang Violence Reduction Project; discuss how the Commander can take increased responsibility for its management and possible expansion in the district; and encourage the Commander to use community contacts and provide job contacts for targeted gang youth. The Gang Violence Reduction Project assists probation administrators with community agency contacts; facilitates the decentralization of probation activities targeting gang youth in Little Village; and facilitates collaboration among probation administrators, community youth workers, and police in the development of special group counseling and job development programs. In addition, the program has assisted the Probation Administration in modifying its procedures to permit sharing of selected probation data with community youth workers. The Early Warning Data System (EWDS) maintained by the Illinois Criminal Justice Information Authority provides monthly reports on the types of gang violence in Little Village and comparable areas in other police districts. Additional police and EWDS data are gathered to determine patterns of change in gang crime. Performance Measures and Evaluation Methods A planning and evaluation process included the implementation of baseline community and individual gang member surveys. Individual-level data from police and juvenile and adult criminal courts on program youth have also been made accessible. Staff meeting, program service, and other reports have been developed for accountability, program process, and project outcome measurement purposes. Research and evaluation material reflecting the program process and project outcome are periodically provided to the project employees. Formal and informal discussions are conducted at staff meetings about the nature and causes of street gang crime within a historical and comparative or cross-cultural perspective. Retreats, special seminars, and classes are conducted on the gang problem, with attention to the education interests and needs of project staff. A great deal of data are collected to assess the program's efficacy. Information gathered includes minutes of staff meetings; interviews with informants; cellular phone records; responses from local community residents, agency representatives, and gang youth served; workers' program reports; monthly program summaries; interviews with gang members; field research observations; and other reports of the Chicago Police Department. This information indicates the number of youth contacted; the purpose and frequency of the contacts; the actions taken by the workers; the results of the workers' efforts; the number and kinds of jobs offered to and taken by targeted youth and for how long; the number of school referrals and the kinds of school-related problems that exist; the number of home visits and types of problems encountered there; the quality of worker contact with youth; the level of gang member satisfaction with worker services; the number of gang contacts; the number of gang fights, shootings, and injuries; the number of gang member deaths, both gang- and non-gang-related; the planned activities in which youth engage; the number and types of actions by workers to avert gang conflict; the level of community awareness of the Gang Violence Reduction Project; the number, purposes, and conclusions of meetings with the Commander of the 10th District; and perceptions of the community as to police effectiveness with regard to gangs. The activities of NAGV are monitored through the minutes of their meetings; monthly reports to the Chicago Police Department; the minutes of their 2-day retreat and special community meetings; and funding proposals. These show the number of meetings held; the number and kinds of actions taken at board meetings; the number and kinds of plans and activities organized and implemented; and the quality of the organization's leadership. Qualitative and quantitative changes in the involvement of the Police and Probation Departments are assessed through monthly program reports; memoranda to probation and police administrators; the number and purpose of interagency meetings at the main probation and Little Village decentralized offices; and the number of instances of Probation Department collaboration with the police and community workers and the nature of the problems addressed. Other performance measures include the number of interagency and community work staff meetings at which gang research or training data were presented and the nature of the content provided. An evaluation of the project was conducted by the University of Chicago, School of Social Service Administration. Interviews were conducted with gang youth in the summer and fall of 1995, in large measure comparing the results of earlier analyses. Three different program cohorts of the target gangs, the Latin Kings and the Two Six, were included in the analysis. For the first cohort (N=72), data from three points in time were examined to determine changes in violent and deviant behavior, educational attainment, employment, gang status or attachment, and involvement in legal and illegal moneymaking activities. The second program cohort (N=31) included individuals interviewed in 1994 and 1995, allowing for a 2-year comparison. Finally, a third program cohort (N=45), composed primarily of younger gang members (14-16 years of age), was interviewed for the first time in 1995 and evaluated, both separately and in relation to the other cohorts. Respondents in all three cohorts were asked about their activities in relation to a series of 16 crimes, 9 of which involved violence either with or without a weapon and 7 of which were property-related crimes. The youth were also asked about drugselling behavior. A series of indexes was constructed from these questions and used to determine the ratio of violence to property crime as well as violence to drugselling activity. Program Evaluation Findings and Results An examination of Chicago Police Department data, verified by EWDS, indicated that the target, Little Village, compared with six other high- gang-violence areas, mainly Hispanic/Latino communities, had the lowest level of increase of violent gang incidents over a 4-year project period, compared with a 4-year preproject period. The lowest increase in gang incidents, for a combination of gang homicides, aggravated batteries, and aggravated assaults, occurred in Little Village. Homicide, however, in this combination, did increase compared with several of the other areas. The Little Village police beat in District 10 also underwent the lowest rate of increase in violent gang incidents for the numbers of serious gang offenders, and the second lowest rate in increase in Hispanic/Latino serious violent gang offenders, compared with the other six areas over the same 4-year program period. The Gang Violence Reduction Project targeted 17-to-24-year-olds and, to some extent, gang youth 16 years of age and under. The program-targeted age group experienced the lowest rate of increase in gang incidents compared with other districts in the sample. The group 16 years of age and under in Little Village also achieved the lowest rate of increase in gang incidents compared with other districts in the 4-year project period. The percentage increase in gang incidents for older young adults 25 years of age and over was substantial, but the number involved was relatively small. The evaluation was also able to distinguish incidents committed by targeted and nontargeted gangs in Little Village. The Latin Kings and the Two Six were still responsible for the preponderance of serious violent gang crimes in the area, but generally experienced a smaller combined rate of increase in number of offenders involved in gang homicides, aggravated batteries, and aggravated assaults, compared with the other Hispanic/Latino and African-American gangs in the areas that were not targeted. However, for targeted gangs, the rate of increase of other types of gang crime, particularly drug arrests, which were mainly for possession of marijuana and cocaine, was almost three times more than the increase for the other Hispanic/Latino and African-American gangs in Little Village. As the project targets Little Village and the gangs in that area are Hispanic/Latino and African-American, there are no statistics for Caucasian and Asian gangs. Police arrest histories provided a different set of outcome variables, perhaps more reliable with respect to their impact on the project than self- reports of criminal behavior. The results, contrary to self-reports, indicated that, compared with the 3-year preprogram period, the average number of arrests for all crimes, as well as for violent, property, and drug crimes, increased over the course of the 3-year program period. This was true especially for the two gangs and specifically for those gang members under 19 years of age. However, individuals 19 and older experienced a reduction in total crime, violence, and property crime at statistically significant levels, with some increase in drug-dealing arrests that was not statistically significant. Individuals served by youth workers only (and usually with fewer prior arrests than youth served by a combination of different types of workers) also appeared to do better for some of the categories of crime. Those youth 19 and older served in the coordinated approach experienced a reduction in violent crime activity. Within the project period, i.e., comparing the third to the first program year, results show that Latin King members and individuals 19 and older, as well as those in the coordinated service group, experienced reductions in the average number of arrests for total crime, violent crime, and property crime. In many instances, these reductions were statistically significant. There were also reductions for some categories of crime -- particularly property crime -- among those members of the Two Six gang and individuals under 19 when the project began. Further, for some subgroups and some crime categories, those in the gang-worker-only (youth worker) group did as well or better than those receiving coordinated services. The results pertaining to arrests for drug-related offenses were mixed. The Latin Kings experienced a small increase in the average number of arrests for these crimes, but there was no change for the Two Six. A series of multiple regression models was constructed to determine which sets of variables were most promising under controlled conditions in predicting a reduction in total arrests and arrests for violent crime, including murder, aggravated battery, simple battery, aggravated assault, simple assault, armed robbery, and robbery, for the project and no-service control groups. Those in the project group who were 19 years of age and older had fewer arrests in the 3-year project period compared with project participants who were under 19 years and those of all ages in the no-service control group. The model, which accounted for 12 percent of the variance in total arrests over the 3-year project period, included the coordinated service variable. Youth receiving coordinated services who had more prior arrests had fewer arrests in the 3-year project period than those in the coordinated services group with a smaller number of prior arrests, as well as project cases in other types of project service groups and those in the no-service control group. The best model for project cases only with respect to the variance in total arrests over the 3-year project period was the service variable, which measured service from community youth workers only. This model accounted for 18 percent of variance in total arrests. The coefficient for the service variable indicated that those who received services only from community youth workers had fewer arrests in the 3-year period compared with all those not in this group. The project was apparently most effective in assisting older youth to significantly reduce their criminal activities--especially violence--more quickly than would have been the case if no project services were provided. The project did not appear to be effective with younger youth. Possibly, an insufficient project focus on younger youth and involvement with schools, which was more important for younger than older youth, may partially account for this lack of positive project effect. Approximately 100 residents in the high-gang-violence blocks and representatives of 50 local community groups or agencies were interviewed in both Little Village and Pilsen (the most similar comparison community) at Time 1 and Time 2, two years apart during the project period. Comparisons were made between the views of residents of Little Village and of Pilsen, and also between residents living in the Latin King and Two Six territories. Residents in Little Village, especially those in Latin King territory, reported significantly greater improvement in community conditions, that is, their perceptions about the scope, severity, and their own experiences related to the gang problems were more positive than those of residents living in Pilsen or Two Six territory. Between Time 1 and Time 2, significantly more Little Village residents thought the community was better and fewer thought the community was worse. More residents perceived an increase in safety. There was less fear of walking the streets of the neighborhood; decreased worry about being a victim of crime, gang and nongang; and a reduction in gang violence and gang property crime. Pilsen residents perceived reductions in concerns about safety and crime, but these changes were for the most part not statistically significant. These findings are consistent with those obtained from individual interviews and self-reports and from the aggregate-level police data comparisons. It is likely that residents' views in Little Village, and especially Latin King territory, reflected reduced levels of gang crime actually occurring, a response in large measure to the presence of the Gang Violence Reduction Project. ------------------------ Chapter 6 IOWA Batterer's Education Program The evaluation research presented in this report was conducted by Richard Moore, Administrator of the Statistical Analysis Center, and Terry Hudik, Justice System Analyst of the Iowa Criminal and Juvenile Justice Planning. The Byrne Grant Program has supported the Batterer's Education Program (BEP) by funding the eight District Departments of Correctional Services to either hire or contract program coordinators and facilitators. Additionally, Byrne grant funds have supported the Department of Corrections (DOC) training programs for facilitators for the BEP. This evaluation of the BEP was requested and supported by DOC, in part with funding from the Byrne Grant Program. Program Overview Iowa Code section 708.2A(4) requires that all persons convicted of domestic abuse assault must be sentenced to at least 2 days in jail and ordered by the court to complete a BEP. The BEP is designed to teach batterers how to stop using controlling and abusive behavior in relationships with their partners; the program promotes the philosophy that there is no justification for behavior that includes physical, emotional, and psychological abuse. The Byrne Grant Program has supported this program by providing funding to the eight District Departments of Correctional Services (DCS) to either hire or contract for program coordinators and facilitators. During the last 48 months, all DCS district offices have received funding for development or enhancement of programs. FY 1996 funding is supporting six of the districts. Two districts have reached the limit for Federal funding and requested State support for their programs. As a result of program evaluation demonstrating performance and need, their requests have been funded. Additionally, Byrne funds have supported DOC detraining programs for facilitators for the BEP. Because of the increased demand for additional programs, ongoing training and refresher courses have been needed during the past few years. In FY 1994, 2,749 offenders were ordered by the court to attend BEPs; in FY 1995, the number increased to 3,483. Goals and Objectives The goal of the BEP is to eliminate controlling and abusive behavior by abusers toward their partners. The objectives of the BEP are to: o Protect and keep victims safe. o Hold batterers accountable so they can effect personal change (learn alternatives to coercive, dominating, and violent behavior). o Reduce physical, emotional, and psychological abuse. o Support a community response to end domestic violence. Program Activities/Components The BEP operates statewide through the eight DCS districts. Judicial districts contract with agencies in their areas to provide program services for offenders in 1 or more counties, so they are available in all 99 counties. Service providers include community colleges, mental health centers, domestic violence shelters, DOC services, and a variety of social service agencies. The most common contracts are for program coordinators and facilitators because program guidelines require that classes be conducted by two trained and certified facilitators (one male and one female). Many of the contracts require personnel to contact the victim to ensure his or her safety, provide information about the BEP, and refer the victim for services to existing agencies. Through classroom and other group activities, BEP participants experience a curriculum designed to eliminate the use of controlling and abusive behavior in their relationships with their partners. Program components include: o An intake interview with a BEP staff person. o Attendance at an orientation session. o Weekly 2-hour group sessions for 16-24 consecutive weeks. o Satisfactory completion of homework. o Payment in full for the course. (Payment is set on a sliding scale according to income and ranges from $2 to $36 per session.) Performance Measures and Evaluation Methods Multiple methodologies were used in conducting this research, including an analysis of existing laws as they related to the BEP, an analysis of BEP participation and completion data as maintained by DOC, a review of the standards formulated by DOC for the individual BEP programs, survey instruments, onsite record searches, and interviews. The first task undertaken was the analysis of existing laws relating to the BEP, including both statutory and case law. From this analysis, a "Domestic Abuse Legal Model" was formulated that described how the criminal justice system was apparently designed to function in domestic abuse cases, based on existing laws. Using the legal model as a foundation, a survey instrument was developed and sent to all BEP coordinators for completion. The survey sought both to identify those counties in which the domestic abuse processing procedures appeared to deviate from the legal model and to quantify the degree of deviation. When the initial survey did not produce the desired level of information, a second survey instrument was developed with the assistance of DOC and was then distributed to the BEP coordinators. The combined results of the two surveys produced sufficient information to proceed with the research effort. The data from the BEP coordinators were used as a basis to select counties for further study. The counties were divided into four groups according to population, and from each group three counties not being served by a BEP or showing higher degrees of deviation from the legal model were chosen for onsite visits. During the visits to the selected counties, project staff examined the records of the district court and, from those records, extracted data regarding domestic abuse and certain other cases. Domestic abuse cases were reviewed for initial charges made and by whom they were made, the disposition of the case, and, in the event of a conviction, the sanctions imposed by the court. Other cases, such as disorderly conduct, public intoxication, assault that was not domestic abuse, harassment, etc., were reviewed in an attempt to determine whether these cases involved instances of domestic abuse, but were not filed as such, and who filed them. Pro se domestic abuse charges were also examined to determine the extent to which these cases resulted in the alleged offender being ordered to participate in the BEP. The data were then collected into a database and subjected to various quantitative and qualitative analyses. Project staff also interviewed criminal justice officials, such as the county attorney or assistant who prosecuted the majority of the domestic abuse cases, law enforcement administrators, and judges, to determine their policies regarding the arrest, charging, prosecution, and sentencing of domestic-abuse-related cases. Staff also determined at what point in the arrest, prosecution, and adjudication process offenders were referred to the BEP for each location examined. Phase two of the research project concentrated on compiling and analyzing data contained in the DOC computerized BEP database, the computerized criminal history (CCH) records maintained by the Iowa Department of Public Safety, and the court records maintained by the various clerks of the Iowa District Court. Analyses of the BEP database were conducted to identify certain characteristics of offenders involved with the BEP and to determine BEP completion rates. This same information, together with the data collected from CCH and court records, was used in assessing the extent to which BEP participants were charged with domestic assault following their involvement with the program. BEP Completion Data For purposes of these analyses, completion of BEP was defined as having successfully completed the prescribed BEP course of study, as defined and implemented by the organization conducting the individual program that the client attended. If a client was terminated from the program for cause and subsequently completed the program, he or she was considered to have successfully completed the BEP. Nonsuccessful completion was defined as those BEP clients who did not successfully complete the prescribed BEP course of study as defined and implemented by the organization conducting the individual program that the client attended. In this study, selected differences among some of the study variables were examined to determine whether they were statistically significant. The goal of the analysis was to further examine program completion rates by controlling for the extent to which they were affected by selected characteristics of BEP programs and participants, such as sex, race, age, judicial district, length of program, program delivery method (DOC or private), and adherence to legal model. The procedures used in the study to determine statistically significant differences were conducted to allow for at least a 95 percent probability that if a given difference between study variables is identified as statistically significant, it is due to a relationship between those variables rather than due to chance. The lack of statistical significance does not mean that an observed difference between two variables has been proven to be due to chance. Rather, it would be more appropriate to say that a relationship between the variables could not be verified--it may or may not exist. BEP Recidivism Data To further assess the effectiveness of the BEP, an examination of how the successful completion of the program has an impact on the recidivism of the program's participants was conducted. Current Iowa law requires all persons convicted of domestic assault to enroll in a BEP. Such a requirement precludes a comparison of the recidivism of domestic assault perpetrators who participate in a BEP with the recidivism of domestic assault perpetrators who do not participate in any BEP. Given the data collected by DOC from the BEP providers, however, it was possible to identify two groups of offenders for whom a comparison of information on recidivism might lend itself to the examination of BEP effectiveness: those who successfully complete the BEP program and those who are terminated from the program and do not successfully complete it. Before attempting to determine the domestic abuse recidivism rate of these individuals, an indicator of recidivism had to be chosen. The BEP process was reviewed, and it was determined that the indicator for this study would be the rearrest and prosecution of an offender on domestic assault charges after a preliminary hearing by the court. This evaluation attempts to answer the following question regarding recidivism: Does the observed overall relationship between completion of the BEP and rearrest hold equally true for participants of different ages or for programs of different lengths? The recidivism data are restricted to assessments of the extent to which the overall observed difference between the recidivism rate of persons who complete the BEP and those who are terminated from the BEP is significant when controlling for certain BEP participants or program characteristics, such as sex, race, age, judicial district, length of program, adherence to legal model, and type of program. Program Evaluation Findings and Results Analyses of the data collected in this evaluation effort did indicate that there was a statistically significant difference (though small) between the rearrest rates of BEP participants who completed the program and those who were terminated from the program: Persons completing the BEP program were somewhat less likely to be rearrested for domestic abuse assault than were persons who had been terminated from the program. Program Completion Perhaps one of the more informative, though simplistic, findings of this research is that about 40 percent of all State Fiscal Year 1994 offenders participating in BEP did not complete the program. When BEP completion rates were examined and controlled for certain program and offender characteristics, it was found that not all types of offenders or program approaches experienced the same completion rate: o African-American participants were less likely to complete the BEP than were Caucasian participants. o 17- to 21-year-olds were less likely to complete the BEP than were participants in other age groups. o 16-week BEP programs had higher completion rates than did 24-week programs. o Offenders were more likely to complete a BEP in areas where justice system officials were perceived as supportive of the intent of Iowa's domestic abuse laws and BEPs. o Women and men completed the BEP at about the same rate. o Completion rates were similar in BEPs staffed by DCS employees and those staffed by contractual service providers. Recidivism Rates Individuals completing the BEP were rearrested for domestic assault at a somewhat lower rate (15 percent) than those who did not complete the program (20 percent). It is important to note that not all types of offenders or program approaches experienced the same difference in rearrest rates of those who completed the BEP and those who were terminated from the program. o Males who completed the BEP had lower rearrest rates than those who did not, but this was not true for females. o Caucasians who completed the BEP had lower rearrest rates than those who did not, but this was not true for African-Americans. o Participants aged 22-26 who completed the BEP had lower rearrest rates than those who did not, but this was not true for other age groups. o Offenders who completed BEP programs provided through contract service providers had lower rearrest rates than those who did not complete the program, but this was not true for programs provided directly by DCS staff. o Lower rearrest rates were found for BEP participants who completed the program in areas where justice system officials were perceived to be supportive of Iowa's domestic abuse laws, but not for participants in areas with officials who were not. o The length of the program (16 or 24 weeks) was not related to the finding that those who completed the BEP had lower rearrest rates than those who did not complete the program. Uniformity of BEP Policies and Practices BEP program policies and practices were observed to vary, sometimes widely, in their formal or informal handling of attendance issues, offender disruption and nonparticipation, termination of the offender from the program, and readmission to the program. When added to the varied ways in which other system components respond to domestic violence, such differences among BEPs may contribute to concerns over the purpose and effectiveness of the program. ------------------------ Chapter 7 NEW YORK Drug Treatment Alternative to Prison Program The evaluation research presented in this report was conducted by Douglas Young, Ph.D., Senior Research Associate, Vera Institute of Justice. In 1992, New York State, noting the promising results of the Brooklyn Prosecutor's program, supported expansion of the Drug Treatment Alternative to Prison (DTAP) Program to two other boroughs in New York City--Queens and Manhattan, and to the special prosecutor. Edward Byrne Memorial formula grant funds were allocated, and the Special Prosecutor Program was initiated in Brooklyn in 1992, in Queens in 1993, and in Manhattan in 1994. Technical assistance to the sites was provided by the New York State Division of Criminal Justice Services and the Office of Alcoholism and Substance Abuse Services. A private agency, the Legal Action Center, was enlisted to assist with contract issues and with issues of confidentiality, and the Vera Institute of Justice was contracted to help develop a program evaluation. Program Overview DTAP was designed and implemented by the Kings County (Brooklyn) District Attorney in 1990, in response to increased pressure on public budgets. The program offers nonviolent repeat felony defendants, who typically face 3 to 6 years in prison under New York State's mandatory sentencing laws, the option of completing long-term residential drug treatment in lieu of incarceration. If they complete treatment, the charges are dismissed; if they fail to do so, they face prison terms of at least 18 months. The prospect of prison is enhanced by special enforcement teams that apprehend and return absconders. The result is an alternative, cost- effective criminal justice initiative that maintains community safety while addressing the underlying causes of drug-related crime. The program, a partnership of criminal justice and drug treatment, has gained cooperation from initially resistant agencies within the criminal justice system. The public defender was uncertain of DTAP because it is a prosecutor-led initiative, but the resistance was overcome by the inclusion of the public defender's office in the planning and implementation phases of the project. The project expanded to include the Special Narcotics Prosecutor (a citywide office) in 1992, Queens District Attorney in 1993, and Manhattan District Attorney in 1994. By mid-1996, 1,300 drug offenders had entered the program through 4 sites, and the number of community-based treatment providers serving DTAP participants grew from 2 in 1990 to 39 in 1996. Goals and Objectives The short-term goals were to lower costs by diverting prison-bound defendants to less expensive treatment beds and to retain relatively large numbers of them in treatment. The long-term goals were to reduce drug use, reduce criminal recidivism, and improve the vocational and social capabilities of program participants. The objectives to achieve these goals were to: o Identify at least 300 second-felony defendants judged eligible for admission to DTAP treatment by the prosecutor and treatment staff. o Identify an equal number of residential treatment slots in privately run therapeutic communities (TCs) that could be made available to DTAP participants on a priority basis. o Collaborate with prosecutors and treatment providers to ensure appropriate, efficient selection of participants and reliable reporting of their progress. o Implement enforcement mechanisms, including special warrant squads, to ensure the rapid apprehension of absconders. o Maintain a program completion ratio of at least 60 percent (i.e., at least 60 percent of all those admitted to DTAP complete treatment and have their charges dismissed). o Maintain a criminal recidivism rate of no more than 25 percent for graduates (i.e., no more than 25 percent of graduates are rearrested within 1 year of completing the program). o Evaluate the impact of the program in light of these goals and objectives. Program Activities/Components In 1992, New York State, noting the promising results of the Brooklyn Prosecutor's program, supported the expansion of DTAP into two other boroughs in New York City, Queens and Manhattan, and to the Special Prosecutor. Edward Byrne Memorial Formula Grant Funds were allocated, and the Special Prosecutor program was initiated in 1992, the Queens program in 1993, and the Manhattan one in 1994. Technical assistance was provided by the State Division of Criminal Justice Services and the Office of Alcoholism and Substance Abuse Services. A private agency, the Legal Action Center, was enlisted to assist with confidentiality and contractual issues, and the Vera Institute of Justice (Vera) was contracted to help develop a program evaluation. Most of the core elements of the Brooklyn model were adopted by the three new sites. All DTAP participants are nonviolent defendants charged with felonies who, if convicted, would receive mandatory prison terms as second-felony offenders. All participants are determined to be in need of, and motivated to participate in, drug treatment. All agree to participate in a residential treatment program that lasts 14 to 24 months in lieu of prison. All programs serving DTAP are run by private, community-based agencies employing the TC treatment model. If participants complete the treatment program, charges against them are dismissed. Failure to do so is met with a prison sentence at least as long as the one the defendant would serve in the absence of DTAP. Each program employs a special enforcement team of investigators who verify participants' community contacts prior to treatment and who search for and return absconders. The programs do differ in some important respects. The Brooklyn District Attorney reviews all cases appearing in the borough's drug courts, soliciting program applications from defendants with prior nonviolent felonies charged with B felony narcotics sales in strong "buy and bust" cases. The other programs inform the defense bar about the program and its admission criteria and take applications initiated by the defense in criminal court. The legal mechanism underlying the Manhattan, Queens, and Special Narcotics programs is deferred sentencing, whereby DTAP defendants must plead to C felony charges before entering the program. In contrast, Brooklyn defers prosecution on the DTAP defendant's case; individuals who abscond from these programs face minimum prison terms ranging from 18 months to 3 1/2 years. One other difference concerns flexibility in enforcing the plea policy: Second referrals to treatment are considered in the newer programs, but are not made in the Brooklyn program. All treatment is delivered in long-term residential facilities called therapeutic communities. Services offered augment drug treatment and include HIV education, counseling and testing; onsite medical care; vocational training; and assistance in finding employment and housing. The most frequently used DTAP treatment providers have been Daytop Village and Veritas, which have each admitted more than 70 participants from 1 or more of the 4 sites. Other TCs admitting more than 10 participants include Odyssey House, Samaritan Village, Damon House, Phoenix House, Promesa House, and Inward House. Performance Measures and Evaluation Methods The DTAP programs at each prosecutor site collect data on program performance that are then compiled and aggregated by the State. Data include the number of defendants screened for admission, accepted in the program, and placed in treatment; the number who abscond, are discharged, or otherwise leave treatment early; and, of those leaving treatment, the number who remain at large. At any given time, each program must be able to report the number of DTAP participants currently occupying beds at each private treatment provider used by the program. Data on program completions are also compiled. Additionally, the State's substance abuse office, together with the District Attorney sites, maintains a count of the number of treatment slots designated for each DTAP, their location (the provider agency), and the number of such slots currently available. A summary of research methods follows. A. Program admission, termination, and disposition data analyzed for this report cover the period from December 1992, when the first participants were admitted to the expansion programs, through mid-June 1996. Data were compiled by DTAP staff; Vera then checked and validated retention and termination data at treatment providers' sites for expansion units. Through 1995, Vera interviewed most DTAP participants entering treatment from the expansion programs within a week or so of their admission. A total of 217 intake interviews were conducted, representing 77 percent of all expansion site admissions during this period. The core assessment tool used in the interview, the Addiction Severity Index, is designed to assess respondents' history and extent of current problems in seven areas: employment, alcohol use, illegal activity, drug use, medical conditions, psychiatric conditions, and family relations. The interview also includes a motivational measure that has been used to predict retention in TCs, an experimental scale developed by Vera researchers to assess perceptions of legal coercion, and a questionnaire on reasons for entering treatment. The primary statistical techniques employed are one-way analyses of variance and contingency table analyses. In each case, a test of significance is performed. (The technical details of results are available from the Vera Institute of Justice upon request.) Any conclusions drawn from this research must be regarded as tentative given the absence of an experimental design with control or comparison groups. (See Diverting Drug Offenders to Treatment, Year Two Report on DTAP Expansion, Vera Institute of Justice.) B. Vera is comparing retention among Brooklyn DTAP participants and a statistically matched group of probationers, parolees, and others ordered by the court to attend the same treatment facilities DTAP uses. Other analyses focus on the effectiveness of specific coercive strategies, such as closely monitoring the progress of treatment, using special enforcement squads, and imposing severe penalties for dropping out of the program. (See Retaining Offenders in Mandatory Drug Treatment Programs: The Role of Perceived Legal Pressure, Summary, Vera Institute of Justice.) The quasi-experimental research design is devised to test two central hypotheses: o DTAP participants stay in treatment for longer periods and show greater retention rates compared with a matched sample of other legally mandated clients attending the same treatment programs. o Legal coercion, as measured by the Perceived Legal Pressure scale, significantly and independently predicts the retention of all legally mandated clients in the treatment program. C. Vera, the New York City Criminal Justice Agency, and other researchers are expanding their investigation. The 5-year collaborative study will measure the impact of DTAP and compare its costs and benefits with those associated with traditional court processing of repeat felony drug offenders. Vera's primary contribution to this research is an exhaustive study of treatment retention comparing DTAP participants with all other criminal-justice-involved clients who attend the treatment facilities DTAP uses. A longitudinal quasi-experimental design is being used for three substudies: impact, legal coercion, and benefit-cost. The retention, posttreatment outcomes, and program costs are being compared for a sample of 200 DTAP offenders and a sample of 150 offenders with similar charges, criminal records, and demographics, as well as a desire for drug treatment upon release from prison during the same period. A new sample of 200 clients of the same treatment programs used by DTAP participants, but referred from other criminal justice sources, is being compared with DTAP participants for retention and legal coercion analyses. Impact Evaluation Substudy. Posttreatment outcomes of DTAP participants (experimental) are compared with a matched sample of drug offenders who are prosecuted and sentenced to prison terms. Subjects were interviewed before they entered treatment or prison and upon reentry into the community. Followup interviews are to be conducted at 6 months and at 1-year after completion of treatment. The experimental sample consists of 150 DTAP participants who successfully completed TC treatments and 50 who dropped out or were expelled and were subsequently convicted and sentenced to prison. The inclusion of failures in this sample corrects for potential selection bias inherent in using only program completers for comparison. The dropout subjects were selected from among those whose dropout rates and subsequent lengths of prison sentence resulted in a 1- year at-risk time closest to those of the prospective DTAP samples. Like the DTAP completers, the dropouts were interviewed in the community following their release from prison. The recidivism analyses are supplemented by data from official records for all DTAP participants who were admitted prior to June 1995 and failed to complete treatment. Recidivism rates for this group will be compared with the 150 DTAP completers and 150 prisoners who are members of the comparison group (discussed shortly) to assess whether participation in treatment for a substantial period of time before dropping out and receiving a prison term has a favorable effect on posttreatment criminal activity. The comparison group consists of 150 drug offenders convicted of a second drug felony offense and sentenced to prison from the other boroughs of New York City. They are matched to the experimental group with respect to age, sex, race, penal law conviction charge, length of sentence, criminal history, history of drug abuse, and motivation to seek treatment. A brief screening questionnaire is administered to determine eligibility for the study. To further ensure similarity of the DTAP and comparison groups, an aggregate or frequency distribution matching procedure is used. Aggregate matching involves selecting individuals from the comparison population in such a way as to make the overall distributions of the experimental and comparison groups correspond on the key matching variables. After the potential comparison sample member is initially identified, it is determined whether the individual falls within the distributions observed for the DTAP sample. Those who fall outside the distribution on any of the matching variables or who have matching variables that are fully represented in the comparison sample are excluded from the subject pool. The final two matching criteria, drug use/treatment history and motivation for treatment, are determined from the screening interviews conducted at the prison. As final insurance against nonequivalence, statistical controls in data analysis are employed to adjust for unforeseen or unavoidable intergroup differences that persist after the full samples have been recruited. Legal Coercion Substudy. The design established a new comparison sample of 200 individuals that is more representative of the population of legally mandated treatment clients in New York City. Approximately 100 of these are Treatment Alternative to Street Crime (TASC) clients, who will be treated as a discrete sample in several comparative statistical analyses. The 150 DTAP participants constituting the prospective experimental sample in the impact substudy also serve as the experimental sample in this study. In addition, the 162 subjects who participated in the National Institute of Justice (NIJ) study continue to be tracked under this research. Interviews with 40 supervising criminal justice agents (TASC counselors, probation officers, and parole officers) are also part of this substudy. Benefit-Cost Substudy. Analyses of the net social benefits of DTAP participation will use interview data collected from the DTAP and comparison group sample described in the impact substudy. The analysis of criminal justice system benefits will be based in part upon a different comparison group. This sample will consist of the first 150 paper-eligible candidates for DTAP who either refused the program or were screened out for reasons other than the weakness of the case against them. Court processing and associated data from this sample will yield more accurate estimates of system costs, because it can be assumed that the conviction and imprisonment rates for this group will be equal to those which DTAP participants would have experienced but for their diversion from the program. The system cost attributable to criminal recidivism will be based upon the impact study comparison group. Official records will be used to estimate the savings in court processing and sentencing costs associated with the hypothesized lower recidivism rates of the DTAP sample. (Research in progress under a grant from the National Institute on Drug Abuse, Vera Institute of Justice.) D. Review of official arrest records by the Brooklyn DTAP office. Program Evaluation Findings and Results A. Funded by New York State, Vera's research on DTAP provides evidence of the model's achievements: DTAP participants stay in treatment longer and have higher completion rates compared with people in similar programs; they are unlikely to commit crimes during treatment-- to date, there have been no arrests for violent crimes among participants; and early data indicate low rates of recidivism among DTAP graduates. Most findings are based on data gathered from the sites and treatment providers through June 15, 1996. By that date, 1,035 defendants had been admitted to treatment; 500 participants were in the program; and the number of treatment providers was up to 39. Through October 15, 1996, 18 percent of DTAP participants were arrested for a crime or for absconding after admitting to a crime, none for violent offenses. Overall, 63 percent of those admitted to the program have graduated or remain in treatment. Of all the participants, 89 percent were still in treatment at least 3 months after admission, 75 percent stayed at least 6 months, and 63 percent stayed at least 1 year. These rates are 1 1/2 to 4 times those reported by comparable treatment programs. Virtually all treatment experts agree that retention leads to decreased drug use and criminal recidivism. (See Diverting Drug Offenders to Treatment, Year Two Report on DTAP Expansion, Vera Institute of Justice.) B. Retention rates for DTAP participants and the comparison sample were found to be similar in multivariate analyses, disproving the study's first hypothesis. DTAP retention was much higher than retention rates reported in the general literature for long-term residential care; however, the comparison sample's rates were equally high. At 3 months after admission the DTAP sample was 77 percent while the comparison sample's rate was 69 percent--not a significant difference. Little in the overall sample description accounts for such high retention. It is possible that the treatment programs studied are especially effective at retaining clients; another plausible explanation is that these individuals make up a subgroup of all criminal-justice-involved treatment clients who experience a relatively high level of legal pressure. Yet another possible explanation is that the high retention in both groups reflects a generalization or diffusion of DTAP practices and effects to other clients who attend the same TCs as DTAP participants. The strong record of apprehension and enforcement of absconders may serve as an example for all legally mandated clients, thereby enhancing the credibility of threats made by judges, TASC case managers, and probation/parole officers. Diffusion could work both ways - - the presence of the case managers, for example, may have a favorable impact on DTAP and other participants. There is no way to confirm the explanation of this diffusion given the mixing of clients in these treatment programs. The study's second hypothesis, that perceived legal pressure would be an important predictor of treatment retention, was confirmed by statistical analyses that showed the unique impact of Perceived Legal Pressure (PLP) scale scores on different measures of retention. Multivariate analyses looked at the impact of the PLP (a questionnaire Vera designed with NIJ funding) and other predictors (e.g., age, and drug, psychological, and medical history) on four different retention outcomes. PLP score was the most powerful predictor in three of these analyses and among the most powerful in the fourth. (See Retaining Offenders in Mandatory Drug Treatment Programs: The Role of Perceived Legal Pressure, Summary, Vera Institute of Justice.) C. Vera, the New York City Criminal Justice Agency, and others. Research in progress. D. Based on a review of official arrest records, researchers at the Brooklyn DTAP office have found that for 2 years after completing the program, their graduates are half as likely to be arrested as offenders with similar criminal records. ------------------------ Appendix A Identifying Effective Criminal Justice Programs: Guidelines and Criteria for the Nomination of Effective Programs The Bureau of Justice Assistance (BJA) has created the Intensive Program Evaluation (IPE) Initiative to respond to the Attorney General's charge to "find out what works and spread the word." This new initiative establishes a mechanism to validate the effectiveness of criminal justice programs based on published criteria, including evaluation results, and to disseminate information about approved effective programs through Department of Justice networks directly to practitioners. The approval of effective programs for dissemination relies on the evaluations to be conducted by the Program Effectiveness Review Panel. This Panel will provide independence and rigor in its recommendations to the Director. In the development of the guidelines, BJA relied heavily upon the Panel, as well as upon past activities of the National Institute of Justice, to identify exemplary programs. The program objectives are to: o Enhance the ability of State and local agencies to generate and use evaluation results for strategy development, program improvement, and effective program identification. o Identify and document useful approaches to designing and conducting evaluations at State and local levels. BJA needs to identify effective State and local criminal justice programs, practices, and products as part of broader efforts at the national level to improve the criminal justice system by disseminating useful program information to policymakers and practitioners. It is an approach that might be called "leading by example." Through this approach, information on successful programs will be disseminated to the field in a credible and timely fashion. The products from this initiative are the results from the 56 laboratories (50 States and 6 territories) put in place under the Byrne Formula Grant Program. Program effectiveness guidelines and criteria are presented here for use in submitting potential programs, as well as for the Program Effectiveness Review Panel in its review of nominated programs. The Panel reviews the programs and submits its recommendations to the Director. Once the effective programs have been approved, BJA bulletins will be published. I. Abstract The abstract should be a 1-page description (200-300 words) of the program that provides a concise statement of concrete, observable outcomes. The abstract should briefly describe the following aspects of the program: (1) goals, (2) purposes and needs addressed, (3) method of operation, (4) audience, and (5) expected result(s). II. Basic Information Basic information should be approximately one page. A. Project Title Location Contact Person Give the title of the project (including any acronym or abbreviation), the name of the applicant agency, and the address and a daytime telephone number of a contact person within the applicant agency. B. Original Developer Applicant Agency Provide the name(s) and title(s) of those who originally developed the program. Describe the mission of the applicant agency. C. Years of Project Provide (1) date(s) developed, (2) date(s) operated, and (3) date(s) evaluated. D. Source(s) and Level(s) of Development and Dissemination Funding List sources of funding for the project and amounts by year. Categories of sources include Federal, State, local, and other. III. Description of Program Describe the program in approximately five to six pages. A. Background, Foundation, and Theoretical Framework Discuss briefly the history of how and why the program was developed. Present the theoretical or empirical framework upon which the program is based. B. Purposes and Needs Addressed/Problem Statement Describe the specific needs the program was designed to address. Needs should be linked to the target audience and special features of the program. C. Goals Provide a clear and concise statement of the program goals. Include only those goals that relate directly to claims of effectiveness. In the case of evaluation models designed to meet intermediate objectives, make the link to the ultimate purpose of the program. D. Objectives Objectives are the intermediate effects or results to be achieved by the program in pursuing its ultimate goal. Objectives measure the extent to which program goals are being accomplished. Identify appropriate objectives that logically flow from program goals. Objectives should be stated in terms of outcomes (expected effects or results). A distinction should be made between outputs (quantities produced) and effects/results. E. Intended Audience Identify the relevant demographic characteristics of the population for which the objectives are designed. F. Features: How the Program Operates Provide a complete description of how the program actually operates, identifying all features critical to its implementation. Include the following topics as they apply to the project: (1) scope (Does the project supplement or replace an existing program, or is it a component of a larger program?), (2) staff activities and staffing patterns, (3) staff development activities, (4) management activities, and (5) monitoring and evaluation procedures. G. Significance of Program Design Compared with Design of Similar Programs Describe the features of the program that distinguish it from similar programs. Discuss ways in which the program addresses special problems. Note innovative or unique features. IV. Potential for Replication Describe the potential for replicating the program in two to three pages. A. Settings and Participants (Development and Evaluation Sites) Briefly describe the community(ies) where the intervention was developed or field tested. Socioeconomic, ethnic, and geographic descriptions are appropriate. B. Replicable Components and Documentation Indicate which aspects of the program are appropriate for dissemination to other sites. If the program has developed support materials for dissemination, indicate the type of documentation available. C. User Requirements Describe the minimum requirements necessary for implementing the project at another site (e.g., special staff, facilities, staff training time). D. Costs (for Implementation and Operation) Present a brief explanation of the recurring and nonrecurring costs associated with adopting the project. Costs such as personnel costs, special equipment, and materials and supplies that are necessary for installing and/or maintaining the program at an adopting site should be discussed. Costs associated with the development of the original program should be excluded from this discussion. V. Evidence Provide evidence in approximately six to eight pages. A. Impact Statement(s) The specific evaluation methods used to determine the impact of the program should be explained. Succinctly state the major accomplishments of the program and its activities. Include a brief description of the type of evidence used to support the fact that an impact was determined and a description of the nature of the change that was demonstrated. Generally, an impact statement includes (1) the target group for which results are available, (2) the nature of the change, (3) the process used for measuring the impact of the program, and (4) the standards by which to judge whether the gains achieved are significant. A clear impact statement is critical because the panel judges the adequacy of evidence based on the claim. Further, the statement identifies the project objectives/outcomes that will be approved for dissemination (i.e., only those objectives/outcomes that are reflected in the impact statement(s) and supported by convincing evidence will be approved). B. Description of Methodology 1. Design An evaluation design usually addresses three factors: (1) the timing of data collection (e.g., pretests and posttests or different points in a time series), (2) the groups involved (e.g., a group receiving the program and a comparison group receiving an alternative program), and (3) the way in which a standard of comparison will be determined (e.g., a treatment group's gain or change will be compared with national or State benchmarks). Describe the type of design used for each claim and the reason for the choice. Address any assumptions or problems inherent in the research design that was used. 2. Sample The discussion of sampling procedures should answer four questions: (1) Who participated in the study? (2) How was the sample selected? (3) How many participants were included in the final sample? (4) How representative is the sample of the target population and program participants as a whole? 3. Instruments and Procedures This section should describe the instruments and/or procedures and how each assessment technique relates to the outcomes. Provide sufficient information so that a judgment can be made about the technical strength and appropriate use of the measure (e.g., validity, reliability, levels, subscales). It is especially important to describe validity and reliability procedures for project-developed instruments; in such cases, the procedures for instrument development and field testing also should be explained. 4. Data Collection Describe the procedures used to select and train testers and the actual strategies used to assure quality control during data collection. Indicate the periods of data collection, the persons responsible for supervising the data collection, and the scoring and data summary procedures. It is especially important to describe in detail the data collection and quality control procedures for qualitative evaluations. 5. Data Analysis If data are quantitative in nature, indicate the statistical technique(s) and levels of significance used in the analysis. If data are qualitative in nature, describe the procedures used to code and categorize or reduce information for summary purposes. Describe ways in which linkages were made across data elements to draw and verify conclusions. C. Description of Results Present detailed results of analyses in table or chart form, if appropriate. Sufficient detail should be provided for the reader to check conclusions independently. Also, summarize the results for the claims in narrative form, relating the specific outcomes to the accomplishment of goals. D. Summary of Supplementary Evidence Provide additional evidence that supports the results, including anecdotal information, perceptions of quality, and levels of satisfaction. Supplementary evidence can also be evidence of generalizability. E. Interpretation and Discussion of Results 1. Relationship Between Effect and Treatment Summarize the results of all data related to the claim that the treatment was effective. Link the results to specific features of the program design. 2. Control of Rival Hypotheses Provide evidence of program attribution, i.e., evidence that suggests that the effects can be attributed to the program and not to some other equally plausible factor. As appropriate to the design, show how the following alternative explanations can be eliminated from consideration: maturation, other treatments, historical factors, statistical regression, attrition, differential selection of groups, and testing. (Note: Sound evaluation design can control most rival hypotheses; however, other data may be used to show attribution of effects.) F. Significance of Results 1. Relationship of Results to Needs Demonstrate how the results obtained are important: How do the results meet the needs for which the project was designed? Establish the importance of the needs, and demonstrate that the results are broad enough and powerful enough to be viewed as important. 2. Comparison of Results With Results From Other Programs Compare the results with results of similar projects or national or statewide initiatives, if appropriate. ------------------------ Appendix B Sources for Further Information For more information on Improving the Nation's Criminal Justice: Findings and Results from State and Local Program Evaluations, contact: Bureau of Justice Assistance 810 Seventh Street NW. Washington, DC 20531 202-514-5947 Bureau of Justice Assistance Clearinghouse P.O. Box 6000 Rockville, MD 20849-6000 1-800-688-4252 301-519-5212 (Fax) Justice Research and Statistics Association 777 North Capitol Street NE., Suite 801 Washington, DC 20002 202-842-9330 ------------------------ Bureau of Justice Assistance Information General Information Callers may contact the U.S. Department of Justice Response Center for general information or specific needs, such as assistance in submitting grants applications and information on training. To contact the Response Center, call 1-800-421-6770 or write to 1100 Vermont Avenue NW., Washington, DC 20005. Indepth Information For more indepth information about BJA, its programs, and its funding opportunities, requesters can call the BJA Clearinghouse. The BJA Clearinghouse, a component of the National Criminal Justice Reference Service (NCJRS), shares BJA program information with State and local agencies and community groups across the country. Information specialists are available to provide reference and referral services, publication distribution, participation and support for conferences, and other networking and outreach activities. The Clearinghouse can be reached by: Mail P.O. Box 6000 Rockville, MD 20849-6000 Visit 2277 Research Boulevard Rockville, MD 20850 Telephone 1-800-688-4252 Monday through Friday 8:30 a.m. to 7 p.m. eastern time Fax 301-519-5212 Fax on Demand 1-800-688-4252 BJA Home Page http://www.ojp.usdoj.gov/BJA NCJRS World Wide Web http://www.ncjrs.org E-mail askncjrs@ncjrs.org JUSTINFO Newsletter E-mail to listproc@ncjrs.org Leave the subject line blank In the body of the message, type: subscribe justinfo [your name]