Title: Office of Research and Evaluation Solicitation for Evaluations of Bureau of Justice Assistance Discretionary Funds Projects--San Bernardino County Operation Nightlight Series: Solicitation Author: National Institute of Justice Published: February 2003 Subject: Funding resources 5 pages 10,000 bytes --------------------------- Figures, charts, forms, and tables are not included in this ASCII plain-text file. To view this document in its entirety, download the Adobe Acrobat graphic file available from this Web site. --------------------------- San Bernardino County Operation Nightlight Grant: 2000-DD-VX-0022 Grantee: San Bernardino County, California, Probation Department Grant Period: September 2000-Present Project Funding: About $1.8 million Contact: Michelle Scray (909-387-5785) Project Summary. The original project theory and logic is based on the documented success of the Boston Nightlight program. The target population is all juveniles placed on probation by the court. The project has enforcement and prevention services. The enforcement component pairs together five teams of police and probation officers who work in the field to conduct home visits to juveniles on probation, enforce curfew and truancy violations, and provide other enforcement activities. These are the IMPACT teams--Intervention and Management of Probationer Accountability through Compliance to Terms. The prevention component has IMPACT teams refer selected juvenile probationers and at-risk youth to "reward" activities (e.g., sporting events, fairs, etc.). Some of the youth also are exposed to the IMPACT teams' version of a "scared straight" program. Scope of Evaluation. The project goals are to (1) decrease crime committed by juveniles on probation and (2) reduce juvenile crime citywide. The evaluation should focus on these two goals, which are measurable through available data. Summary of Evaluability Assessment Activity. Activities to assess this project included the following: (1) reviewed grant proposal, grant progress reports, and other materials about the grant (e.g., BJA grant manager's memo, newspaper article); (2) discussed project with NIJ's Lois Mock, who had made a site visit in May 2001; (3) interviewed Chris Condon, grant coordinator, by phone; and (4) visited the site on October 30, 2002. The site visit involved meetings with the chief of police; police department's project coordinator; chief, deputy chief, and project coordinator of the probation department; head of the Justice Center (treatment services), and the local evaluator (University of California- San Bernardino). A 2-hour meeting also was held with three-fourths of the IMPACT team, and key site personel and Ed Zedlewski and Winnie Reed of NIJ participated in a telephone conference call. Finding. An outcome evaluation could be implemented with likely success: o Sufficient data are available to track project activities, and sample sizes are adequate. The local evaluator has made improvements in the agencies' data- collection schemes. o Local agency staff are very supportive of an evaluation. They want to know what works and why. o Initial data seem to indicate a degree of success in reducing juvenile crime. An evaluation would help inform the degree to which the drop in crime is due to the program's activities and other factors. An evaluation of this project would reinforce the importance of blending the police and probation cultures to deal with the problem of gangs and juvenile crime. The evaluation might also shed light on the respective influences of enforcement versus preventive programs in reducing juvenile crime. Analysis This project logic and theory is patterned after the Boston Nightlight program. While the Boston program was written about extensively and reported as a successful program, the exact components of what worked and why are not clear. Some literature also exists on the effectiveness of intensive probation supervision (Joan Petersilia and Susan Turner, Intensive Supervision for High- Risk Probationers: Findings From Three California Experiments, December 1990). However, this project targets juveniles. Additionally, the Urban Institute's evaluation of the Children At Risk program examined the importance of looking beyond treating the youth to treating the entire family in a holistic approach. San Bernardino's Operation Nighlight began in September 2000 with an $800,000 discretionary grant from BJA. The project goals are to decrease crime committed by juveniles on probation and reduce juvenile crime citywide. The target population is all juveniles in the city of San Bernardino who are placed on probation by the court. The number averages about 800 per year. This accounts for about 20 percent of the probation department's countywide juvenile probation population. The project has enforcement and prevention services. The enforcement component pairs together five teams of police and probation officers who work in the field to conduct home visits to juveniles on probation, enforce curfew and truancy violations, and provide other enforcement activities. These are the IMPACT teams--Intervention and Management of Probationer Accountability. The prevention component has IMPACT teams refer selected juvenile probationers and at-risk youth (i.e., those who have come to the attention of police but are not yet on probation) to "reward" activities (e.g., sporting events, fairs, etc.). Some of the youth also are exposed to the IMPACT teams' version of a "scared straight" program. The project has been fully implemented for at least 18 months. It also appears to be well managed and administered. The police and probation agency's respective project coordinators have been managing the project since its inception. They are both very enthusiastic and dedicated to the program and handpick the staff involved in the project. The project's logic seems sound. The interventions are directed at the target audience--juveniles on probation and their families. The interventions are not, however, distributed uniformly to this population. In other words, some probationers get more interventions than others, based on a determination of need by the teams. For example, once placed on formal probation (court supervision), each probationer is visited by the IMPACT team. The team reviews probation conditions with the probationer and his or her parent(s) or guardian(s). After this initial visit, the probationer may not be visited again unless merited. The main reason for this is that the teams want to maintain flexibility. They don't manage caseloads per se. All probationers are assigned to a general probation officer. This officer will likely have such a large caseload that he or she cannot do home visits or much direct supervision. The IMPACT teams fill this gap by conducting home visits to reinforce that "probation" is serious--it has teeth; failure has consequences. The IMPACT teams decide which probationers to visit based on three possibilities: they review case files and decide targets themselves, the general probation officers make a referral based on perceived need (e.g., the youth has missed an appointment or a treatment (counseling) session), or a patrol officer stops a probationer on the street (e.g., traffic stop) and calls for assistance because the youth is on probation. The prevention interventions are equally non-uniform. Probationers are frequently ordered into some type of service program (e.g., anger management, family counseling, community service, etc.) by the court, based on the probation officer's recommendation. Over and above this, the IMPACT team has some limited programs that they implement, for example, taking youths to sporting events and adventure camps as a reward and taking some youths to a prison or lockup to give them a "scared straight" experience. The criteria for selecting which probationers are exposed to these supplemental services is not in writing or well developed. In addition, some at-risk youth are mixed in with the probationers in attending these supplemental services. Output Performance Measures The police and probation departments have been collecting a variety of process measures related to program activities. These include, but are not limited to-- o New arrests. o New technical violations. o Home visits (attempted and completed). o Searches conducted during home visits. o Guns or drugs found during home visits. o Curfew violations. o Vehicle and pedestrian stops. o Enrollments in counseling. Local Evaluation Dr. Larry Gaines, University of California, San Bernardino, has a contract ($40,000) to conduct an evaluation of some of the project's activities. A summary of his plans follows: Effects on crime: Track juvenile crimes (e. g., status offenses, graffiti, violent crime by juveniles, misdemeanors, etc.) for several years before and after the program started. Compare findings in San Bernardino with four other cities (Highland, Rialto, Fontana, and Colton) in the county. Each city is served by the same probation department but they do not have the IMPACT teams. Displacement: Tracking the extent to which juvenile probationers from San Bernardino commit new crimes in surrounding cities and vice versa. A sample of probations will be compared using before and after data. Effects on juveniles: Compare data on program outcomes, such as new arrests or technical violations, through a random sample of 100 cases drawn from San Bernardino and three other "control" cities in the county. The San Bernardino sample must have received an orientation home visit and at least one other home visit by the IMPACT team. This analysis will not include any data on the prevention activities. Also, the samples are fairly small.