Title: Office of Research and Evaluation Solicitation for Evaluations of Bureau of Justice Assistance Discretionary Funds Projects--Building Bridges: A Correctional Option Program for Ex-Offenders Series: Solicitation Author: National Institute of Justice Published: February 2003 Subject: Funding resources 17 pages 32,000 bytes ------------------------------ Figures, charts, forms, and tables are not included in this ASCII plain-text file. To view this document in its entirety, download the Adobe Acrobat graphic file available from this Web site ----------------------------- Office for Victims of Crime Resource Center at 800-627-6872.Building Bridges: A Correctional Option Program for Ex-Offenders Grant: 2000-DD-VX-0051 Grantee: Chattanooga Endeavors, Inc. Grant Period: October 1, 2002-September 30, 2003 Project Funding: $282,000 Contact: Timothy Dempsey (423-266-1888) Project Summary. Chattanooga Endeavors, Inc., is a reentry program for ex-offenders. It recruits unemployed, adult felons from several sources, but an evaluation would likely be based on probationers and parolees. (Probationers would have served some jail or prison time.) Building Bridges is a 6-week program that focuses on interpersonal skills and maladaptive behavior, substance abuse, education, and work experience. There is up to 1 year of case management. The objective is to reduce criminal recidivism principally through employability. The program has not much changed from the grant submission. Analysis What do we know about projects like these? What could an evaluation add to what we know? Programs that identify the need for ex-offenders to become employed and that seek to provide ex-offenders with the means to overcome impediments to employment are not new, as exemplified through a cursory inspection of several National Institute of Justice Program Focus documents: o Chicago's Safer Foundation: A Road Back for Ex-Offenders (1998). "Founded in 1972, the Safer Foundation in Chicago is the largest community-based provider of employment services for ex-offenders in the United States Safer helps ex-offenders not only to find good jobs, but also to develop a mindset that helps to ensure they will remain employed and succeed in life." o Successful Job Placement for Ex-Offenders: The Center for Employment Opportunities (1998). "The Center for Employment Opportunities in New York City . . . provides day labor for participants, most of whom have been released . . . from boot camp. In addition to allowing the participants to earn a daily income, the work crews help the participants structure their lives. . . . The work crews are short-term means of achieving CEO's overall mission: placing ex-offenders in permanent, unsubsidized, full-time jobs that provide benefits. . . ." o Washington State's Corrections Clearinghouse: A Comprehensive Approach to Offender Employment (1999). The program provides " . . . some direct services (for example, teaching job readiness courses in prisons and contracting with community-based organizations . . . to provide job search assistance to ex-offenders), . . . brokering services from other agencies . . ., and . . . coordinating activities across agencies. . . ." o Texas Project RIO (1998). The program ". . . provides job preparation services to inmates while they are still incarcerated . . . helps prepare inmates for employment . . . developed a pool of more than 12,000 employers who have hired parolees referred by the program." o The Orange County, Florida, Educational and Vocational Program (1997). ". . . provides . . . intensive educational and vocational programs to most inmates in its 3,300-bed jail." Additionally, Endeavors has some commonality with emerging views of reentry programs (Jeremy Travis, Easing the Transition from Prison to Freedom: Community Roles, National Public Radio, November 2001). Although these seem like effective programs based on a reasonable premise, formal rigorous evaluations appear to be largely lacking. Endeavors operates outside the prisons with a population who have come from jail or prison, so it varies in important ways from some of the programs summarized above. Furthermore, clients do not reside at the program. What do we know about other similar projects from the literature? Early programs in the 1960s were outgrowths of the idea that job training would reduce subsequent criminality. The Job Training Partnership Act (JTPA) provided Federal training funds for a wide array of services similar to those found in the current mix of programs, including vocational and technical training, basic education and literacy services, and job placement. While most of the funding was directed at youths, some participants were adults with arrest records. In the 1970s, the Transitional Aid Research Project (TARP) provided unemployment compensation and job training and placement services in a number of sites in an attempt to reduce recidivism. Evaluation of these programs, however, has showed no effect on recidivism from any combination of services. In the same era, the Living Insurance for Ex-Offenders (LIFE) program provided equally disappointing results (Berk et al., 1980). A larger replication of this program found no differences between groups getting combinations of services or nothing (Rossi, Berk, and Lenihan, 1980). In the 1970s, the Wildcat Services Corporation program provided supported work for unemployed former heroin using ex-offenders. In this program men worked on work crews for subsidized wages, gradually increasing their work responsibilities and training options. Because of the drug use tie in this program, counseling was also available. All participants had 18 months to find a full-time job. A Vera Institute study of this program showed increased employment stability and earnings, but the effect diminished at the end of 3 years. The effect on recidivism was similar. In the 1980s and 90s, a number of studies were conducted on the relationship between employment and recidivism. Harer (1994) found in a sample of Federal releasees that recidivism was higher among minorities, as well as those who were employed prior to incarceration, those with the most stable post-release housing, and those placed in pre-release employment. These findings were substantiated by Finn and Willoughby (1996) in a study of ex-offenders who participated in the JTPA programs in the late 1980s. These findings prompted more recent programs for ex- offenders that deal with substance abuse, cognitive restructuring, and housing. The Opportunities to Succeed program (OPTS), a more recent program, focuses on substance abuse as well as employment issues for ex-offenders. This program provides intensive supervision, drug treatment, skills and vocational training, family services, and medical services. In a random assignment evaluation of OPTS' effectiveness, researchers (Rossman et al., 1998) found a positive effect on employment among the substance abusing offenders in the program. These programs share many common elements: job readiness training, some skills development, and placement assistance. Some also include supported work (i.e., phased or staged work experiences) and substance abuse services. However, ex-offender programs have not received a great deal of rigorous evaluations. Endeavors seems to offer the opportunity for a sound evaluation. Also, Chattanooga is a fairly small city (roughly 100,000), and this evaluation would contribute to understanding how employment programs could work outside of large urban areas, such as New York and Chicago. What audience would benefit from the evaluation? What could they do with the findings? There are general elements of the Endeavors program that should interest anyone seeking to develop or modify an employment-oriented program for ex-offenders. Interest would likely be less for those interested in with- prison programs, and it is likely to be less for those who seek to deliver services through a residential community corrections setting. Clearly there are programs more structured and expensive than Endeavors, especially those that provide services through residential settings. If Endeavors is effective, this may provide useful information about delivering employment assistance through less expensive means. Is the grantee interested in being evaluated? There is an interest in evaluation because it provides a means to better the program and an evaluation that demonstrates effectiveness is a vehicle for continued funding. The site is supporting an evaluation currently, although it does not appear that the ongoing evaluation will provide a rigorous measure of program outcome. What is the history of the project? From program documentation: "Endeavors originated as a demonstration project under the local Dismas House operation aspiring to (1) capture a larger segment of the ex-offender population for treatment and rehabilitation and (2) make a greater contribution to the efforts to curb the high rate of crime in the Hamilton County area. After 2 years of research and development (funded primarily by the Southeast Private Industry Council), the project concluded with the recommendation that greater benefits could be gained by applying existing resources to the preparation of ex-offenders for the workforce than were previously gained by the food and shelter of Dismas House. In 1999, the Board of Directors decided upon an ambitious restructuring of the Dismas House program to focus exclusively on employment for ex- offenders, and Chattanooga Endeavors was created as a local nonprofit organization. For the 2001-02 fiscal year, it will cost . . . roughly $450,000 to operate Endeavors. Funding is provided from the U.S. Department of Justice, the Tennessee Department of Corrections, and various private foundations. Additional funding comes from direct appeals to local corporations, religious organizations, individuals, and from local events." At what stage of implementation is it? From the grant application: "During the first year of funding through the Byrne Grant . . . we received 502 applicants from adult offenders. The vast majority of these offenders were eligible for our program; however, many remain incarcerated in the Tennessee Department of Corrections. Our intake during this 12-month period (i.e., the second year) was 165. Sixty- two percent completed our core curriculum . . . ." The program is currently in its third year of funding. During this third year, it intends to increase the intake to 250 ex-offenders per year. What are the project's outcome goals in the view of the project director? Chattanooga Endeavors is a reentry program for ex-offenders. It recruits unemployed, adult felons from several sources, but an evaluation would likely be based on probationers and parolees. (Probationers would have served some jail or prison time.) There is a 6-week program that focuses on interpersonal skills and maladaptive behavior, substance abuse, education, and work experience. There is up to 1 year of case management. The objective is to reduce criminal recidivism principally through employability. Does the proposal/director describe key project elements? Can you sketch the logic by which activities affect goals? The project's target population is unemployed, adult felony offenders in Hamilton County, Tennessee. Clients come from a variety of justice-related sources, but our assessment and the assessment of the Executive Director is that an evaluation should focus on probationers and parolees. Endeavors' mission is to reduce criminal recidivism, and it attempts to accomplish this mission principally by getting its clients into employment and keeping them in employment. The intervention comprises three segments: a 4-week session (individual and group) focused on developing interpersonal skills and dealing with maladaptive behaviors, a 2-week session dealing with employment issues and finding a job, and up to 1 year of case management. The program does not provide substance abuse treatment, but it does provide relapse prevention. Those who relapse are referred to treatment and dropped from the program. In addition to a Board of Directors, there is an Executive Director (with an M.A. degree and considerable experience in rehabilitation) and an Office Administrator. There are also four ex-offenders on staff, serving as program coordinator, admissions coordinator, job development coordinator, and clerical assistant. A coordinator of volunteers will be hired under the new grant. (Volunteers are part of the program, although the role seems fairly small prior to hiring the coordinator.) The program uses consultants; it has hired a university-based evaluator. The Executive Director feels that the program required 1 year to achieve stability, which is to say that it had been stable for 1 year before the current funding. The first year was required partly to formalize the operational model and partly to recruit, orient, and train ex-offenders as staff members. The Director feels that having ex-offenders as staff is a key to program success. Program Logic Model. Endeavors assumes that steady employment at a "good" job demonstrates a commitment to conventional behavior and reduces the stresses that often lead to criminal recidivism. A good job is one that pays a livable wage and that holds the prospect of advancement through experience and training. Most ex-offenders are unable to find or hold good jobs because of disabilities. Endeavors attempts to deal with those disabilities. First, the Endeavors model provides "resocializing activities" to deal with deficiencies in interpersonal skills and maladaptive behaviors that would make it difficult to find or hold a job. Endeavors provides an intensive 4- week psycho-educational curriculum, including sessions on-- o Core communication. o Self-awareness. o Problem solving. o Decisionmaking. o Goal setting. o Interpersonal skills. o Self-management. o Anger management. o Collaborative team-building skills. Second, Endeavors offers 2 weeks (40 hours) of job-acquisition activities. This period includes-- o Occupational exploration. o Employability skills. o Marketplace realities. o Applications. o Resumes. o Employment interviews. o Employer expectations. o Managing a career. o Job acquisition. Third, Endeavors provides case management (30 hours) during the remainder of the year that includes-- o Group case management. o Individual case management. o Compass support group. o Career plan implementation. Throughout this period of case management, Endeavors offers help on relapse preventions, education and GED training, and computer skills training. These three components comprise the Endeavors program. According to the grant application, Endeavors intends to implement a Transitions Program that would re-admit clients who failed an earlier stint. We presume this enhancement would be excluded from the evaluation. The program uses a Community Building group process, developed by Dr. Scott Peck, as its basic model. The Community refers to the small group that participates in the first phase of the Endeavors program. The program had used the services of The Foundation for Community Encouragement during its first 2 years; it is using a consultant from Tulane University for current program operations. Other core activities are-- o Educational curriculum (computer-oriented teaching). o Relapse prevention program. o Biblio-Therapy (self-awareness through poetry). o Computer training. o Violence prevention. Of note, the program is intended for ex-offenders who are under supervision. Failures in the program can have consequences for supervision status. As noted, relapse to substance abuse is grounds for termination from the program. Are there other local programs providing similar services that could be used for comparison? The alternatives are standard probation and parole. Endeavors provides an enhancement to the services routinely provided to probationers and parolees. Will samples that figure into outcome measures be large enough to generate statistically significant findings for model effect sizes? The size of the population served by Endeavors should be adequate. We are uncertain about the size of the population that would be considered as comparison subjects, but the best evidence (from discussions with Endeavors staff) is that a large population of ex-offenders is available. Is the grantee planning an evaluation? Yes, an evaluation is ongoing. It does not appear capable of answering questions about program outcome except by way of comparing the outcomes of program participants with the outcomes of drop-outs. This is not considered to be a strong design for an outcome evaluation, although it can provide useful information for program improvements. What data systems exist that would facilitate evaluation? What are the key data elements in this system? Tennessee probation and parole offices have some electronic records, although they go back only 2 years. These records have textual elements. Records are at the State level. Police arrest records are presumed to serve as a "trigger" for the probation and parole records. Probation and parole is of sufficient duration that official records are expected to cover the period of recidivism that is likely to be of interest to any evaluation. Probation and parole offices are currently working on a risk assessment study, so there is a precedent for doing a recidivism study. The current evaluator (Dr. Shelia Van Ness) says that she has received good cooperation from police and probation and parole offices for her evaluation. She commented that most records are paper records with data from intake assessments. Data include: o Demographics. o Highest wage rate. o Time served. o Confinement location. o Occupation. o Drug and alcohol use. o Psychological information. In general, how useful are the data systems to an impact evaluation? The data systems seem to offer sufficient information to support an outcome evaluation. However, there would be an appreciable need for data coding, because most of these systems are not electronic. Evaluation Potential. The purpose of Endeavors is to reduce criminal recidivism by preparing ex-offenders for the workforce and by assisting them in obtaining meaningful employment. An outcome evaluation would seek to determine whether or not Endeavors' clients had lower rates of criminal recidivism than did similarly situated ex-offenders who did not participate in Endeavors. An outcome evaluation would also seek to determine whether or not Endeavors increased employment and whether or not the mechanism through which Endeavors works is enhancing employment prospects. More narrowly put, however, we think that Endeavors can only be evaluated regarding the services provided to adult, felony probationers and parolees. According to the Executive Director, probationers would all have served some jail or prison time, so Endeavors might be thought of as a reentry program. It seems impractical to evaluate separate aspects of the Endeavors program. For example, it seems impractical to judge whether or not Biblio- Therapy is effective, because the separate aspects of the Endeavors program are combined into an inseparable treatment plan. There does not appear to be sufficient variation in any subcomponent to tease-out the importance of that subcomponent to the overall program impact. This is not a serious impediment to evaluation. Because the ex-offenders are under supervision, recidivism could be measured from probation/parole files. It seems needless to predetermine specifically whether recidivism should be defined as return to prison, rearrest, parole violation, etc. A longer followup period might be determined from public records. We presume that probation and parole officers are familiar with employment histories. The program also calls employers, although this does not seem like a practical approach for monitoring the employment of a comparison group. Observing a contrast is key for any outcome evaluation. The contrast indicates what likely would have happened had the program not been implemented. A control group (random experiments) or a comparison group (quasi-experiments) often provides the contrast. A control/comparison group would be deemed a good contrast if it (1) had the same attributes as the treated group (statisticians would say the two groups have the same support set) and (2) selection bias is either absent or can be taken into account by statistical models. However, the contrast can arise in other ways, one of which suggests itself here. According to the Executive Director, probation/parole officers make differential use of Endeavors. This assessment was confirmed by a representative from probation and parole offices. To make the point, suppose that probation officer "A" never users Endeavors, that probation officer "B" uses it for half of his eligible probationers, and that probation officer "C" always uses Endeavors. If Endeavors is effective and if all probation officers receive the same mix of probationers, then one would expect probations from "C" to have better outcomes than probationers from "B" and probations from "B" to have better outcomes than probationers from "A." This is a simplified illustration, of course, but its logic translates into the more complicated setting that describes the way that Endeavors gets its clients. Regression-based procedures with instrumental variables could be used to analyze such data. The resulting estimates of treatment effectiveness would be free of whatever selection bias enters into the decisions made by probation/parole officers to refer probationers/parolees to Endeavors. This can be a very powerful quasi- experimental design for evaluating a treatment program where selection bias is otherwise an important concern. Arguably then, probation/parole officers' caseload of Endeavors-eligible probationers/parolees should comprise the data. It would be practical to sample if the caseload is too large. The sample would include all the referrals to Endeavors and perhaps a probability sample of others. Note: All referrals to Endeavors would be included, not all admissions. The instrumental variable model can still tease-out an unbiased measure of the treatment effect. Hamilton County seems to offer a reasonable setting to justify making attribution to project activities, because (according to grant materials) there are no other comparable programs to Endeavors. If this is true, then one could reasonably infer that Endeavors' unique contribution accounts for whatever favorable outcomes are observed. Because Endeavors had been in operation for 1 year prior to the current award and because the nature of the program has not changed fundamentally during the current grant period, a sufficient sample size of at least 200 treated subjects and at least 200 comparison subjects could be expected by the end of the current grant. Because data would come from probation/parole records, there is not problem with using data retrospectively. The question is: How long is required for a followup? A 1- year followup is crucial because the literature suggests that an ex-offender who is employed 1 year after prison will stay employed. The Executive Director is actually interested in extending the programs ability to monitor for 3 years. Once the followup period extends beyond the probation/parole period, however, the cost of data collection--especially for comparison subjects--might become extremely expensive to collect because an evaluator would have to track subjects to learn about employment. Of course, criminal records could be tracked, provided there is a useful criminal history information system and it is sometime possible to track earnings records for groups of five or ten subjects through the Social Security Administration. Overall, this seems like an evaluable program. The program will have used a consistent treatment protocol for 2 years. It maintains client records and outcome data are available through probation and parole records, which are expected to be available to evaluators. An instrumental-variables approach provides the means to overcome selection bias as an explanation for program outcomes. On the other hand, we do not see strong impediments to an evaluation, provided evaluators do not attempt to interview subjects. (Interviewing subjects during the treatment process is not a concern; we are concerned with follow-up interviews.) The evaluation would be limited to subjects who received probation and parole supervision. This is a well-defined stand-alone program. The costs of administering the program should be straightforward. One benefit from the program is reduced crime, which would be estimated by making reasonable assumptions about arrest rates per crime and the cost of crime, as such cost estimates appear in a published literature. Another benefit is reduced criminal justice processing costs, principally for reincarceration. Reasonable estimates should be available from the county and State; otherwise, national proxy estimates might be used. It is more difficult to estimate the benefits from employment and reduced substance abuse, except as they reduce criminal behaviors, although some benefit should be attributed to these. In truth, however, if there is a detectable program effect, then the benefits from reduced crime and criminal justice processing would likely make a cost-benefit case. The project is transferable, and the processes are underway to transfer the program to other sites in Tennessee. The most difficult part of such a startup is to find, orient, and train ex-offenders as staff. Outcome measures would take the form of arrests, probation/parole violations and revocations, and reincarceration. They would also include employment histories, earnings records, and employment stability. Relapse to drug use would be included. Given the outcome measures, we would want to include a risk-score for recidivism in any statistical model (prior arrests, prior incarcerations, age, gender, etc.), and index of past drug use, and measures of past employment. The quality of project-specific data will not be known until completion of an evaluation. Recruitment numbers and where they come from? Intake numbers are known. If a recidivism study were based on historical data, then several hundred people who entered Endeavors would be available for study. If the evaluation used interviews, the study size might be problematic, because recruitment would have to be prospective. Numbers of participants served over a realistic time frame? As noted, this does not seem to be a problem. Site Visit Evaluability Assessment Is the project being implemented as advertised? Yes. What outcomes could be assessed by whom? Criminal recidivism should be measurable, either from State criminal history systems or from probation and parole records. Employment would be more difficult to monitor unless that information is captured in probation and parole files. Interviews could augment information, but they are likely to be very expensive. Have the characteristics of the target population changed over time? There have not been material changes from the first year until the beginning of the second year. How large would the target and comparisons be at one year? The program recruited 165 ex-offenders during its first year and it seeks to recruit 250 ex-offenders during its second year. The size of a potential comparison group is presumed but not guaranteed to exceed the size of the treatment group. What would the target population receive in a comparison sample? Probation and parole supervision and whatever ad hoc services that might be available through these agencies. What are the shortcomings/gaps in delivering the intervention? This appears to be a coherent, well-thought-out intervention. What do recipients of the intervention think the project does? During the site visit, one program participant said that he liked the program because it focused on his "strong points." He said that the program helped him to deal with problems that had bothered him all his life. How do they assess the services received? The person we interviewed was positive. So, too, was a second participant whom we met briefly because of the lack of time. What kinds of data elements are available from existing data sources? Probation and parole offices have some electronic records, although they go back only 2 years. These records have textual elements. Records are at the State level. Police arrest records are presumed to serve as a "trigger" for the probation and parole records. Probation and parole is of sufficient duration that the official records are expected to cover the period of recidivism that is likely to be of interest to any evaluation. Probation and parole offices are currently working on a risk assessment study, so there is a precedent for doing a recidivism study. The current evaluator (Dr. Shelia Van Ness) says that she has received good cooperation from police and probation and parole offices for her evaluation. She commented that most records are paper records with data from intake assessments. Data include: o Demographics. o Highest wage rate. o Time served. o Confinement location. o Occupation. o Drug and alcohol use. o Psychological information. What specific input, process, and outcome measures would they support? As noted, the evaluation should be able to assess criminal recidivism, and it may be able to get measures on employment. Input measures probably need to come from probation and parole records, and these may be incomplete. Program records would only provide information on program participants. How complete are data records? It was difficult to assess. What routine reports are produced? The program maintains a tracking system in ACCESS. It went unexamined. Can the target population be followed over time? Yes, because they will be on probation or parole, which are lengthy terms in Tennessee. Can services delivered be identified? Yes, to the extent they are recorded in probation and parole records. Otherwise, the level of services might be inferred from a limited number of interviews. Services delivered by Endeavors would be known. Can systems help diagnose implementation problems? The project is past the implementation stage. Do staff tell consistent stories about the project? Yes. Are their backgrounds appropriate for the activities? Yes, this appears to be the case. The program relies on ex-offenders for much of its staffing. What do project partners receive? Not known. What changes is the director willing to make to support the evaluation? Not discussed, except to conclude that random assignment is not feasible. Would you recommend the project be evaluated? What kinds of evaluation designs do you propose? Overall, this seems like an evaluable program. The program will have used a consistent treatment protocol for 2 years. It maintains client records and outcome data are available through probation and parole records, which are expected to be available to evaluators. An instrumental-variables approach provides the means to overcome selection bias as an explanation for program outcomes. On the other hand, we do not see strong impediments to an evaluation, provided evaluators do not attempt to interview subjects. (Interviewing subjects during the treatment process is not a concern; we are concerned with follow-up interviews.) The evaluation would be limited to subjects who received probation and parole supervision. What should the BJA grant manager know about this project? We have nothing to add.