Title: Increasing the Understanding and Control of Battering and Batterers Series: Solicitation Author: National Institute of Justice Published: February 2003 Subject: Funding resources 18 pages 35,000 bytes ------------------------------ Figures, charts, forms, and tables are not included in this ASCII plain-text file. To view this document in its entirety, download the Adobe Acrobat graphic file available from this Web site. ------------------------------ U.S. Department of Justice Office of Justice Programs National Institute of Justice National Institute of Justice Solicitation February 2003 Increasing the Understanding and Control of Battering and Batterers Notice: NIJ continues to experience delays in delivery of U.S. mail. If you are sending time-sensitive materials to NIJ, and in particular proposals responding to NIJ solicitations, we recommend that you use an alternative to the U.S. Postal Service. For delivery by commercial carriers and courier services, our zip code is 20001. APPLICATION DEADLINE: April 10, 2003 ------------------------------ U.S. Department of Justice Office of Justice Programs 810 Seventh Street NW. Washington, DC 20531 John Ashcroft Attorney General Deborah J. Daniels Assistant Attorney General Sarah V. Hart Director National Institute of Justice For grant and funding information, contact: Department of Justice Response Center 800-421-6770 Office of Justice Programs National Institute of Justice World Wide Web Site: http://www.ojp.usdoj.gov/nij ------------------------------ Increasing the Understanding and Control of Battering and Batterers Special Notice o Priority will be given to proposals for research that will inform State and local criminal justice policy and practice. o Non-binding letters of intent to submit an application under this solicitation are due by March 7, 2003. o Each proposal should contain a detailed listing of previous NIJ grant award performance, a privacy certificate, and a Protection of Human Subjects Assurance Form. o Ninety (90) days prior to the completion of the grant period, grantees are expected to submit in draft form a 400 word abstract, a 2,500 word executive summary, and a technical research report. o Deliverables and reporting time lines are being enforced, including final technical reports and data sets as well as standard progress and financial reports. I. Introduction The National Institute of Justice (NIJ), a component of the Department of Justice's Office of Justice Programs, is soliciting proposals for research and evaluation on batterers and battering. NIJ has identified a number of specific areas in need of special attention. These include rigorous program evaluations, accounting for change in batterers, methodological and measurement development, responses to battering, research to inform program development, and cost-benefit analysis. NIJ anticipates awarding up to $1,000,000 for up to 4 grants, depending on the complexity of the study designs and length of the projects. Batterer intervention is part of NIJ's larger portfolio on violence against women research. NIJ was given responsibility for this portfolio through the Violence Against Women Act of 1994 (Pub. L. No. 103-322) and the Violence Against Women Act of 2000 (Pub. L. No. 106-386). NIJ's current research program on Violence Against Women is based on the National Academy of Sciences' reports Understanding Violence Against Women[1] and Violence in Families: Assessing Prevention and Treatment Programs,[2] along with related research and evaluation. NIJ's most recent awards for research on violence against women and family violence can be accessed at http://www.ojp.usdoj.gov/nij/vawprog/grntlist.html, or a copy can be obtained from the Department of Justice Response Center at 800- 421-6770 (in the Washington, D.C. area at 202-307-1480). This solicitation is based also on a batterer intervention workshop sponsored by NIJ in January 2002. Notes from the workshop, Batterer Intervention: Where Do We Go From Here?, are available at http://www.ojp.usdoj.gov/nij/vawprog/batterer_intervention.html. Applicants are reminded that NIJ grants are funded for the full length of the project (unless otherwise noted) and the full cost should be in the total budget requested. Projects may be jointly supported with other agencies if applicants have obtained funding from or applied to other governmental or non-governmental funders. However, NIJ will not seek funding partners on behalf of applicants. Funding secured or pending from other sources should be noted in the proposal. Applications for funding should also be noted. NIJ recognizes that incidences of females battering males do exist. However, this solicitation focuses on violence perpetrated against women. Letters of Intent To accommodate the volume of proposals submitted under this and other solicitations, NIJ requests that non-binding letters of intent be received by March 7, 2003. The Institute will use these letters to anticipate the number of peer panels needed, and to identify conflicts of interest among potential peer panel reviewers. There are two ways to notify us of your intent to submit an application: send email to tellnij@ncjrs.org, or send a letter via overnight courier (due to delays in delivery of US mail) to: Increasing the Understanding and Control of Battering and Batterers, Office of Research and Evaluation, National Institute of Justice, 810 7th Street, NW. Washington, D.C. 20001 (overnight courier ZIP code). II. Background Batterer intervention programs (BIPs) have become the predominant formal response to partner battering. To date, over 35 BIP evaluations have been conducted. Because these evaluations have yielded inconsistent results,[3] there is intense debate about the effectiveness of BIPs, with important implications for the safety of battered women and their children. In January 2002, NIJ sponsored a batterer intervention workshop entitled Batterer Intervention: Where Do We Go From Here? The goal of this workshop was to identify issues currently hindering evaluations of BIPs. (Notes from this workshop are available at http://www.ojp.usdoj.gov/nij/vawprog/batterer_intervention.html.) One outcome of this workshop was the recognition of the explicit need for additional research to better understand the phenomenon of battering and how to better control batterers, including the use of BIPs. With this solicitation, NIJ is seeking proposals that advance the empirical understanding of batterers and battering as well as the effectiveness of BIPs and their accompanying theory, ultimately to improve policy and practice. NIJ especially seeks research that has the potential to be useful in reducing the level of violence against women, increasing the safety of battered women, and improving the criminal justice system responses to battering. NIJ has identified a number of priority research areas, including: o Batterer Intervention Program Evaluation NIJ seeks rigorous evaluations of BIPs to determine their effectiveness. Proposed evaluations could compare existing BIPs or BIPs based on new conceptual models (such as client-treatment matching or specific population programs). NIJ is also interested in evaluations that compare different programs. For example, the proposed evaluation could compare programs that address battering only to programs that address multiple issues (e.g., substance abuse, mental health, and battering). Evaluation is also needed on specific components of the program (e.g., does program length matter?) to determine which aspects of the program are necessary to bring about change. Experimental designs are strongly encouraged. However, NIJ will consider other evaluation designs as required by the needs and constraints in a particular setting. Any proposed evaluation should focus specifically on whether the program reduces violence against women. o Accounting for Change in Batterers Research is needed on what accounts for change in batterers, e.g., batterers' perceptions of justice. Research is also needed on what batterers actually learn in BIPs. This type of information will be useful in refining batterer intervention programs. o Methodological and Measurement Development Methodological advancement (e.g., comparison of traditional experimental evaluation with statistical modeling such as instrumental variable analysis) and measurement development (e.g., risk assessment scales, innovative outcomes) are needed to advance knowledge about the state of battering and batterers. o Responses to Battering Research on formal and informal responses to battering is needed. Examples of such informal responses may be community responses to battering, systemic responses to battering, family/peer responses to battering, and alternatives to BIPs such as detention facilities, half-way houses, or house arrest. o Research to Inform Program Development Basic research on the causes and persistence of battering is needed to inform BIP development, for example, research on batterers' and victims' risk factors and the development and persistence of violence. Of special interest is research to inform specialized pre-trial bail programs designed to reduce revictimization prior to trial. o Research on the National Scope of Battering Programs Research is needed that would provide a national perspective on the state of battering programs in this country. Possible topics include: the number of courts throughout the country that have mandatory batterer intervention, what types of BIPs are courts using, how many batterers utilize private psychological services, and how have legislatures responded to the issue of batterer intervention (e.g., how many States have enacted criteria for BIPs). o Cost-Benefit and Cost-Effectiveness Analysis Criminal justice officials often have to defend the use of BIPs in the context of limited resources. In an era of increasing fiscal accountability, cost- benefit and cost-effectiveness analyses are important considerations to State and local practitioners and policymakers. III. Requirements NIJ has identified a number of important factors that must be addressed in the proposal. o A Strong Theoretical Base Applicants must provide a strong theoretical basis for the program being evaluated or the hypotheses being tested. Further theory development is also encouraged, for example, by borrowing from theories in other fields such as aggression or depression. o Plan for Ensuring the Safety of Women Applicants must explicitly delineate a plan for ensuring the safety of female study participants. o Informed Consent Applicants are required to describe their plans for informed consent as this can impact the implementation and results of the study. o Definition of Success For evaluation research, applicants should explicitly describe their definition of success, including how many participants have to meet some specified criteria for the program to be considered successful; what the program seeks to change, e.g., behavior, attitude, or women's perception of safety; whether change is required immediately upon entering the program or at some subsequent point in time; and finally, how much change is required for the program to be considered successful, e.g., complete cessation of battering or a reduction in battering. o Policy and Practice Implications In a separate heading, applicants are required to address explicitly the policy and practice implications of their research. Two questions must be answered: 1) What information is likely to be learned from this research that we do not already know and 2) How is this information going to help practitioners in their daily work? IV. Suggestions for Applications NIJ seeks to underscore the importance of several factors in conducting meaningful research on battering and batterers. These suggestions are not requirements for funding, but applicants who include one or more of the factors listed below will be given special consideration. Some of these recommendations result from NIJ's batterer intervention workshop, but may be applicable to other areas of research on batterers and battering. Applicants are encouraged to incorporate as many innovative ideas into their proposals as is feasible. o Partnerships Applicants are encouraged to have existing researcher-practitioner partnerships where appropriate. NIJ also encourages applications in which the composition of the research team reflects the diversity of the population under study. Finally, NIJ encourages mentoring relationships in which new researchers collaborate with established researchers in the field. o Innovative Outcomes Evaluations to date have been narrowly focused on violence reduction outcomes. NIJ encourages applicants to expand outcomes included in this research to include outcomes related to: women's experience (e.g., sexual assault, perceptions of fear, ability to make decisions, whether there are subsequent female victims); children (e.g., the child's safety, the batterer's use of the child to control the woman, quality of the father-child relationship), and batterers (e.g., psychological aggression, ability to handle daily hassles, what batterers learn in BIPs). When re-offense variables are included as outcomes, time of re-offense must be addressed. Where appropriate, applicants should include the use of quantitative and qualitative measures from multiple sources (e.g., the victim, offender, counselors, police, children). In many cases, this may require the development of measures. o Evaluation Designs NIJ encourages the use of rigorous experimental designs where appropriate and feasible. Applicants using an experimental design will need to develop a detailed plan to ensure adherence to random assignment. This will include written agreement from key practitioners to use an experimental design and an indication of their commitment to adhere to random assignment. Applicants must provide justification for the chosen control group (e.g., wait-list controls). Alternative evaluation designs will be considered.[4] Applicants are strongly encouraged to borrow evaluation designs from other fields, such as substance abuse treatment or sex offender treatment. Whatever design is chosen, applicants are encouraged to include a process evaluation as part of a comprehensive evaluation (e.g., impact or outcome). Process evaluations alone will not be considered. o Longitudinal Research Research using longitudinal designs is needed to examine factors associated with the initiation, maintenance, escalation, or diminution of battering over time and across the life course. However, given the limited funding of this solicitation, NIJ encourages innovative approaches to longitudinal research (e.g., time series studies, panel studies). o Recruitment Issues of recruitment are critical to this kind of research. Some recruitment issues include: generalizability of the sample of batterers (e.g., recruitment from courts vs. BIPs); impact of the screening process on the study; external validity of the study; effective methods for gaining access to female victims (e.g., partnership with community agencies); and strategies to improve recruitment efforts (e.g., ensure the research is useful to communities). o Attrition Problems with both victim and offender attrition have plagued this body of research. Therefore, it is imperative that applicants provide a detailed plan for reducing attrition. Also, applicants must describe how dropouts and partial attendees will be treated in the analyses. o Dissemination of Results NIJ is interested in expeditious and innovative methods to disseminate the results from all projects supported under this solicitation. Methods that would supplement the more traditional dissemination approaches are encouraged. Results of projects are expected to be reported to NIJ as they become available, at a minimum, through semi-annual progress reports and at the grantees' meetings. Applicants should discuss the types of other reports or products that would become available over the course of the project and the audiences, including practitioners, to which they would be addressed. V. Selection Criteria NIJ is firmly committed to a competitive process in making grant awards. External peer review panelists consider both technical and practical merits of a proposed topic. Successful applicants must demonstrate to an independent peer review panel that: 1) the gap in knowledge they seek to fill is critical to understanding crime and justice and is policy relevant; 2) the proposed research design is rigorous and appropriate to posited research questions; and, 3) principal investigators are highly qualified to execute the design within the proposed budget and time lines. Institute staff then make recommendations to the NIJ Director based on the results of the independent reviews. Final decisions are made by the NIJ Director after consultation with Institute staff. Reviews of grant applications are based upon the following criteria: o Impact of the Project --Importance of the problem; --Potential for significant advances in areas such as violence and victimization, drugs and crime, crime prevention, law enforcement, courts, corrections; --Potential for advancement of scientific understanding of the problem area; --Relevance to improving the policy and practice of criminal justice and related agencies and for improving public safety, security, and quality of life. o Quality and Technical Merit --Awareness of existing research; --Soundness of methodology, analytic, or technical approach; --Innovation and creativity, where appropriate; --Feasibility of proposed project and awareness of pitfalls. o Capabilities, Demonstrated Productivity, and Experience of Applicants --Qualifications and experience of personnel to proposed project; --Demonstrated ability to manage proposed effort; --Adequacy of proposed budget including time and personnel; --Past performance on NIJ grants and contracts. o Budget Considerations --Total cost relative to perceived benefit; --Budget appropriate to level of effort; --Use of existing resources to conserve costs. The application review process (including peer review, decision-making and other considerations) may take up to six months to complete. Notices of award and non-award are distributed simultaneously about 180 days subsequent to the closing date of a solicitation. Notifications will be sent to the address as indicated on the Application for Assistance. Information regarding award status will not be available until notifications are distributed. VI. How to Apply Individuals interested in submitting proposals in response to this solicitation must complete the required application forms and related documents Applicants must include all of the following information and completed forms to qualify for consideration: PART A: o Application for Federal Assistance - Standard Form (SF 424) o Proposal abstract o Table of contents o Budget Detail Worksheet o Budget Narrative o Negotiated indirect cost rate agreement (if appropriate) o Program narrative o References/Bibliography o Names and affiliations of all key persons including applicants and subcontractor(s), advisors, consultants, and advisory board members. Include name of principal investigator, title, organizational affiliation, department (if institution of higher education), address, phone, fax, and e- mail address. o List all previous and current NIJ awards made to principal investigators including grant numbers, information on final reports and other deliverables to NIJ (whether submitted or outstanding) and a listing of all publications (by NIJ or other publishers) resulting from each grant award. PART B: o Privacy certificate o Protection of Human Subjects Assurance Identification/Certification/Declaration (Form 310) o Environmental Assessment (if required) o Geographic Areas Affected Worksheet o Assurances o Certifications Regarding Lobbying, Debarment, Suspension, and Other Responsibility Matters; and Drug-Free Workplace Requirements (one form) o Disclosure of Lobbying Activities o Appendices: --Letters of cooperation from organizations collaborating in the research project --Vita/Resume --Other materials The Application Package is available at the NIJ Web site under the URL: http://www.ojp.usdoj.gov/nij/funding_app.htm The Application for Assistance. The Standard Form 424 should be the first page of the application. Please follow the instructions in completing the Standard Application for Assistance Form. In item 10 of the Application, be sure to include 16.560 as the Catalog for Federal Domestic Assistance Number. The Title should be "Increasing the Understanding and Control of Battering and Batterers." Enter the title of your proposal in Item 11. Start and end dates in Item 13 should be adjusted to accommodate a six month grant making process. The Proposal Abstract. The proposal abstract is a very important part of the application. Along with the Topics Designation Form, the abstract is used in sorting applications for review by the appropriate independent peer panel. Once an award has been granted, the abstract is computerized and serves as a summary available to all interested parties for the duration of the grant. When read separately from the rest of the application, the abstract should serve as a succinct and accurate description of the proposed work. Applicants should concisely describe research goals and objectives, research design, and methods for achieving the goals and objectives. Length is not to exceed 400 words. Use the following two headers and instructions in developing the abstract. o Research Goals and Objectives: --Statement of Purpose. State the problem under investigation including goals and objectives of the proposed research project and anticipated relevance of the project to public policy and/or practice. --Research Subjects. If applicable, describe subjects who will be involved in the proposed project including the number of participants, age, gender, race/ethnicity, and other pertinent characteristics such as how you will gain access to participants. o Proposed Research Design & Methodology: --Methods. Describe the research method and/or design including data to be used in addressing research questions, data collection procedures and instrumentation, access to data, and other methods or procedures unique to the proposed study. --Data Analysis and Products. Describe proposed techniques for data analysis and all expected products including interim and final reports, instrumentation, and data to be archived under NIJ's Data Resource Program. The Program Narrative. The program narrative should provide enough information and detail to adequately describe the proposed project and should include: o Purpose, Goals, and Objectives o Review of Relevant Literature o Methodology o Data Analysis, Anticipated Results and Products o Implications of the proposed research for policy and practice o Staff/Research Management Plan o References, Tables, Charts, Figures & Research Time-line (not within page limit) Page limit. The number of pages included in the "Program Narrative" section of the application must not exceed thirty (30), double-spaced pages in 12-point font, with one inch margins. Tables, charts and figures describing the research design, calendar, analysis plan, etc. are encouraged and will not be counted in the 30 page limitation. Due date. Ten (10) copies (one unbound) of complete proposals must be received at the National Institute of Justice by 4:30 p.m. on April 10, 2003. Extensions to this deadline are not permitted. Faxed or electronically transmitted copies are not accepted. NIJ reserves the right to return incomplete applications, those not responsive to the scope of this solicitation, or those not complying with format requirements. Names and affiliations of the author(s) of the proposal should be clearly identified. Proposals that are incorrectly collated, incomplete, or handwritten may be considered as submitted or, at NIJ's discretion, may be returned without further review. Proposals exceeding the page limit will not be reviewed. No additions to the original submission are allowed. The length of the research study being proposed should be appropriate to the study design. However, to ensure results are useful they must be available in order to inform policy in a timely fashion. Therefore, study time frames of two years or less are encouraged. Application Materials. Applicants should obtain two packets: 1) application forms (including a sample budget worksheet); and, 2) guidelines for submitting proposals (including requirements for proposal writers and requirements for grant recipients). To receive them, applicants can: o Access NIJ's Web site and download application information at: http://www.ojp.usdoj.gov/nij/funding.htm o Request hard copies of the forms and guidelines by mail from the NCJRS at 800-851-3420 or from the Department of Justice Response Center at 800-421-6770 (in the Washington, D.C., area, at 202-307-1480). o Request copies by fax. Call 800-851-3420. o Please note that instructions provided in this announcement supersede those outlined in the document: Guidelines for Submitting Proposals for National Institute of Justice-Sponsored Research. VII. Performance Guidelines To ensure compliance with the Government Performance and Results Act (GPRA), Public Law 103-62, this solicitation notifies successful applicants that they will be required to collect and report information and data that measure the results of the grant. In order to ensure accountability, the following performance standards are established for all NIJ grantees under this solicitation. Research & Evaluation Performance Standards Grantees are required to submit in draft form the following deliverables ninety (90) days before the grant end date: Deliverables Required 90 Days Before the Grant End Date o Abstract (400 words) o Executive Summary (2,500 words) o Research Report o Electronic data and supporting documentation capable of being re- analyzed and used by other researchers All NIJ draft research reports are peer reviewed upon submission. The reviews are forwarded to the principal investigator with suggestions for revisions. The principal investigator is then required to submit the revised final report, abstract, executive summary, final data set and codebook/data dictionary by the end date of the grant. The abstract, executive summary and final report are to be submitted in both paper and electronic/diskette versions. The data set and codebook/dictionary are also to be submitted in electronic form. Data sets must be received by the grant end date. Grant applicants should ensure that the proposed time line and budget accommodates these requirements. In addition to the abstract, executive summary, research report and data set, grantees are required to submit financial status reports and progress reports. Other Required Reports o Quarterly financial status reports (Standard Form 269-A) o Final financial status report (Standard Form 269-A) o Semi-annual Categorical Assistance Progress Reports (OJP Form 4587) o Final Categorical Assistance Progress Report (OJP Form 4587) Financial Status Reports Financial status reports (SF 269-A) are to be submitted quarterly no later than 45 days following the end of each calendar quarter. Two copies of the financial status report must be submitted to the Office of the Comptroller every quarter during which the award is active even if there has been no financial activity during the reporting period. Additionally, a final financial report is due 120 days after the end date of the award. Future awards and fund drawdowns will be withheld if financial status reports are delinquent. Progress Reports Recipients of funding are also required to submit semi-annual Categorical Assistance Progress Reports (OJP Form 4587). Two copies of the progress report must be submitted to the Office of the Comptroller twice a year even if there has been no substantive activity during the reporting period. The progress report should describe activities during the reporting period and status or accomplishment of objectives as set forth in the approved application for funding. For the duration of the award, progress reports must be submitted within 30 days after the end of the reporting periods (January 1 through June 30 and July 1 through December 31). Additionally, a Final Progress Report providing a summary of achievement of the goals and objectives of the award, significant results, and any products developed under the award, is due 90 days after the end date of the award. Future awards and fund drawdowns may be withheld if progress reports are delinquent. Forms for submitting financial and progress reports are available at the NIJ Web site. Report formats will also be provided by the Office of Justice Programs at the time of the grant award. This information will facilitate future program planning and/or research efforts and will allow OJP to provide the Congress and others with measurable results of its grants. Research Report The final research report, due in draft form 90 days prior to the end of the grant, should provide a comprehensive overview of the study and should include a detailed description of the research design, data, and methods, a full presentation of scientific findings, and a thorough discussion of the implications of the research findings for criminal justice practice and policy. Evaluation Report For evaluation studies, the research report should also include a section on Measuring Program Performance. This section should outline the measures used to evaluate program effectiveness, modifications made to those measures as a result of the evaluation, and recommendations regarding these and other potential performance measures for similar programs. (This information will be particularly valuable to NIJ and other federal program agencies in implementing performance measures for federally funded criminal justice programs.) CONFIDENTIALITY OF INFORMATION AND HUMAN SUBJECT PROTECTION NIJ has adopted new policies and procedures regarding the confidentiality of information and human subjects protection. The Department of Justice (DOJ) regulations at 28 Code of Federal Regulations (CFR) Part 22 require recipients of NIJ research funds to protect personally identifiable information that is collected from all research participants. The regulations at 28 CFR Part 22 require applicants for NIJ funding to outline specific procedures for protection of private information about individuals as part of the Privacy Certificate submitted with the application package. In addition to the regulations in Part 22, DOJ has adopted policies concerning protection of human subjects which are the same as those established by the Department of Health and Human Services in 45 CFR Part 46, Subpart A, also known as the "Common Rule." The DOJ regulations are set forth in 28 CFR Part 46. In general, 28 CFR Part 46 requires that all research involving human subjects conducted or supported by a federal department or agency be reviewed and approved by an Institutional Review Board (IRB) before federal funds are expended for that research. If IRB approval is required for this project, a copy of the IRB's approval as well as supporting documentation concerning the IRB's institutional affiliation, its polices and procedures, and necessary assurances must submitted to the National Institute of Justice prior to the initiation of any research activities that are not exempt from the provisions of 28 CFR Part 46. Further information regarding Confidentiality and Human Subjects Protections may be found in Guidelines for Submitting Proposals for National Institute of Justice-Sponsored Research. Guidance and information. Applicants who wish to receive additional guidance and information may contact the U.S. Department of Justice Response Center at 800-421-6770. Center staff can provide assistance or refer applicants to an appropriate NIJ professional. Send applications to: Office of Research and Evaluation Increasing the Understanding and Control of Battering and Batterers National Institute of Justice 810 Seventh Street N.W., Rm. 7324 Washington, DC 20531 [overnight courier ZIP code 20001] SL 000604 Notes 1. National Research Council, 1996, Understanding Violence Against Women, N. A. Crowell and A. W. Burgess, eds. Panel on research on violence against women, Washington, D.C.: National Academy Press. This volume can be purchased from the National Academy Press at 800-624- 6242 or 202-334-3313 or http://www.nap.edu. 2. National Research Council, 1998, Violence in Families: Assessing Prevention and Treatment Programs, R. Chalk and P. A. King, eds. Committee on the Assessment of Family Violence Interventions, Washington, D.C.: National Academy Press. This volume can be purchased from the National Academy Press at 800-624-6242 or 202-334-3313 or http://www.nap.edu. 3. Babcock, J.C., Green, C.E., & Robie, C. (under review). "Does batterer's treatment work? A meta-analytic review of domestic violence." Davis, R.C., and B.G. Taylor, "Does Batterer Treatment Reduce Violence? A Synthesis of the Literature,"Women and Criminal Justice 10(2)(1999): 69-93. 4. For a review of alternative designs, see, e.g., Gondolf, E. W. (2002), Batterer intervention systems: Issues, outcomes, and recommendations, Thousand Oaks, CA: Sage; Rossi, P. H., Freeman, H. E., & Lipsey, M. W. (1999), Evaluation: A systematic approach, 6th ed. Thousand Oaks, CA: Sage. ------------------------------ For more information on the National Institute of Justice, please contact: National Criminal Justice Reference Service Box 6000 Rockville, MD 20849-6000 800-851-3420 e-mail: askncjrs@ncjrs.org