Title: Office of Research and Evaluation Solicitation for Evaluations of Bureau of Justice Assistance Discretionary Funds Projects Series: Solicitation Author: National Institute of Justice Published: February 2003 Subject: Funding resources 17 pages 27,000 bytes ------------------------------ Figures, charts, forms, and tables are not included in this ASCII plain-text file. To view this document in its entirety, download the Adobe Acrobat graphic file available from this Web site. ------------------------------ U.S. Department of Justice Office of Justice Programs National Institute of Justice National Institute of Justice Solicitation Sarah V. Hart, Director February 2003 Office of Research and Evaluation Solicitation for Evaluations of Bureau of Justice Assistance Discretionary Funds Projects APPLICATION DEADLINE: April 11, 2003 ------------------------------ U.S. Department of Justice Office of Justice Programs 810 Seventh Street N.W. Washington, DC 20531 John Ashcroft Attorney General Deborah J. Daniels Assistant Attorney General Sarah V. Hart Director National Institute of Justice For grant and funding information, contact: Department of Justice Response Center 800-421-6770 Office of Justice Programs National Institute of Justice World Wide Web Site: http://www.ojp.usdoj.gov/nij ------------------------------ Office of Research and Evaluation Solicitation for Evaluations of BJA Discretionary Projects ------------------------------ What's New? Changes From Previous Solicitations o Selection criteria include a component on utility. Applicants should explain how their data collection and analysis plans will provide findings relevant to policy and practice. o Reporting requirements for all NIJ grants have changed. NIJ is placing greater emphasis on timeliness and completeness. o Reporting requirements for evaluations are different from those for other research projects. All evaluations now require intermediate reports; final reports must now cover additional topics. Details of these changes are presented below. ------------------------------ I. Introduction The National Institute of Justice is the research, development, and evaluation arm of the U.S. Department of Justice. NIJ is a component of the Justice Department's Office of Justice Programs. In the FY 2002 Justice Appropriations Bill, Congress placed 88 earmarks totaling $94.5 million into discretionary funds administered by OJP's Bureau of Justice Assistance (BJA). The Bill also set aside resources to evaluate these earmarks. The Justice Department determined that NIJ would oversee these evaluations. This solicitation calls for proposals to evaluate selected projects supported by these BJA discretionary funds. These evaluations will provide significant value to a number of audiences. They will inform the Congress of the performance of grantees receiving these funds, by providing information about the projects' effectiveness and utility. They will contribute to our understanding of similar programs operated by State and local professionals throughout the United States; and they will inform future policy and contribute to improved practice in these fields. II. Scope of Evaluation Using several criteria, including grant size, award history, evaluability, and potential contribution to knowledge, NIJ and BJA reviewed each project supported with discretionary funds in order to identify a high-priority subset for possible outcome evaluations. Following this initial review, the candidate projects in the subset were subjected to more thorough evaluability assessments designed to examine each project's scope, activities, and potential for rigorous evaluation. These detailed assessments identified 5 (five) projects as the final set of evaluation projects. Evaluability assessments for each of these 5 projects are available electronically on the NIJ Web site (http://www.ojp.usdoj.gov/nij, under "NIJ Funding Opportunities"). NIJ intends to award grants under this solicitation for each of the 5 projects. Applicants should assume that the primary audience for the evaluation consists of Federal program funders and State and local program developers. Evaluation designs should address three issues relevant to this audience: 1) effectiveness--attribution of project outcomes to project activities; 2) transferability, or feasibility for adoption by other organizations; and, 3) return on investment--whether the projects are cost- effective or, wherever feasible, cost-beneficial. Applicants should review each evaluability assessment carefully and use the information to formulate an evaluation design. Each project has named a site liaison to provide additional project information. Applicants should contact the liaison to inquire about additional information and also to explore possible ways that the site staff would be asked to support the evaluation. All awards will be cooperative research agreements. NIJ will monitor these awards intensively in order to provide substantial input to aid in the production of sound outcome findings and minimize expenditure of evaluation funds that will not provide useable products. NIJ will require grantees to submit frequent (e.g., bi-weekly) statistics on project implementation in the early phases of the evaluation in order to identify and correct project irregularities that could threaten the success of the evaluation. In addition, BJA will collect quarterly performance measure data from all discretionary funds projects. Evaluation data collections must be coordinated with this BJA effort. NIJ will negotiate reporting frequencies and statistics with the applicant before the award is made. NIJ reserves the right to terminate an award if it believes that there is little chance of completing a sound outcome evaluation. Applicants may propose to evaluate as many of the projects as they believe that they can manage effectively. Each evaluation must be submitted in a separate proposal. NIJ will assess each proposal on its individual merits. Recognizing that organizations may wish to propose the same personnel on more than one project, NIJ will negotiate key personnel issues for multiple awards to the same organization after technical peer reviews have been completed. Applicants should propose evaluation timetables and durations consistent with the objective of performing a rigorous and successful outcome evaluation. NIJ will conduct a two-day cluster meeting of all evaluation grantees in Washington, DC, about three months after awards; applicants should budget for this meeting in their applications. Applicants should also budget the production of a detailed evaluation design and workplan within 60 days of the award. In addition to submitting final reports, applicants must annually submit substantive reports of interim findings that describe evaluation findings to date and provide feedback to project managers. III. Selection Criteria NIJ is firmly committed to a competitive process in making these awards. External peer review panelists consider the technical merits of the proposal. Successful applicants must demonstrate to an independent peer review panel that: 1) the proposed evaluation design is rigorous and appropriate to the project scope; 2) the principal investigators are highly qualified to execute the design within the proposed budget and time lines; and, 3) the evaluation design and the dissemination plan ensure the utility of findings for both research and practice. Institute staff make recommendations to the NIJ Director based on the independent reviews. Final decisions are made by the NIJ Director after consultation with Institute staff. Reviews of grant applications are based upon the following criteria: o Quality and Technical Merit -- Awareness of existing research and evaluation findings in the problem area; -- Soundness of methodology and overall approach; -- Innovation and creativity, where appropriate; -- Feasibility of proposed project and awareness of pitfalls. o Capabilities, Demonstrated Productivity, and Experience of Applicants -- Qualifications and experience of personnel with respect to outcome evaluations and with respect to the substantive area addressed; -- Demonstrated ability to manage efforts of this scale and complexity; and -- Soundness of proposed budget and management plan, including time and personnel. o Project Utility -- Relevance of evaluation design and analyses to information needs of policy and practice audiences; -- Likely contributions to substantive knowledge base; and -- Plan for communicating results. The application review process (including peer review, decisionmaking, and other considerations) may take up to six months to complete. Notices of award and non-award are distributed simultaneously about 180 days after the closing date of a solicitation. Notifications will be sent to the address indicated on the Application for Federal Assistance (SF 424). Information about award status will not be available until notifications are distributed. IV. How to Apply Applicants must include all of the following information and completed forms to qualify for consideration: PART A: o Application for Federal Assistance - Standard Form (SF 424) o Project Designation Form o Proposal abstract o Table of contents o Budget Detail Worksheet o Budget Narrative o Negotiated indirect rate agreement (if appropriate) o Program narrative o References/Bibliography o Names and affiliations of all key persons including applicants and subcontractor(s), advisors, consultants, and advisory board members. Include name of principal investigator, title, organizational affiliation, department (if institution of higher education), address, phone, fax, and e- mail address. o List of all previous and current NIJ awards made to principal investigators, including grant numbers, information on final reports and other deliverables to NIJ (whether submitted or outstanding), and a list of all publications (by NIJ or other publishers) resulting from each grant award. PART B: o Privacy certificate o Protection of Human Subjects Assurance Identification/Certification/Declaration (Form 310) o Environmental Assessment (if required) o Geographic Areas Affected Worksheet o Assurances o Certifications Regarding Lobbying, Debarment, Suspension, and Other Responsibility Matters; and Drug-Free Workplace Requirements (one form) o Disclosure of Lobbying Activities o Appendixes: -- Letters of cooperation from organizations collaborating in the project -- Resumes -- Other materials The Application Package is available at the NIJ Web site at the URL: http://www.ojp.usdoj.gov/nij/funding_app.htm. The Application for Federal Assistance. Standard Form 424 should be the first page of the application. Please follow the instructions to complete this form. In item 10 of the Application, be sure to include 16.560 as the Catalog for Federal Domestic Assistance Number. Indicate the title as "Evaluation of BJA Discretionary Projects." Enter the title of your proposal in Item 11. Start and end dates in Item 13 should be adjusted to accommodate a six-month grant making process. For this solicitation, proposed projects should not have a start date earlier than October 11, 2003. The Topics Designation Form. The Topics Designation Form appears at the end of this solicitation. It should be the second page of the application package and indicate the specific project to be evaluated. The Proposal Abstract. The proposal abstract is a very important part of the application. Along with the Topics Designation Form, the abstract is used in sorting applications for review by the appropriate independent peer panel. Once an award has been granted, the abstract is computerized and serves as a summary available to all interested parties for the duration of the grant. When read separately from the rest of the application, the abstract should serve as a succinct and accurate description of the proposed work. Applicants should concisely describe evaluation goals and objectives, evaluation design, and methods for achieving the goals and objectives. Length is not to exceed 400 words. Use the following two headers and instructions in developing the abstract. o Evaluation Goals and Objectives: -- Statement of Purpose. State the problem under investigation, including goals and objectives of the proposed project, and anticipated relevance of the evaluation results to public policy and/or practice. -- Research Subjects. If applicable, describe subjects who will be involved in the proposed project, including the number of participants, age, gender, race/ethnicity, and other pertinent characteristics such as how you will gain access to participants. o Proposed Evaluation Design and Methodology: -- Methods. Describe the evaluation design, including data to be used in addressing the key research dimensions of effectiveness, transferability, and cost-benefit; data collection procedures; and instrumentation. -- Data Analysis and Products. Describe proposed techniques for data analysis and all expected products, including interim and final reports, instrumentation, and data to be archived under NIJ's Data Resource Program. The Program Narrative. The program narrative should provide enough information and detail to adequately describe the proposed project and should include: o Purpose, goals, and objectives o Review of relevant literature o Design and methods, including limitations o Data analysis, anticipated results o Products and anticipated audiences o Implications of the results for policy and practice o Staff/research management plan o Tables, charts, figures, and research timeline (not within page limit). Page Limit. The number of pages in the Program Narrative section of the application must not exceed thirty (30) double-spaced pages in 12-point font, with one-inch margins. Tables, charts, and figures describing the research design, calendar, analysis plan, etc., are encouraged and will not be counted in the 30-page limit. Due Date. Ten (10) copies (one unbound) of complete proposals must be received at the National Institute of Justice by 4:30 p.m. on April 11, 2003. Extensions to this deadline are not permitted. NIJ recommends the use of an overnight courier (use ZIP Code 20001) or hand delivery for the submission of applications. Faxed or electronically transmitted copies are not accepted. NIJ reserves the right to return incomplete applications, those not responsive to the scope of this solicitation, or those not complying with format requirements. Names and affiliations of the author(s) of the proposal should be clearly identified. Proposals that are incorrectly collated, incomplete, or handwritten may be considered as submitted or, at NIJ's discretion, may be returned without further review. Proposals exceeding the page limit will not be reviewed. No additions to the original submission are allowed. The length of the evaluation being proposed should be appropriate to the outcome questions under study. However, to ensure results are useful, they must be available in order to inform policy in a timely fashion. NIJ requires applicants to report interim evaluation findings. Applicants should also allocate 90 days at the end of the project for reviews and revisions to final reports. Application Materials. Applicants should obtain two packets: 1) application forms (including a sample budget worksheet); and, 2) guidelines for submitting proposals (including requirements for proposal writers and requirements for grant recipients). To receive them, applicants can: o Go to the NIJ Web site and download application information at: http://www.ojp.usdoj.gov/nij/funding.htm. o Request hard copies of the forms and guidelines by mail from the National Criminal Justice Reference Service at 800-851-3420 or from the Department of Justice Response Center at 800-421-6770 (in the Washington, DC, area at 202-307-1480). o Request copies by fax. Call 800-851-3420 and select option 1, then option 1 again for NIJ. Code is 1023. o Please note that instructions provided in this announcement supersede those outlined in the document: Guidelines for Submitting Proposals for National Institute of Justice-Sponsored Research. V. Performance Guidelines To ensure compliance with the Government Performance and Results Act (GPRA), Public Law 103-62, this solicitation notifies successful applicants that they will be required to collect and report information and data that measure the results of the grant. In order to ensure accountability, the following performance standards are established for all NIJ grantees under this solicitation, including deliverables and other required reports: Grantees should use plain English in writing funding proposals and all grant products. (See http://www.plainlanguage.gov for assistance.) However, we recognize that descriptions of research methodologies may necessarily involve some technical information. The grantee should put this technical information in context for the nonresearcher. Research and Evaluation Performance Standards Grantees are required to submit in draft form the following deliverables ninety (90) days before the grant end date: Deliverables Required 90 Days Before the Grant End Date o Abstract (400 words) o Executive Summary (2,500 words) o Research Report o Electronic data and supporting documentation capable of being re-analyzed and used by other researchers All NIJ draft research reports are peer reviewed upon submission. The reviews are forwarded to the principal investigator with suggestions for revisions. The principal investigator is then required to submit the revised final report, abstract, executive summary, final data set, and codebook/data dictionary by the end date of the grant. The abstract, executive summary, and final report are to be submitted in both paper and electronic/diskette versions. The data set and codebook/dictionary are also to be submitted in electronic form. Data sets must be received by the grant end date. Grant applicants should ensure that the proposed time line and budget accommodate these requirements. In addition to the abstract, executive summary, research report and data set, grantees are required to submit financial status reports and progress reports. Other Required Reports o Quarterly financial status reports (Standard Form 269-A) o Final financial status report (Standard Form 269-A) o Semi-annual Categorical Assistance Progress Reports (OJP Form 4587) o Final Categorical Assistance Progress Report (OJP Form 4587) Financial Status Reports Financial status reports (SF 269-A) are to be submitted quarterly no later than 45 days following the end of each calendar quarter. Two copies of the financial status report must be submitted to the Office of the Comptroller, Office of Justice Programs (OJP), every quarter during which the award is active even if there has been no financial activity during the reporting period. Additionally, a final financial report is due 120 days after the end date of the award. Future awards and fund drawdowns will be withheld if financial status reports are delinquent. Progress Reports Recipients of funding are also required to submit semi-annual Categorical Assistance Progress Reports (OJP Form 4587). Two copies of the progress report must be submitted to the Office of the Comptroller twice a year even if there has been no substantive activity during the reporting period. The progress report should describe activities during the reporting period and status or accomplishment of objectives as set forth in the approved application for funding. For the duration of the award, progress reports must be submitted within 30 days after the end of the reporting periods (January 1 through June 30 and July 1 through December 31). Additionally, a Final Progress Report, providing a summary of achievement of the goals and objectives of the award, significant results, and any products developed under the award, is due 90 days after the end date of the award. Future awards and fund drawdowns may be withheld if progress reports are delinquent. Forms for submitting financial and progress reports are available at the NIJ Web site. Report formats will also be provided by OJP at the time of the grant award. This information will facilitate future program planning and/or research and will allow OJP to provide the Congress and others with measurable results of its grants. Research Report The final research report, due in draft form 90 days before the end of the grant, should provide a comprehensive overview of the study and should include a detailed description of the research design, data, and methods, a full presentation of scientific findings, and a thorough discussion of the implications of the research findings for criminal justice practice and policy. Evaluation Report For evaluation studies, the research report should also include a section on Measuring Program Performance. This section should outline the measures used to evaluate program effectiveness, modifications made to those measures as a result of the evaluation, and recommendations regarding these and other potential performance measures for similar programs. (This information will be particularly valuable to NIJ and other Federal program agencies in implementing performance measures for federally funded criminal justice programs.) CONFIDENTIALITY OF INFORMATION AND HUMAN SUBJECT PROTECTION The Department of Justice (DOJ) regulations at 28 Code of Federal Regulations (CFR) Part 22 require recipients of NIJ research funds to protect personally identifiable information that is collected from all research participants. The regulations at 28 CFR Part 22 require applicants for NIJ funding to outline specific procedures for protection of private information about individuals as part of the Privacy Certificate submitted with the application package. In addition to the regulations in Part 22, DOJ has adopted policies concerning protection of human subjects which are the same as those established by the Department of Health and Human Services in 45 CFR Part 46, Subpart A (also known as the "Common Rule"). The DOJ regulations are set forth in 28 CFR Part 46. In general, 28 CFR Part 46 requires that all research involving human subjects conducted or supported by a Federal department or agency be reviewed and approved by an Institutional Review Board (IRB) before Federal funds are expended for that research. If IRB approval is required for this project, a copy of the IRB's approval as well as supporting documentation concerning the IRB's institutional affiliation, its policies and procedures, and necessary assurances must be submitted to the National Institute of Justice prior to the initiation of any research activities that are not exempt from the provisions of 28 CFR Part 46. Further information regarding Confidentiality and Human Subjects Protections may be found in Guidelines for Submitting Proposals for National Institute of Justice-Sponsored Research (on the NIJ Web site). Guidance and information. Applicants who wish to receive additional guidance and information may contact the U.S. Department of Justice Response Center at 800-421-6770. Applicants may call Winifred Reed (202-307-2952) or Edwin Zedlewski (202-307-2953) for substantive information only. Send applications to: Evaluation of BJA Discretionary Projects National Institute of Justice 810 Seventh Street N.W., Room 7323 Washington, DC 20531 NIJ recommends the use of an overnight courier (use ZIP Code 20001) or hand delivery for submission of applications. For more information about the National Institute of Justice, please contact: National Criminal Justice Reference Service Box 6000 Rockville, MD 20849-6000 800-851-3420 e-mail: askncjrs@ncjrs.org ------------------------------ SL 000605 ------------------------------ Office of Research and Evaluation Evaluation of BJA Discretionary Projects Project Designation Form (Include this form as the second page of the application) Please check the project box that this proposal would evaluate. NIJ has indicated some key outcome variables and parameters of interest for each project, and has linked each title to a detailed evaluability assessment. In all cases, NIJ expects that a quasi-experimental design is feasible. Applicants may depart from this guidance by providing an appropriate rationale. Building Bridges: A Correctional Option Program for Ex-Offenders (TN) o Outcome Variables -- Recidivism -- Employment Continuity -- Family Reintegration -- Drug Relapse o Potential Comparison Groups -- Halfway Houses/Pre-release Centers -- Intensive Supervised Probation/Parole -- Conventional Job Readiness/Training Programs o Additional Evaluation Interests -- Cost-effectiveness Relative to Alternatives -- Transferability Considerations Statewide Ridge House Collaborative (NV) o Outcome Variables -- Recidivism -- Employment Continuity -- Family Reintegration -- Drug Relapse o Potential Comparison Groups -- Intensive Supervised Parole -- Conventional Post-release Treatment Programs o Additional Evaluation Interests -- Isolate "Spirituality" Effects -- Cost-effectiveness Relative to Alternatives -- Transferability San Bernardino Operation Nightlight (CA) o Outcome Variables -- Rearrests -- Probation Violations -- Other Measures of Delinquency o Potential Comparison Groups -- Other Juvenile Probationers in San Bernardino County o Additional Evaluation Interests -- Cost-effectiveness Relative to Regular Probation -- Relative Effectiveness of Deterrence Actions vis-a-vis Prevention Actions Agricultural Crime, Technology, Information and Operations Network (CA) o Outcome Variables -- Theft Rates -- Arrests/Clearances -- Investigator Productivity o Potential Comparison Groups -- Counties around Sacramento -- Counties in South Texas or Central Florida o Additional Evaluation Interests -- Cost-effectiveness/Cost Avoidances--Property Losses, Insurance Rates -- Transferability of Program Components Anchorage Wellness Court (AK) o Outcome Variables -- Alcohol Relapse -- Rearrests -- Family Reintegration -- Work Continuity o Potential Comparison Groups -- Program Eligibles Excluded Because of Capacity -- Matched Sample of Program Refusals o Additional Evaluation Interests -- Cost-effectiveness/Cost Avoidances--Jail, Courts, Other Supervision -- Transferability of Program Components