PLENARY SESSIONS
Program Evaluation
José Vila del Castillo
Moderator
Representative for Mexico and Central America
United Nations International Drug Control Program (PNUFID)
Thank you very much for inviting me to
moderate this section on program evaluation.
I am sure you will agree that we have chosen
speakers who are very highly qualified and
highly respected authorities in the prevention
and treatment of drug abuse.
Evaluation of Prevention
Programs
Abraham Wandersman, Ph.D.
University of South Carolina
United States
Dr. Wandersman’s session described how
evaluation can be helpful in achieving substance
abuse prevention, program improvement and
program outcomes. The session highlighted:
- Empowerment and Evaluation
- Getting to Program Outcomes: A results-based
approach to accountability.
Dr. Wandersman pointed out that the goal of
Empowerment Evaluation is to improve overall
program success. It provides program
developers with tools for assessing the planning,
implementation and results of programs. Thus
program practitioners have the opportunity to
make significant adjustments and contributions
to the effectiveness of their program.
Empowerment Evaluation encourages:
- Improved planning
- Improved quality of program implementation
- Utilization of program outcomes with which to
evaluate the program
- Development of a continuous quality
improvement system and
- The net result of increased probability of
achieving results.
Dr. Wandersman also presented a model he
referred to as “Getting To Outcomes: A results-based
approach to accountability.” This model
identified ten key questions. By answering his
“10 accountability questions” listed in the
following chart, preventionists in the audience
were told that they could achieve results-based
accountability in the substance abuse prevention
programs.
Ten Accountability Questions follow with the
steps needed for strategic planning and
evaluation focus.
|
10 Accountability Questions | Steps Needed for Strategic
Planning and Evaluation Focus |
| 1. What are the underlying needs and conditions that must be
addressed? | Needs, assets/resources assessment |
| 2. What are the goals, target populations, and objectives, i.e., desired outcomes? |
Goal setting |
|
3. Which science (evidence) based models and best practice
programs can be useful in reaching the goals? |
Consult literature & promising practice programs |
|
4. What actions need to be taken so the selected program
“fits” the community context? |
Feedback on comprehensiveness and fit of program |
|
5. What is the plan for this program? | Planning |
|
6. What organizational capacities are needed to implement the
plan? | Organization capacities |
|
7. Is the program being implemented with quality? | Process evaluation |
|
8. How well is the program working? | Outcome and impact evaluation |
|
9. How will continuous quality improvement strategies be
included? |
Lessons learned |
|
10. If the program is successful, how will it be sustained? | Sustainability plans |
Evaluation of Treatment
Programs
D. Dwayne Simpson, Ph.D.
Texas Christian University
United States
Numerous studies based on almost 300 drug
abuse treatment programs and 70,000 patients
over the past 30 years have shown that
treatment can be highly effective in reducing or
eliminating drug use, criminality and related
problems. However, all patients do not have the
same needs and all programs are not equally
effective, so treatment evaluation research has
expanded in recent years to focus on how to
maximize treatment effectiveness and efficiency.
General findings show that—
- Problem severity dictates the appropriate
type and intensity of treatment needed.
- Patients with moderate-to-high problem
severity levels usually need at least three
months of treatment (and for chronic opiate
addiction, this increases to a year or longer)
before significant benefits can be
documented following release. As problem
severity increases, the need for and benefits
of intensive residential care rises. Good
assessments of patient needs and progress
are therefore essential.
- Cognitive stages of treatment readiness (or
motivation) influence the chances that
patients will engage and benefit from
treatment. Special cognitive-based
“induction” strategies for poorly motivated
patients can be effective antidotes,
especially in correctional settings.
- Several distinct, sequential phases of
treatment (e.g., referral, induction,
engagement, early recovery and continuing
care) are related to addiction recovery
outcomes of patients. Establishment of
therapeutic rapport is particularly important.
- Specialized interventions have been
developed that can improve each of these
crucial steps of the therapeutic continuum.
Jesús Cabrera Solís
Director
Centers for Youth Integration
Mexico
Optimizing valuable resources is an overarching
principle in selecting effective drug abuse and
addiction treatment. Mr. Cabrera, underscored
the following points:
Network of Service Providers
At CIJ privileged rights are granted to:
- mixed operating units that offer prevention
programs, community mobilization, and
treatment,
- ambulatory therapeutic service units, which
are promoted in areas of high demand for
services, and
- residential therapeutic units, strategically
located across our country for the purpose
of providing specialized services to complex
cases.
Financial justification
The financial justification is based on the
following example:
- At CIJ, residential service is offered through
3 month programs; 122 cases are seen, of
which 70 cases or 57% go through recovery
and are released.
- The annual operational cost for this type of
service is approximately U.S. $356,000.
- Through the ambulatory service system, and
for a similar line of cost, effective treatment
can be offered to 1,284 cases annually, of
which 449 or 35% can be treated and
released.
- At centers with mixed operating units,
preventive programs are offered to 70,500
persons, services are provided to 512 cases
with addiction problems, of which 130 cases
or 25% are treated and released.
Based on these concepts, the 54 operating units
that form the institutional network for service
providers offer prevention, treatment, and
community mobilization programs; 4 units
provide ambulatory treatment services; and 3
units provide residential services.
Behavioral Training
The institutional training for modality programs
is composed of the following:
- Therapists are trained in basic, intermediate,
and advanced levels,
- Instructors are trained in individual, family,
and group therapy,
- Specialized training is offered to address
specific modalities, such as cognitive
behavioral therapy for treating persons
addicted to cocaine and are registered and
using ambulatory services, and
- Training in supportive therapy modality,
such as acupuncture.
Productivity of networking service provider
- On average, at each therapeutic unit in CIJ
services are provided to 300 patients per
year, of which 70 are treated and released;
- If these figures are obtained in the 3000
and 9000 treatment centers registered in
Mexico and in the United States, the
projected result would indicate that 900,000
and 2,700,000 persons would be treated for
drug addictions, of which 108,000 and
324,000 persons would treated and
released.
Recommendation: To favor and support
ambulatory services over the residential ones in
the design of the networking for service
providers and training programs for therapists.
Evaluation of Prison-Based Therapeutic Communities:
Current Status and Future Steps
George De Leon, Ph.D.
Center for Therapeutic Community Research
United States
Summary of Key Findings:
- Over 80% of admissions to community
based TC’s have criminal histories.
- TC treatment for CJS clients is effective in
community based programs in showing
reductions in drug use and crime. Improvements are related to length of stay.
- Estimates of the percentage of inmates in
state correctional facilities with serious
substance abuse histories range from 50-80%.
- Modified TC programs in prison and jail
settings are effective in reducing recidivism
and relapse to drug use.
- Modified TC programs in prisons plus post
release aftercare produce the largest and
most consistent reductions in recidivism to
crime and in drug use.
- Aftercare programs which are “continuous”
with the prison-based primary treatment
programs appear to be particularly effective.
- The large majority of inmates with
substance abuse problems do not elect
treatment in prison. Among those who do
enter prison TC treatment, most do not
voluntarily elect to continue their treatment
in post release after care settings.
- Individual motivation appears to be a critical
factor in completing prison-based treatment
as well as post release aftercare.
Conclusion:
Prison-based treatment is highly effective in
reducing relapse to drug use and recidivism to
crime when it is followed by aftercare treatment
in the community after release from prison.
However, only a minority of substance abusers
in prison enter treatment in prison or go on to
aftercare.
The implication for treatment, policy and
research: Based upon the science to date, the
impact, effectiveness and cost effectiveness of
prison-based treatment can be significantly
improved.
Four specific recommendations are briefly
outlined.
- Establish continuity of care initiatives:
Treatment initiated in prisons must be
extended after release from prison.
Moreover, aftercare programs should be
continuous with the philosophy and
approach implemented in prison-based
treatment.
- Enhance Treatment Utilization Initiatives:
Strategies are needed to increase the
proportion of inmate substance abusers who
will enter and complete prison-based
treatment and who will continue in post
release aftercare treatment.
- Implement Quality Assurance And Training
Initiatives: Efforts are needed for guiding the conduct
of prison-based treatment and aftercare
treatment programs. These include
standards for accreditation of treatment
programs within prisons to assure the
fidelity of treatment delivery. Such efforts
should be accompanied by uniform training
initiatives for criminal justice and treatment
personnel.
- Define Research and Evaluation Priorities:
Evaluation and research studies should
address the above stated broad
recommendations:
(1) evaluations of the effectiveness of
integrated vs. non integrated treatment
system
(2) research on motivational and other
strategies to increase treatment
utilization
(3) research on models of training
(4) studies of treatment matching e.g.,
clarifying the subgroups of inmate
substance abusers who require prison
treatment plus aftercare, prison
treatment only or post release treatment
only.
