THE NEED FOR IMPLEMENTATION EVALUATION CAN BE ILLUSTRATED BY A PROGRAM EVALUATION WHICH YIELDED NEGATIVE RESULTS THAT LED TO THE PROGRAM'S TERMINATION BEFORE IMPLEMENTATION. THE FIRST STEP IN PROGRAM EVALUATION IS TO DETERMINE WHETHER THE PROGRAM WAS PROPERLY IMPLEMENTED. MAKING POLICY OPERATIVE IS A FUNDAMENTAL PROBLEM IN GOVERNMENT INTERVENTION; EVALUATION QUESTIONS SHOULD BE FRAMED IN THE CONTEXT OF PROGRAM IMPLEMENTATION. ALTHOUGH THE IDEAL EVALUATION INCLUDES BOTH OUTCOMES EVALUATION AND IMPLEMENTATION EVALUATION, EVALUATION RESEARCH HAS BEEN DOMINATED BY OUTCOMES EVALUATION. STANDARDIZED ACHIEVEMENT TESTS ILLUSTRATE OUTCOMES EVALUATION IN EDUCATIONAL RESEARCH; COMPARATIVE RECIDIVISM RATES, IN CRIMINAL JUSTICE RESEARCH. PURE OUTCOMES EVALUATION GIVES DECISIONMAKERS LITTLE INFORMATION ON WHICH TO ACT. IMPLEMENTATION INFORMATION OFTEN HAS GREATER VALUE THAN OUTCOMES INFORMATION. THE FIRST STEP IN UNLOCKING THE 'BLACK BOX' APPROACH TO EVALUATION IS FINDING OUT WHETHER THE PROGRAM MOVED FROM AN IDEA TO INITIAL IMPLEMENTATION. NEXT, CHARACTERISTICS OF THE FULLY OPERATIONAL PROGRAM SHOULD BE DETERMINED. EVALUATION DISASTERS CAN RESULT FROM FAILURE TO RECOGNIZE THAT PROGRAM IMPLEMENTATION IS NEITHER AUTOMATIC NOR CERTAIN. IMPLEMENTATION EVALUATION MUST BE ADAPTIVE AND FOCUSED ON USERS. THREE TYPES OF IMPLEMENTATION EVALUATION ARE EFFORT EVALUATION (QUANTITY AND QUALITY OF PROGRAM ACTIVITY), PROCESS EVALUATION (INTERNAL DYNAMICS OF A PROGRAM), AND TREATMENT SPECIFICATION. TREATMENT SPECIFICATION IS AN ALTERNATIVE TO LABELING, AS IT PERMITS IDENTIFICATION OF INDEPENDENT VARIABLES THAT ARE EXPECTED TO AFFECT OUTCOMES. USE OF LABELS TO AGGREGATE INEFFECTIVE PROGRAMS WITH EFFECTIVE PROGRAMS IS PROBABLY A MAJOR SOURCE OF NULL FINDINGS IN EVALUATION RESEARCH. DIFFERENT TYPES OF EVALUATIONS SHOULD BE MATCHED TO THE INFORMATION NEEDS OF DECISIONMAKERS AND INFORMATION USERS. A NOTE IS INCLUDED. (CFW)