Title: Perspectives on Crime and Justice Lecture Series 1999-2000 Series: Research Forum Author: National Institute of Justice Published: March 2001 Subject: Sentencing; crime patterns and future trends; drug abuse and crime; gun violence; program evaluation 117 pages 240,000 bytes ------------------ Figures, charts, forms, and tables are not included in this ASCII plain-text file. To view this document in its entirety, download the Adobe Acrobat graphic file available from this Web site or order a print copy from NCJRS at 800-851-3420 (877-712-9279 For TTY users). ------------------ U.S. Department of Justice Office of Justice Programs National Institute of Justice Perspectives on Crime and Justice:1999-2000 Lecture Series Volume IV March 2001 National Institute of Justice U.S. Department of Justice Office of Justice Programs 810 Seventh Street N.W. Washington, DC 20531 Office of Justice Programs World Wide Web Site http://www.ojp.usdoj.gov National Institute of Justice World Wide Web Site http://www.ojp.usdoj.gov/nij To find out more information about the National Institute of Justice, please contact: National Criminal Justice Reference Service P.O. Box 6000 Rockville, MD 20849-6000 800-851-3420 e-mail: askncjrs@ncjrs.org National Institute of Justice Perspectives on Crime and Justice:1999-2000 Lecture Series Franklin Zimring Richard B. Freeman William A. Vega Lawrence W. Sherman Heather B. Weiss March 2001 NCJ 184245 NIJ Amy Mazzocco Program Monitor The Professional Conference Series of the National Institute of Justice supports a variety of live, researcher-practitioner exchanges, such as conferences, workshops, planning and development meetings, and similar support to the criminal justice field. The Research Forum publication series was designed to share information about this and other forums with a larger audience. Opinions or points of view expressed in this document are those of the authors and do not necessarily reflect the official position of the U.S. Department of Justice. Preface Policy discussions and debates about crime and justice are too often reduced to buzzwords and emotional charges and countercharges that voice ideologies rather than ideas. It is understandable that crime and justice evoke intense feelings. A great deal is at stake: the safety of our communities--even matters of life and death. This makes it all the more important that discussion and debate should be reasoned, conducted dispassionately at the highest level of political discourse, and informed by sound and scientifically based research. The Perspectives on Crime and Justice lecture series was designed to create an opportunity for policymakers and researchers to take time to reflect on the latest and best research on topical issues of crime and justice. Through this series, NIJ has for the past 3 years presented discussions by some of the Nation's most prominent scholars in criminology and related disciplines. The remarkable series of speakers continued this year. We heard from Franklin Zimring on the politics of punishment, Richard B. Freeman on the relationship between unemployment and crime, William A. Vega on the relationship between immigration and crime, Lawrence W. Sherman on strategies to reduce gun violence, and Heather B. Weiss on reinventing evaluation to improve child and family interventions. The lectures from this series, published here, are intended to elevate the level of debate and bring the results and implications of current research to the attention of decisionmakers at every level of government. ------------------ Contents Preface Lecture 1: The New Politics of Criminal Justice: Of "Three Strikes," Truth-in-Sentencing, and Megan's Laws Franklin Zimring, J.D. Lecture 2: Does the Booming Economy Help Explain the Fall in Crime? Richard B. Freeman, Ph.D. Lecture 3: A Profile of Crime, Violence, and Drug Use Among Mexican Immigrants William A. Vega, D.Crim. Lecture 4: Reducing Gun Violence: What Works, What Doesn't, What's Promising Lawrence W. Sherman, Ph.D. Lecture 5: Reinventing Evaluation to Build High-Performance Child and Family Interventions Heather B. Weiss, Ed.D. ------------------ The New Politics of Criminal Justice: Of "Three Strikes," Truth-in-Sentencing, and Megan's Laws Presentation by Franklin Zimring Professor The Earl Warren Legal Institute University of California at Berkeley December 8, 1999 Washington, D.C. What are the changing political conditions that have been driving the legislative process on issues of crime and punishment in the United States in recent years? First, crime is a more important legislative issue than ever before at both State and Federal Government levels. Second, rather than following a cyclical pattern in which major new laws might relieve the need for further action in the immediate future, the pressure for punitive legislation is more persistent than ever before. Finally, the politics of criminal punishment in the 1990s is characterized by hostility toward not only criminals but also government officials. The changing politics of criminal punishment have had a major influence on the volume of new punishment laws and on their content since the mid-1980s. Although scholars and practitioners are aware of the new political environment of criminal justice, it has rarely been a central topic of scholarly analysis. I will discuss several major themes relating to the new politics of punishment, hoping that this introduction will tempt others to investigate these matters in greater depth. A variety of punishment laws are emerging from State and Federal legislative bodies, fueled by a political environment that is more crime centered, more polymorphously punitive, and more distrustful of government than the traditional politics of American criminal justice. Products of this new climate include "three strikes and you're out" laws (enacted by 25 States and the Federal Government within a 2 1/2-year period), truth-in-sentencing reforms, Megan's Law disclosures, "10-20-life" mandatory minimum sentences for gun crimes, and chemical castration schemes. Rather than diminishing the pressure for further punitive changes, one punitive political success seems to pave the way for others. My survey of this new politics comes in four installments. First, I cast doubt on two theories that explain the cause of this new political atmosphere: the "crime wave" and "mad as hell" explanations. Second, I describe three characteristics of the new politics that have helped shape penal legislation. Third, I offer my theory of the causes of recent political change. Finally, I suggest promising ways to limit the negative impact of the new politics on criminal justice. Two Simple Explanations According to two popular explanations, the intense new politics of punishment is (1) a result of rising crime rates or (2) the product of increasing citizen hostility toward criminals. Neither explanation fits well with recent American history. The bulk of the U.S. violent crime increase occurred between 1964 and 1974, when the homicide rate doubled. Between 1974 and 1980 homicide rates declined, then climbed up to the 1974 level in 1980, dropped through 1984, and climbed again through 1991. This crime pattern does not follow the punitive policies pattern on either the upside or the downside. Despite the "crime in the streets" theme of the 1968 presidential election, the U.S. prison population declined through 1972, and there was little legislative activity on punishment at the State level. When sentencing reform heated up in the mid-1970s, there was little of the strident punitive emphasis that later characterized the new politics of punishment. By the time the punitive political setting had developed, the sharp increase in crime of the 1960s and early 1970s had been history for a decade. The new political paradigm also was not closely linked to the crime rate on the downside. Crime has been declining in the United States since 1991, and the cumulative impact of that decline was more substantial by 1998 than in any earlier post-World War II period. Yet, the political pressure for new punitive responses has not let up significantly. Instead, the politics of punishment has become a version of "heads I win, tails you lose," in which decreases in crime are evidence that hard-line punishments work, whereas increases are evidence that they are needed. The sustained crime drops of the 1990s have not produced an era of good feeling. For example, the continued angry pressure for hard-line responses to juvenile offenders persisted in 1998 and 1999 in the face of the sharpest drop in juvenile crimes on record from 1994 to 1998. The juvenile crime pattern is typical. The second popular theory of the origins of the harsh new politics of penal policy fits the historical record better than reaction to rising crime rates but still seems quite unsatisfactory. The driving force of this version is the hostility of the public to crime and criminals. In the famous phrase from the motion picture "Network," the man in the street is suddenly "mad as hell and not gonna take this anymore." This theory shifts attention from variations in crime rates to variations in the public mood about crime, an important plus for the mad-as-hell approach. The timing is more flexible too, because it takes time for citizen anger to be aroused and more time for anger to be channeled into political action. But this explanation for political change wrongly assumes that the man in the street 30 years ago did not think burglars and robbers should be locked up. We suspect that populist attitudes about most crime and most criminals always have been consistently negative in the United States. Yet if hostility and punitive preference are nearly constant over time, populist sentiments cannot explain the recent sharp change in the political climate. It seems more likely that citizen fear and anger are necessary conditions that have been present all along and have interacted with other changes in recent times to generate a distinctive new political climate. I will return to this matter. Three Characteristics of the New Political Landscape Before advancing my own theories of cause, I would like to single out three characteristics of the recent politics of punishment that deserve special attention: (1) the loose link between the symbolism and the implementation of punishment laws, (2) the zero-sum rhetoric supporting punishment proposals, and (3) the paradoxical politics of distrust in penal legislation. Loose Linkage Legislation concerning criminal punishment serves two very different public purposes. One is the symbolic denunciation of crime and criminals, a statement of condemnation that enables the political community to make its detestation of crime manifest in legal form. A second public purpose is to change the behavior of courts, prisons, or parole authorities. For most members of the public, symbolism is the most important aspect of penal legislation. Therefore, no profound linkage is needed between symbolic legislation and major changes in the way the system punishes criminals. This loose link between symbolic and operational impacts has traditionally allowed new criminal laws to bark much louder than they bite, to satisfy the need for symbols of denunciation without making much difference in the penalties meted out to most offenders. But recent events have shown that the loose connection between symbolic and operational impacts can work both ways. The U.S. Federal system and California enacted laws labeled "three strikes and you're out" in 1994. By late 1998 the U.S. Federal statute resulted in 35 special prison sentences, while the California law produced more than 40,000 special sentences. The difference is more than a thousandfold. In other words, laws serving the same symbolic function can produce very different results in different settings. The Federal law was traditional in that it barked louder than it bit. The California law, on the other hand, bit louder than it barked because 90 percent of California's enhanced prison sentences were given not to people who had two prior convictions (strikes) but to offenders who had only one prior conviction. The law was 10 times as broad as its label, "three strikes and you're out." When citizens are concerned more with the symbolism of penal laws than with their results, the same rhetoric can lead to very different operational law reforms. In such cases, the practical impact of new penal laws is determined more by who controls the planning and drafting processes and what they want than by the level of public support for labels like three strikes. And those who wish to maximize impact can ride slogans a long way. The California version of three strikes has resulted in 9 times as many prison terms as all of the 25 other three-strikes laws combined. The chronically loose link between symbolic and operational impact will lead to high-stakes competition for control of legislative drafting as a recurrent phenomenon in the politics of modern criminal justice. The Zero-Sum Fallacy A second feature of the recent politics of punishment also concerns the relationship between the symbolic and the operational aspects of penal law. The rhetoric supporting new punishment proposals in current politics often seems to assume that criminals and crime victims are engaged in a zero-sum contest: Anything that hurts offenders by definition helps victims. If the competition between victims and offenders is like a football game, then any detriment to the offender team helps the victim's score. The zero-sum assumption also conveniently avoids questions about exactly how (and to what extent) measures that hurt criminal offenders might also help their victims. The law of equivalent benefit in true zero-sum settings implies that anything that hurts the other team helps the competition in equal measure. Therefore, to choose punishment policy in a true zero-sum setting, a citizen must simply decide whether she prefers victims or offenders. What makes this approach illogical is the fact that crime and criminal justice is not a zero-sum game. When victims of violent crime are given public funds to compensate them for their economic losses, does that benefit automatically hurt criminal offenders? Of course it does not, because there is no zero-sum relationship to government policy toward criminal punishment and crime victims. But assuming there is one generates a justification for endless cycles of increased suffering on false grounds. Perhaps some believe that the symbolic denunciation of offenders always supports the social standing of crime victims, but that does not mean that the pain of punishment creates equal and opposite reactions in victims. The Paradoxical Politics of Government Distrust The punishment of criminals is at root an exercise of government power. It might, therefore, seem reasonable that citizen support for harsh measures against criminals would rise with increasing citizen trust in government and that citizen support for excessive punishments would decline when confidence in government falls off. In this interpretation, support for harsh punishment would be a disease of excess confidence in state authority. But the recent pattern is the opposite: Support for mandatory penalties and truth-in-sentencing laws increases with additional distrust of, for example, parole officials, judges, and other professionals meting out punishment. Distrust in government can raise the stakes in criminal punishment policy. Citizens worry that judges will identify with offenders and treat them with inappropriate leniency. A bad judge in this view "coddles" criminals and thus acts against the interests of the ordinary citizen. The imposition of stern penal measures such as the mandatory punishment term guards against such governmental weakness. But the mandatory term is a huge expansion of punishment, rendering excessive outcomes in many cases to ensure sufficiency of punishment in a very few that might otherwise escape their just deserts. Such huge inefficiency is the hallmark of the three-strikes law in California and of truth-in-sentencing reforms generally. The politics of distrust links Megan's Law (which allows citizens rather than just police access to information on sex offenders' addresses) to three strikes and to truth in sentencing. Megan's Law reflects distrust of police, three-strikes and mandatory sentences reflect distrust of judges, and truth in sentencing reflects distrust of parole authorities. The Punishment Lobby and Structural Shifts If fear of crime and hostility toward criminals are persistent features of public opinion, what accounts for the intense new phase of the politics of punishment? I suggest two changes in political conditions relating to punishment policy that have interacted with broader changes in State and local politics to create an altered political climate. The first is the growth of single-issue lobbies dedicated to criminal punishment issues. The second is the reduced distance between the symbolic politics of crime and the locations in government where punishments are set. The single-issue punishment lobby is a new element in American State politics. Mothers Against Drunk Driving was one early example in the 1980s. Victims' rights organizations came on the scene in the late 1980s. In California, we have a prison guards' union with a strong pecuniary interest in expanding the scale of imprisonment and a large budget for political contributions. Single-issue lobbies have changed the politics of punishment in several ways. They have mobilized citizen fear and hostility, shaping these emotions into a hard-line consensus for additional punitive legislation. In addition, many of these groups--for example, the guards' union and the authors of the California three-strikes law--care about not only the symbols of punishment but also making punishment more severe. Truth-in-sentencing legislation, like mandatory minimum penalties, is another reform designed to create the maximum impact on prison populations. The job of the results-oriented lobby is to push the public consensus into legislative directions where big operational changes are produced. Finally, single-issue lobbies keep the pot constantly boiling. For example, after the three-strikes law was enacted in California, prime mover Mike Reynolds was in danger of working himself out of a job. Without a pending issue, his political importance was in question. So within months, he had introduced a 10-20-life set of mandatory minimum penalties for crimes committed with firearms. When a version of this proposal passed in California, he worked for its enactment in other States. If he had not, he would have needed still another new proposal or been relegated to the sidelines. I believe that the work of single-issue lobbies to keep the political pot boiling destabilizes the jurisprudence of criminal punishment. Layers of new law are added on top of others like stalactites and stalagmites in limestone caves. By 1999 the layers of legislation that determine criminal punishment in California are as unintegrated and collectively unprincipled as any penal code in the developed world--and subject to change without notice. Structural shifts in the governmental organization of punishments have multiplied the impact of lobbies in State legislatures. Such shifts rendered California vulnerable to a three-strikes revolution of maximum impact. The gap between largely symbolic legislation and the operational setting where punishments were determined was traditionally maintained by the power of expert bodies and legal actors to influence punishments. Sentencing was the province of judges, and power over prison release was in the hands of parole authorities. Removing the authority of parole agencies in the 1970s and putting legislatures in charge of determining punishment for individual offenses and offenders drastically reduced the insulation between democratic politics and the governance of punishment. A key function was relocated from the professional to the political arena. Once that occurred in California in 1977, for three strikes to pass was only a matter of the right groups learning to exploit the vulnerabilities of the new governmental organization of punishment. The deprofessionalization of setting punishment started long before the single-issue lobbies grew powerful, but the two are interacting in some jurisdictions to destabilize punishment levels in a new way. The mandatory minimum sentence is the nuclear weapon of the new politics of crime because it purports to remove any discretion from the sentencer in punishing individuals prosecuted for committing mandatory-term crimes. This disempowers judges and makes the identity of the offender irrelevant to the punishment imposed. In practice, the prosecutor simply assumes powers that prosecutors and judges had shared. In theory, however, choosing punishment becomes nonprofessional and entirely under the control of democratic politics. Broad mandatory minimum laws patterned after California's three-strikes law are the ultimate extreme of politicized punishment. They can be mitigated by prosecutorial discretion, but they otherwise make the enterprise of criminal sentencing a nonprofessional act. Criminal sentencing becomes the province of politics, not professional expertise. There is no insulation between political sentiment and the principles of criminal sentencing. Limiting the Negative Impact of the New Politics What are the countermeasures to unitary and extreme political control of punishment? One focus ought to be on separating individual punishment decisions from general sentiments about crime. The legislature that enacts a penal code should rarely, if ever, decide what prison sentence a person convicted of an offense should serve prior to release. This blending of the general and the particular invites disaster. I have similar reservations about binding general rules promulgated by sentencing commissions. A second focus should be insulating the sentencing of offenders far from political sentiments by interposing expert institutions. Sentencing commissions in several States can be seen as deliberate attempts to create new expert institutions as insulation between politics and punishment. But sentencing commissions are both a risk and a benefit as insulators because they often attempt to restrict discretion in individual cases. The sentencing judge is a key expert in a defense against a populist politics of punishment. In any legal system based on proportionality in criminal punishments, individual decisions and individual discretions are necessary. Judicial discretion was one early casualty of the politics of governmental distrust back when distrust of government was a theme from the Left rather than the Right. Nothing could have been further from the intentions of those early critics than most of the laws and policies produced by the new politics of the 1990s. But the unintended consequences of the shift to determinate sentences may greatly increase the operational impact of a punitive regime if it occurs. It is my belief that structural remedies will be more effective than appeals to reason in the politics of crime. Damage control almost always is the first priority in the democratic politics of punishment. Creating distance between symbolic legislation and the determination of punishment in particular cases is the best hope currently available for a sustained program of damage control. These structural approaches are much more than just mechanical tinkering. Keeping the symbolic and operational spheres of criminal punishment separate confronts the duality of criminal punishment in an appropriate and fundamental way. Question-and-Answer Session Elizabeth Fraser, Institute for Law and Justice, Alexandria, Virginia: I learned recently that a couple of the Northwestern States have State legislation that moves sentencing decisions for revocations of parole into the correctional side, rather than requiring them to go back to the court. When there is revocation by sentenced offenders, they go back to the parole body that watches over them and have a hearing under that authority rather than going to the court. Do you think having sentencing decisions go to the correctional authorities is an improvement? F.Z.: In general, if there is a principled rationale for it, I am greatly in favor of "back end" power in determining correctional stays and questions like revocation because of the "dual currency" phenomenon. When you are sentencing a criminal, you are doing two things at the same time: you are condemning crime, making it perfectly clear how terrible the offender is and how innocent and worthy of community support the victim is; you are also deciding how you should allocate a scarce resource like prison time, how dangerous the offender is, and whether he rather than the next person you are going to judge should deserve an extra year in the State prison system. Those are an awful lot of agendas to juggle. The sentencing judge is making a "front end" decision, one that is closer to the crime and further removed from the time when a person gets out of prison. In traditional parole, a "time to be served" decision was made after the dust had settled and closer to the time that release might occur. The notion that you could maintain the focus on operational impact was an advantage of parole (particularly when people were sentenced to very long terms) that we never noticed until we started abolishing it. There is another thing that we never noticed about our State systems: Who pays the bills for prisons? State governments. Who controls prison populations? Usually, local governments. Judges and prosecutors in most States are instrumentalities of local units of government, and sending people to prison is something like a free lunch for local government. If you ship them out of the county (as opposed to putting them in a county jail), the State pays the bills. When we abolished parole and adopted determinate sentencing, centralized State correctional authorities suddenly had no power over their own population. Parole was the one centralized power that States had. They could make those decisions at the back end of prison sentences and influence and respond to prison overcrowding. The most famous example of that in California came during the Reagan administration when, rather than the States' spending new money, a lot of people had the back ends of their prison terms snipped off in the interest of economy. Having said that there is a great deal structurally to be argued for back-end controls, let me also say there are two things that must happen for the back-end control to have credibility in the new political environment. First, the agency should have a claim to expertise. It is not good enough that some faceless bureaucrat does this. Second, there should be some rationale. The California Determinate Sentencing Act of 1976 appeared to have no rationale. The lawmakers thought parole was awful, so instead an oriental carpet sale replaced parole in California: every sentence issued by a judge was cut 50 percent through nearly automatic good time. This way symbolic sentences could be doubled without paying for the operational impact. I think we learned from three strikes, 10-20-life sentences, and everything else that we're doing in California that without a credible rationale, that kind of mechanical discounting function is naked of principle and thus highly vulnerable. Under those circumstances we must create a structure of governance in which the back-end punishment adjustment agencies can say what it is they are supposed to be experts in and how they are doing a job that couldn't be done as well by a legislature. What sorts of things might that be? Judges can look at the particular facts of a particular crime and a particular offender and measure proportionally how that offender compares with other robbers or burglars and to other claims on penal resources. Proportionality is one part of it. We have to remember that criminal sentences are legal decisions. Anybody who tells me that the rehabilitative ideal is dead in the sense that rehabilitative considerations are irrelevant has never visited a drug court or a juvenile court. Considering alternatives to prison is a second claim to expertise--on either actuarial or treatment grounds--for people who are making decisions about individual offenders. These kinds of structural accommodations are good ideas that can work only with credible rationales and claims to expertise. If there is a good final-exam question for a criminal law class on this, it is going to be, "What is the claim to expertise of a sentencing commission?" One thing is scarce resources--the allocation of scarce penal resources on a centralized, rationalized basis. But I'm not sure that sentencing commissions are a good way (and I'm sure that sentencing guideline grids are not a good way) of measuring proportionality in individual cases. And I'm absolutely sure that, while you might want to add to actual punishment at the back end of the punishment system, the one indispensable actor in individual cases and the one indispensable discretion in the criminal justice system in sentencing is the judge. Any system without substantial judicial discretion will sooner or later be gratuitously and excessively punitive. Charlie Sullivan, Citizens United for Rehabilitation (CURE), Washington, D.C.: I certainly agree with your analysis, but I'd like to point out that there are two areas in this prisoner and prison buildup. First, what you are talking about is scarcity of resources, and I don't think that we have looked at the role of the U.S. Department of Justice in this expansion of prisons and prison space. The Justice Department, since the 1994 Crime Act, has given close to $3 billion to States to build more prisons. And a condition of half of that money is that they move into truth-in-sentencing reforms. This is basically "seed money" to move in that direction. Statistics have backed up the idea that, if a person is locked up, they will be able to divert many, many crimes. We almost were at cross purposes. The Justice Department just gave almost another half billion dollars in this last year's appropriations. General Barry McCaffrey, Director of the Office of National Drug Control Policy, talking about drug treatment, says 10 percent of that money could be used by States to provide drug treatment. A year ago, Minnesota Senator Paul Wellstone wanted to divert that money to drug treatment to help mentally ill prisoners, and the Justice Department went against that. What you're saying in all of these areas is that the Democrats (going back to Lyndon Johnson, etc.)--the ones leading and talking about an enlightened policy- -have joined the Republicans. I think it goes back to the Willy Horton case. Politically, the Congressional Black Caucus is the only group that is continuing to talk about an enlightened criminal justice system and enlightened prison policy. The second point, besides the role of the Justice Department in this prison and prisoner buildup (which I think has not been really researched), has to do with the role of the National Rifle Association (NRA). This gets back to the Democrats as well. The NRA was very close to them, particularly in rural areas where the Democrats have dominated. It has kind of been a "marriage" in which Democrats might have said, "Okay, NRA, we are going to listen to you and we are going to lock up the people who commit crimes for longer sentences." I was in Texas when the first mandatory minimum started in the mid-1970s. It came out of the Democratic legislators, who felt this was the way to respond to the NRA. "If you do the crime, you will do the time." They were trying to avoid the gun control issue. (By the way, I think the Justice Department has done a wonderful job, at least on that issue). Because I am with a grassroots prison reform organization of families of prisoners (as you can tell from my question), I am on Capitol Hill a lot, talking about these issues. F.Z.: Let me first take a little bit of the heat off the Department of Justice by saying that the 1994 Crime Act passed by Congress, with so-called truth-in-sentencing incentives (although incentives of a very peculiar kind) was a wonderful example of the new politics of punishment. It was passed in 1994 in an atmosphere of insatiable punitiveness. Despite that, the Republican majority was back 6 months later to try to amend it to make it more punitive--to take out, among other things, the famously labeled "midnight basketball" and to toughen up some policy programs they regarded as equivocal. That $3 billion in grants that you're talking about is in pursuit of one version of truth in sentencing, which is wildly different from other versions, and I want to use it as an example. It requires that prisoners convicted of violent crimes serve 85 percent of their sentences. This aim at violent offenders, if it had any impact at all, might have been a radical redistribution of prison resources. The creation of a double scale in which the State correctional facilities can without penalty "cycle out" their nonviolent drug offenders while keeping in their violent offenders (while that is problematic on some theoretical grounds) is very different, in a practical sense, from having legislatively imposed truth in sentencing for everybody. Yet, truth in sentencing as a Federal law is totally mindless because there are three very different sentencing systems in the States, and the law has hugely different impacts on each. Where there are sentencing commissions with mandatory guidelines that were historically based, the 85 percent really means that people will go on serving sentences that historically had been determined by previous parole release patterns. That's very different from what is going to happen where there are active parole authorities at the State level and the standard sentence that Federal law requires is much longer because it is based on nominal preparole sentences. That's different again from systems that have become determinate through force of law without anybody doing anything--the automatic releases of the California system. So I think that the Federal truth in sentencing incentive is, first of all, an example of the new politics. Second, it is a wonderful example of how having some control over the process of drafting a law can make for huge operational changes in a system if truth in sentencing was going to happen in some form. The violence-only form that the 1994 Federal law gave it was by no means the one that would have maximized the negative operational impact of truth in sentencing. Third, when you look at the money that Congress was trying to give the States and the pressure it was trying to put on the States, and you compare that with what the Federal Government has done in truth in sentencing--how much money has been spent and how stringent the Federal effort has been for making sure that a lot of folks are getting locked up with that money--you see that there is a huge gap. There is no ideological sincerity on the part of the administration of the Department of Justice in the enforcement of those provisions. If you use the Crime Act of 1994 as an example of the new politics, it is still "barking a lot louder than it bites," at least through the end of the year 2000. The political winds may change that, and the structural accommodations may still create a system in which the net effects of this kind of Federal policy are to increase prison populations and increase them substantially. Jeremy Travis, National Institute of Justice, U.S. Department of Justice, Washington, D.C.: The only point I would make about the NRA is that their usual political incentive for tough mandatory penalties is simply to change the subject away from guns. Now, that's fine. The way they used to do that had zero operational impact on the prison system. They didn't care about impact on prisons because their primary ambitions were legislative negatives: to keep the attention off guns. F.Z.: What happened is that, once the NRA started interacting with some of the other single-issue lobbies, they were the big money source (with the prison guards union behind three strikes in California). They also didn't care much if in fact these new laws had a high impact on prison population. So rather accidentally, they got co-opted into the operational impact business and are now supporting laws that "bite a lot harder than they bark" just as easily as they used to support the symbolic laws that had no operational impact. I don't think that theirs is a principled presence in the new politics. I think they have been "swept up" like the rest of us have. Paul Hofer, U.S. Sentencing Commission, Washington, D.C.: In your concern that any sort of determinate rules, mandatory minimum statutes, or guidelines are going to be vulnerable to manipulation if your solution is going to be to empower judges, what about the traditionalists' concern regarding disparity in the power of different judges? F.Z.: The issue of disparity in outcomes was solved in two very different ways in the 1970s. Most of the State determinate sentencing regimes that tried to solve the problem of individual judicial disparity did so in a way that turned out to be unprincipled. That is, what they did (I will use Illinois and California as examples simply because those are systems I studied more thoroughly than Indiana and some of the other early determinate sentencing States) is to say that the judge still has unlimited discretion in deciding whether or not a convicted offender goes to prison. We're not touching that discretion. But, if a sentence of imprisonment is decided upon, we are going to force the judge, constrain him given what offense was committed, to a very narrow selection of terms of imprisonment. From a standpoint of principle, that is a hilarious system. What it was responding to was the notion that two guys are cellmates and one says, "I'm a burglar, and I got 10 years." And the other says, "I got 2." So somebody like former corrections commissioner David Fogel writes a book called We Are the Living Proof and makes sure that no matter how many burglars are out on probation, if two of them end up in the Stateville Penitentiary (which he is running), they will have roughly analogous sentences. As a logical matter, that kind of a system had real flaws. The Federal Sentencing Commission guidelines, the Criminal Justice Act of 1984, and the guidelines of 1987, instead, take a broad look at general disparity--because what could be more important, from the perspective of disparity, than whether offenders go to prison or not--and the way in which they deal with the issue is to create binding or near-binding general notions of an appropriate punishment. The problem is that the criminal justice system is now (and has been for most of the 20th century) muscle bound. For most marginal offenders, the choice in punishment is a choice between doing too little and doing too much. The "too much" is prison, and the "too little" seems to be everything else. We have made some efforts in the intermediate punishment area lately, but we don't believe in the credibility of our efforts to create real punishments that aren't imprisonment, if we look at how our sentencing commissions behave. What that meant with the Federal effort was that the way to constrain judicial discretion on the in-out decision was to create a presumption. What kind of presumption? In the Federal sentencing guidelines/standards, that is one of the world's easiest questions to answer: It was a presumption of penal confinement. With that, what you worry about is that discretions, when displaced, can be displaced with excess punitiveness because of the conservativeness of any decisionmaker. That is, the decisionmaker has to worry about two kinds of mistakes: punishing serious crimes not seriously enough and punishing not-serious crimes too seriously. When forced to a choice in a politically responsive environment, if they need a general rule, they are going to punish more seriously and more severely than they would if they had unconstrained discretion. We have learned in 25 years that the choice of displacing in-out discretion is a tradeoff between allowing like cases to be treated in nonalike ways with high degrees of individual discretion, and a system that is excessively severe for many, if not most, of its cases. If the question then is, which of those two evils would I select? I think I'd go for the former. Nick Turner, State Sentencing and Corrections Project, Vera Institute of Justice, New York, New York: You made reference to the fact that this new politics of criminal punishment and the shift of punishment determination from the professionals to the politicians was an unintended consequence. The push for determinate sentences was from people who were concerned about disparity, racial and otherwise, and the consequences were unintended and perhaps unwelcome as well. Do you think there is another unintended consequence of this new politics-- the extreme fiscal burdens it has placed upon States? One example is California, where spending on prisons and corrections has outpaced or surpassed spending on secondary education. On the flip side, States like Georgia or Alabama are considering setting up sentencing commissions to address these cost expenditures. Can you comment on this as a consequence of the new politics? F.Z.: Well, what I'd like to do is buy a postponement. I'd like to wait 10 years to answer the question of whether the costs of prison are an important restraint on excessive imprisonment. I think it's largely an untested notion--particularly in periods of great State and local prosperity. I'm from California and I probably have the same conflict of interest that the prison guards union has. The reason they are rooting for mandatory minimum punishments is rather obvious. They get paid a lot more than school teachers, and the more prison sentences there are, the more members are going to be hired. My obvious conflict of interest is that I'm an employee of the State university system (although lately we have become almost private). It's not true that California now spends more money on corrections than on secondary education. It does, however, now spend more money on its prison system than it spends on the University of California. And that happens to be the branch of government I work for. The reason I know that difference is because secondary education and junior colleges are protected by a State constitutional initiative and will always get their share of the budget. In times of scarcity, it turns out that the prison system and the university system are competing for the same very limited dollars, and so far, the prison system has done a lot better. It is a now $4 billion system in that single American State. I think we have about $2.6 billion in State support in the university system. I'm not sure that California has now entered the biblical "7 fat years." It can't keep many fiscal reserves because the same initiative system that gave us three strikes makes the State issue refunds if it accidently happens to collect too much in taxes. The problem with prosperity is that it makes prison space seem a good deal more affordable than it would otherwise be and that is one reason I'd like to find out how great the fiscal bite is in the longer term. The other problem with assuming that fiscal factors will slow down prison growth is that we also are learning that there are many political shortcuts to make expenditures (particularly capital expenditures) seem pain free. In California, because of the Proposition 13 and Proposition 9 reforms, it looked like bond measures would have to be approved by citizens. The problem was that even during the new politics of punishment, citizens would vote down bond measures. Do we say that will be the way to keep them from building prisons? Not quite. What we do instead is issue a lot of revenue bonds to build prisons. How can you build prisons on revenue bonds? How are you going to get the revenue? Are you going to charge the prisoners? Well, the same legalese that we taught my students to use for the government to justify school expenditures in the context of these constraints works just as well when you are justifying prisons. I think good can flow from the fact that imprisonment is expensive. From the standpoint of worrying about "overimprisonment," one useful law reform strategy is to try to make imprisonment more expensive. It is a lot more expensive in California than in Texas. But I'm not sure that the decisive battles on imprisonment policy are going to be won on fiscal grounds. I think a couple of principles and some notion of limit in the punishment game would help the fiscal arguments a lot. Michael Siegel, Federal Judicial Center, Washington, D.C.: I appreciate your comments about the difficulty of fighting on the symbolic front. However, one, if we cede this ground too easily, are we making the lives of the operators very difficult? Two, are we sure we are going to get the right operators there, particularly in States where judges are elected rather than appointed? Three, I want to offer the possibility that one leverage area is the media. If you watch a week of television news, you do not know that crime is going down in this country. F.Z.: Let me start with the last. That's right; crime hasn't gone down on television. Jeremy mentioned that I did a study for the MacArthur Foundation on American youth violence, and we found one of the great split-personality situations of all time. Homicide arrests have fallen by half among kids between ages 13 and 17 in the United States over the past 6 years. That's the fastest drop we have ever had for any age group that I have ever observed. On the other hand, I don't pick up any media or political coverage that suggests anything other than the notion that youth violence is going up. So, if we can defy gravity when the statistics show that American youth violence has dropped as fast all over the country as lethal violence has dropped in New York, and nobody is noticing, there is a lot to the notion that propaganda on crime rates and crime risks must be countered if the political pressure is to be resisted. Do we risk leaving operating personnel undefended if we don't fight the good fight symbolically? Yes. Is there a problem if the wrong people then are put in charge? Yes. But again, the point I would make is that with whatever political energy and intellectual capital you have, it is very important to be extremely sophisticated about the structural nature of the operational impacts of legislation and to exploit the area between the symbolic and the actual. Because, you see, the symbolic gap and the punitiveness of populism are not American characteristics and not 1990s characteristics. Those are part of the basic operating principles of the governance of punishment in any modern democracy, and they probably always have been. Learning to play by the rules and play off those effects is going to be a lot more promising than trying to win the hearts and minds of the general population. ------------------ Does the Booming Economy Help Explain the Fall in Crime? Presentation by Richard B. Freeman Professor National Bureau of Economic Research, Harvard University and Professor Centre for Economic Performance, London School of Economics February 23, 2000 Washington, D.C. Starting in the mid-1990s, the United States has been experiencing a great economic boom, as reflected in the rapid growth of some of the major indicators of economic health, such as the gross domestic product (GDP) per capita, rising employment, and low inflation. Unemployment fell to levels that just a few years earlier most experts had thought impossible in a noninflationary economy. In addition, after two decades or so of decline, the real earnings of the less skilled men disproportionately involved in crime began to rise. Over roughly the same period, the rate of crime reported in the FBI's Uniform Crime Reports (UCR) fell, while crime reported in the National Victimization Survey continued the downward trend begun in the 1970s. Such well-defined crimes as homicides dropped substantially. Although in 1998 UCR crime rates still exceeded their early 1960s figures, they were 30 percent below the peak rates of the 1980s.[1] To what extent, if at all, has the booming economy contributed to the drop in crime? To what extent, if at all, did the fall in pay for less skilled workers and weak overall job market of the 1980s help maintain a high level of crime, despite the mass incarceration of offenders? The economic model of crime predicts that individuals will choose between crime and legitimate work, depending on the chances of getting a job and the wages in the legitimate market compared with opportunities for illegal earnings and the risk of apprehension and incarceration or other penalties for illegal activity. It is hard to argue a priori against this formulation of the decision to engage in crime, for it simply assumes that, on the margin, criminals respond rationally to differential opportunities. In the extreme, moreover, economic factors have to matter for some crimes. If we were all billionaires, why would anyone commit property crime? If our families were starving, who would not consider stealing food? But economic incentives change more modestly than from billionaire to pauper. In the 1990s boom, unemployment fell by 3 percentage points or so and real wages rose modestly, while income inequality roughly stabilized. Were these changes enough to affect crime in a substantial way? This essay argues that the answer to this question is yes. The evidence is not unequivocal, and there are empirical problems that create some uncertainty, but the preponderance of studies, particularly the most recent econometric work, supports the claim that the booming economy helped reduce the crime rate. This essay first presents the facts about the economic boom and level of crime and examines the predictions of the economic model about the economic rewards of crime and the supply of persons committing crimes. It then makes the case that economic factors have played a substantial role in the 1990s decline in crime. The next section argues against the case that economic factors played an important role in the decline in crime, as it might be presented by a noneconomist. The essay concludes by linking the effects of the economy and other contending explanations to the change in crime over the past half-century. The Economy and Crime "Just the facts, ma'am, just the facts." --Sergeant Joe Friday of the Los Angeles Police Department, "Dragnet" television series, circa 1960 The 1990s boom was the longest in the 20th century. It brought the U.S. rate of employment per adult to an all-time high and reversed both the decline in real wages for regular workers and the rising inequality that was the hallmark of the previous decade or two. Exhibits 1 and 2 show how the 1990s' boom affected the major indicators of the job market. Exhibit 1 documents the rise in the ratio of employment to population to more than 64 percent and the fall in the unemployment rate to about 4 percent by the end of 1999. The less educated young men and minority young men who are disproportionately involved in crime benefited substantially from the boom. Among out-of-school young black men with high school or less education, the unemployment rate fell more than the unemployment rate in the overall economy (Freeman and Rodgers 1999). Exhibit 2 shows the pattern of wage change in the 1980s and 1990s. The real hourly earnings of all male workers, including the young men who make up the bulk of offenders, rose in the late 1990s after having dropped in the previous two decades. The gains in wages included those with the least skills, due in part to an increase in the minimum wage but due largely to the tight labor market. Although the upswing in earnings occurred later than the upswing in employment and seems more fragile, if the unemployment and wage statistics are combined by multiplying the chance of getting a job (1 minus the unemployment rate) by wages to yield an expected return from labor market activity, clear improvement is found in opportunities in the legitimate economy for all workers, men, and less educated men. Exhibits 3, 4, and 5 present the crime part of the story. They record the UCR rates of crime, property victimization, and homicide. All measures fell in the 1990s, but the timing and extent of the changes from the 1970s to the 1990s differ. The UCR rises through the late 1970s, drops in the early 1980s, then rises again until the 1990s. (See exhibit 3.) The victimization survey shows a different pattern: Victimizations drop in the late 1970s and early 1980s, level off in the mid-1980s, and fall again in the 1990s. (See exhibit 4.) While the UCR crime rate in 1999 was 20 percent below its peak, the rate of property crime victimization in 1999 was less than half the rate in the late 1970s. Homicides, a more clearly defined measure of crime, follow the UCR pattern for the most part. (See exhibit 5.) Because unemployment was high and real wages fell for less educated men in the 1970s and 1980s, the UCR data crudely follow the pattern of change in legitimate market incentives. The victimization data, however, move so differently as to create a problem for economic analysis and, for that matter, for any other effort to account for the changing pattern of crime over time. If explanatory factors fit the UCR time pattern, they will have trouble fitting the victimization time pattern, and the converse is true. But in the 1990s, on which this essay focuses, the two series move in tandem. Many social factors beyond economic calculation affect crime (Wilson 1983; Wilson and Petersilia 1994). Some of these factors changed greatly in the period under study--as much as or more than the legitimate labor market. Incarceration increased massively from the 1970s to 2000, with huge proportions of men in the high-crime demographic groups imprisoned. The parents of young Americans were better educated than in the past. (Among blacks, this reflects the civil rights revolution, which opened educational opportunities in the 1960s.) Also, young Americans came from smaller families (partly the result of the Roe v. Wade decision on abortion) (Donahue and Levitt 1999). The age distribution of the population changed. Policing strategies changed. The number of police per capita increased in the 1990s. Finally, the market for drugs, which greatly affects illegal income opportunities, changed. The economics model holds fixed all factors beyond those that affect the monetary incentive to commit crime and asks the following question: How do the financial returns from crime--as opposed to legal work--affect decisions to engage in crime? Formally, if in the formula below "Wc" is the hourly earnings from successful crime, "p" is the probability of apprehension, "S" is the extent of punishment, "W" is the hourly earnings from legitimate work, and "e" is the probability of getting a legal job, the individual will choose to commit crimes rather than take a legitimate job when the utility ("U") from crime exceeds the utility from work: (1-p) U(Wc) - pU(S) > U(eW) This equation has several implications for empirical analysis. It implies that crime should pay a higher wage than legitimate activities (Wc > W). Although reliable data on the rewards of crime are lacking, the limited known information supports this expectation. Hourly rates of pay from crime appear to be higher than from legitimate activity for criminals, but most people who commit crime do not earn that much annually from it. (See exhibit 6.) Most crime is sporadic, and many people combine legal and illegal work to make a living. The model also implies that attitudes toward risk, measured by the curvature of U, are an important element in the decision to commit crime. The risk averse will respond more to changes in the probability of apprehension than to changes in the difference in wages from crime and legitimate work. Most important, the formula shows that the major determinants of decisions to commit crime--the legal job market, illegal opportunities, and sanctions--are intrinsically related. If the rational model is accepted as a valid description of behavior, it cannot be claimed either that tougher penalties reduce crime while a better labor market does not, or the converse. The individual decision to commit crime is, of course, only the first part of any economic analysis. To measure the supply of crime in society, the decisions of all people must be aggregated to produce a schedule that links the total amount of crime to the incentives. One simple way to represent the aggregate relation is to use p, W, e, Wc, and S (where S is measured in dollars) to form the expected return to crime: (1-p)Wc -pS-eW This return becomes the price in a standard labor supply schedule. The shape of this schedule is critical for assessing the way economic and other factors affect crime. Exhibit 7 shows three supply schedules--(A) a standard upward sloping supply schedule, (B) a vertical or inelastic schedule with no economic responsiveness, and (C) a horizontal or infinitely elastic supply schedule. Exhibit 7 also depicts the "demand for crime," measured by opportunities to earn money from crime relative to legitimate activities. This schedule is a downward sloping relation. More crimes reduce the earnings potential due to the declining marginal productivity of crime. The marginal productivity of crime falls for several reasons: the likelihood that criminals pick off easy targets first, the increased effort that citizens are likely to make to protect themselves and their property as crime grows, the likely expansion of policing and harsher penalties in a crime wave, and the greater risk of competition from other criminals. When the supply schedule is inelastic, one way in which criminal justice policy can reduce crime-- incapacitation--operates with complete efficiency. Every criminal incarcerated reduces the number of crimes by the exactly the number of crimes that the criminal would have committed had he been at large. By contrast, when the supply schedule is elastic, incapacitation has no effect on the amount of crime. When Joe is incarcerated, his neighbor Bill sees the opportunity to make money and commits Joe's crimes. Because evidence on the returns to crime is spotty at best, it is difficult to get a handle on the supply-of-crime schedule. One way to assess the potential shape of the schedule is to examine how increased incapacitation affects crime over a given period. Assume, as a first approximation, that the demand and supply schedules for crime are roughly constant in a given period. Then calculate how much crime should change when more or fewer people are incarcerated. If the actual crime rate falls by that amount, the supply of crime is inelastic. If, by contrast, crime falls hardly at all, the supply of crime must be elastic. The apparent failure of incarceration to reduce the rate of crime by anything like the magnitudes predicted by any model of incapacitation (Zimring and Hawkins 1991; Freeman 1996) implies, crudely, that the elasticity of crime is rather high. The weakness of this argument is the assumption that the demand and supply of crime schedules are unchanged. Given the swings in the drug trade, demand for crime surely rose in the 1980s. There also may have been shifts in the supply of people who commit crime due to changing mores and other factors. Perhaps moral values fell in the 1980s. The abortion-induced decline in unwanted births in the 1970s may have reduced the supply of young people with a propensity for crime in the 1990s (Donahue and Levitt). If both demand and supply schedules shift, no inferences can be made about the shape of the schedules without additional data on those shifts. Some insight into the supply-to-crime relation can be gained from an incapacitation model. Given the huge rise in incarceration, UCR data should have shown a drop in crime from the 1970s through the 1980s. However, crime rose, then stabilized. One plausible interpretation is that the supply curve to crime is relatively elastic, so mass incarceration had little effect on the level of crime. The drop in crime in the 1990s was consistent with an improved economy reducing the incentive for crime. At minimum, the evidence shows that there is much more complex behavior in the pattern of crime than a simple "lock them up" criminal justice policy would recognize. To examine the hypothesis that the legitimate labor market can explain some of that behavior and that the booming economy reduced crime in the 1990s, it is necessary to look directly at the relationship between economic incentives and crime. The Case That Economic Factors Matter The first piece of evidence that economic factors matter is that the population of offenders consists disproportionately of people who have low legitimate job market opportunities (Bernstein and Houston 2000). Whatever the source of data on crime-- prisoners, arrestees, self-reports of criminal activity--the less skilled invariably are disproportionately represented. Indeed, U.S. prisons are filled with young men, roughly half from minority groups, who have less than a high school education and score low on written tests, which places them at the bottom of the job market in earnings potential. The contrast with virtually every other aspect of social life, where the educated and skilled are more active in politics, church attendance, volunteering, and so on, is striking. Although the overrepresentation of people with low earnings in crime could reflect psychological or decisionmaking problems among this population, studies show that people who commit crimes are more likely to be unemployed (or idle when they are of school age) than others with comparable skills and that the same person is more likely to commit a crime when jobless than when employed (Freeman 1999). The second piece of evidence is geographic. Nearly all studies of crime rates among several areas, including those focused on non-labor-market factors, contain unemployment in the local job market as a covariate. Many studies include measures of earnings, incomes, or earnings inequality. In 1983 I reviewed the cross-area evidence and concluded that it supported the claim that unemployment affected crime, but not strongly. Ensuing work provides stronger evidence that crime and unemployment across areas are inversely related. Studies that pool cross-section and time-series data across States or metropolitan statistical areas in the 1990s offer particularly powerful evidence of the crime/job market trade-off. These studies include area dummy variables that eliminate any unmeasured fixed-area factors and year dummies that eliminate any overall trends and thus base conclusions on how differential changes in unemployment or wages across areas affect differential changes in crime across areas. As exhibit 8 shows, the three most recent studies find a substantial relationship between unemployment and crime, with a 1-percent change in unemployment associated with an approximate 2-percent change in crime rates. They also find a relationship between wages of less educated workers/workers in retail trade and crime, with an elasticity that averages about -0.5. Studies of the effect on crime of area earnings or inequality give a wider range of estimates of the supply response, possibly because there is no single wage or inequality measure across or even within studies. The third piece of evidence comes from analyses that relate reports of individual crimes or ensuing criminal justice activities to their reported legal or illegal income. These studies look for positive relationships between the extent of individual crime and earnings. One study (Grogger 1997) uses a formal structural model to estimate an elasticity of supply of time to crime of about 1. The 1980 module of the National Longitudinal Survey of Youth contained a question about the proportion of total income that young people who committed crimes earned from crime. The share of income from illegal sources has six values: zero and five nonzero values (very little, to which I assign the value 0.05, about a quarter [0.25], about half [0.50], about three-quarters [0.75] and nearly all [0.95]). Using this numeric scale, the variable has a mean value of 0.17 for out-of-school youth who said they earned illegal income. Conditional on the number of crimes committed and amount of time worked, the share of income from illegal sources varies with the relative rewards of crime (Wc/W). Columns 1 and 2 in exhibit 9 estimate the link between incarceration and share of illegal income (conditional on age, race, Armed Forces Qualifying Test (AFQT) score, weeks worked, and years of schooling). Note first that the chance of committing enough crimes to be incarcerated is positively related to age, negatively related to schooling and AFQT score, and negatively related to weeks worked. The column 1 estimate of the effect of relative income on criminal behavior is huge, implying an elasticity of crime to relative earnings of more than 1.5.[2] In column 2, I added a control for the number of crimes the youth committed. This reduces the effect of the share of income from illegal sources because it is necessarily highly correlated with the number of crimes committed in 1979. Still, the relative income variable has a powerful and highly significant impact, with an elasticity in the range of those estimated by researchers Gould, Weinberg, and Mustard (1998) using area data. The final piece of evidence comes from national time-series data. A modest cyclical link is found between rates of crime and the state of the economy, conditional on trend factors. But the fact that crime rates rose rapidly from the 1960s through the 1980s while unemployment trended up only modestly and the economy grew makes it clear that changes in legitimate market opportunities are not the sole or predominant factor at work. The long time-series analysis is a reminder that changes in the legitimate economy are only part of the story of changes in crime rates. In addition to the direct evidence of the relationship between measures of the job market and rates of crime, evidence from studies of the effect of sanctions on crime also favors the rational decision framework. Indeed, most empirical analyses find that measures of the probability of apprehension or sanctions have more consistent and statistically stronger impacts on crime than unemployment and wages in the legitimate market. Many analysts view labor market factors and sanction factors as competing: Those with a liberal bent want the labor market to matter more while those of a conservative bent want sanctions to matter more. I view this as a false dichotomy. The economics model implies that both factors work through the same decision calculus. Incapacitation aside, sanctions work by affecting incentives, just as legitimate and illegitimate earnings opportunities do. It may be that the generally stronger results obtained with measures of sanctions reflect the fact that we have better measures of them than of the pecuniary rewards from crime, given our inadequate indicators of the actual earnings from crime. I conclude that the preponderance of evidence supports the claim that the job market affects the supply of crime. The order of magnitude of the estimated response parameters suggests, moreover, that economics matters quite a bit. Gould, Weinberg, and Mustard (1998) estimate that economic incentives explain 7 to 18 percent of the increase in particular crimes from 1979 to 1989, with the major factor being the fall in real wages for less skilled workers over the period. Others estimate that the fall in unemployment explains 40 percent of the drop in crime from 1992 to 1997 (Raphael and Winter-Ebmer 2000). With crime changing rapidly, however, the proportion of the trends that economic factors can explain depends on the time period studied. If an elasticity of crime to unemployment of 2.0 is combined with a 3.5-percent drop in unemployment from 1992 to 1999, the booming economy reduced the rate of crime by 7 percent, which compares with an actual drop of about 27 percent. The rise in real wages of less skilled workers at the very end of the 1990s may have contributed an additional 2 percent or so to the fall in crime, bringing the contribution of changes in the legitimate market to one-third of the overall change. Because none of these estimates includes estimates of changes in illegal earnings opportunities, moreover, they potentially understate the total contribution of changing economic incentives to the drop in crime. In this period at least, the evidence suggests that the economy is an 800-pound gorilla in the crime market. The Case Against the Economics Case How much faith should any sensible soul put in what economists say about crime? Economists are honorable folk, but consider their track record in their own domain. A few years ago, they thought the United States could never combine full employment and stable prices. The Federal Reserve Board claimed that the natural rate of unemployment was around 6 percent and denounced those who thought otherwise as irresponsible radicals. How can we explain today's booming economy? A figment of the imagination of the economically unwashed! A few years ago, economists also thought that the Federal deficit would go on and on and on. How can we explain today's budget surplus? Impossible! Unthinkable! Yes, economists are honorable folk, but they cannot explain much of the change in productivity, inequality, or stock market prices. And when they take their theories to the real world, what do we see? The collapse of long-term capital management, gangster capitalism in Russia. The notion that the job market is an 800-pound gorilla affecting crime is pure hubris and chutzpah. Let us look more closely at the alleged evidence. Consider first the timing of changes in crime and the economy. Over the long run, the economy has improved; real wages rose for most of the 20th century. By the standards of 1900, we are all quite well off, with cars, color TVs, telephones, CDs, and more. Only a few of us are billionaires but we are almost all far from paupers who need to rob to feed our families. Has crime fallen during this long period? No. Unemployment was far lower in the 1980s than in the Great Depression. Is crime lower? No. We do not have to go back in history to find flaws in the economists' case. When did the U.S. crime rate begin to rise? In the early 1960s. What was the state of the job market then? Full employment with the real wage of less skilled workers increasing. The UCR crime rate doubled from 1960 to 1969 while GDP per capita grew by more than 2 percent a year. Even in the 1990s, the timing of the drop in crime and the improvement in the economy is not quite right: The UCR index began falling in 1992 while unemployment rose to its peak 1990s level. Much recent economic evidence comes from comparisons of changes in crime and economic conditions area to area. But in which city did crime fall the most in the 1990s? New York City, whose economy was at best sluggardly. Compare the Big Apple and Seattle. In 1998 New York City had an unemployment rate of nearly 8 percent while Seattle had an unemployment rate of nearly 3 percent. But the rate of crime for the New York metropolitan statistical area was 4,208 per 100,000, while that for Seattle was 6,208 per 100,000. Whatever accounts for the low crime in New York City--Mayor Rudolph Giuliani and former Police Commissioner William Bratton's heralded policing policies, the success of the Yankees, changes in the drug business, or the influx of immigrants--it is not a low unemployment rate.[3] Indeed, economic factors do not come close to explaining the variation in crime rates among different areas. Cities, precincts within cities, and blocks within precincts have vastly different rates of crime that cannot possibly be attributed to economic incentives (Glaeser, Sacerdote, Sheinkman 1995). And if we are looking at geographic data, do not forget that there are other countries in the world. Inequality in Europe is much lower than in the United States. Is property crime lower? No. Finally, consider the data on individual crimes. The very notion that there is a tradeoff between crime and legitimate work far exaggerates the evidence. Fagan and Freeman (1999) point out the porous boundary between legitimate and illegitimate work. The mugger or drug dealer may hold a regular job while committing crime or may switch from month to month between crime and legal work. If that is the world of crime, why should anyone believe that a tighter labor market reduces crime? But what is most disingenuous about the economists' claim that the tight job market explains so much of the 1990s drop in crime is that they do not run their analytical calculations against any other possible explanations. Where are the estimates of the effect of changes in social attitudes, or policing practices in the 1990s, or the drug market? The rational calculus model may treat illegal earnings, legitimate opportunities, and sanctions in one unified framework, but there is a difference between changes in wage and employment opportunities and better policing or greater sanctions or changes in the drug trade. There is surely a place for economic incentives in criminal behavior, but it's more like a lemur or rhesus monkey than an 800-pound gorilla. Economic Factors in a Consistent Narrative The problem with assessing the economics case is that, rhetoric aside, the evidence is by no means uniform. The case against economics makes valid points. Economic factors cannot explain large parts of observed patterns, and they seem inconsistent with the rise in crime in the 1960s. But the same is true of most other univariate explanations. That the UCR crime rate did not decline in the 1970s and 1980s despite mass incarceration can be interpreted as evidence either that incarceration does not reduce crime or that it can, but its effect is dwarfed by other factors. Demographic trends fail to explain much of the fluctuations in crime rates, although young men are the major offenders. Some analysts believe that new policing strategies have worked, and it is difficult to explain the pattern among different cities without considering local criminal justice policies. But crime fell in areas with very different policing strategies. Changes in the size of families and reductions in the number of unwanted children due to abortion may explain some of the drop in crime, but proponents of this hypothesis do not claim that this is the entire story. If you think that any single factor can explain a multifaceted social phenomenon like crime, you are not a social scientist assaying the evidence. But this does not mean that we are left with a huge "residual" in our effort to explain changes in crime rates over time. We can weave a consistent story of the swings in the 1990s and earlier. This story is not a story solely of job market incentives, although incentives play a significant part. My "narrative" contains four factors that affect crime: social mores and the way citizens view illegal behavior, demand for drugs and other illegal activities, criminal justice policies, and the job market. As best I can tell, crime rose in the 1960s and 1970s despite full employment because of (1) a shift in attitudes toward legal authority--evinced, for example, in race riots and protests against the war in Vietnam--which made citizens more willing to commit crime and produced less effective policing, and (2) growing demand for illegal drugs. The mass incarceration of the 1970s and 1980s stabilized but failed to reduce the rate of crime because the job market for less skilled young men was poor. Finally, crime fell in the 1990s because of the strong job market, combined with criminal justice policies including continued incarceration of criminals, to lower the economic incentives to commit crime.[4] To be sure, this narrative has enough factors operating to fit almost any pattern of change (economists can get anywhere in the positive price-quantity quadrant by shifting supply and demand), but it is not empty because it directs attention at factors that are quantifiable. Question-and-Answer Session Michael E. Siegel, Federal Judicial Center, Washington, D.C.: I am a political scientist. Do you have any sense of or data on the effect of employment on white-collar crime? R.F.: No. Mostly I would break it down between property crime and violent crime. You would expect that the economic factors should matter in property crime but not so much, if at all, in violent crime. If you look at the tables for some of my estimates, they say "property crime." In fact, you will always get a bigger response of property crime to economic incentives than of violent crime to those incentives. Devon Brown, Office of the Corrections Trustee, District of Columbia Government, Washington, D.C.: I am a behavioral scientist. Do you have any data on whether offenders were employed or underemployed at the time they committed their criminal act? R.F.: Yes. We have some data on that. First, the prison inmates' survey asks questions such as, "What were you doing before you were arrested?" They show that offenders do have a much higher unemployment rate than nonoffenders with, for example, similar skills and low education. Several years ago David Farrington and coworkers in the U.K. followed people for a certain period of their life (Farrington et al. 1986). They found that someone who, for example, committed 10 crimes over 5 years was more likely to commit those crimes when unemployed than when holding a job. The evidence is overwhelming. A. Franklin Burgess, Superior Court of the District of Columbia: I'm from the legal profession. What about the Depression in the 1930s? Did it tell you anything statistically about the relationship between the economy and crime? R.F.: Crime was zooming, but I think Prohibition was overwhelming any Depression effect on crime. When you calculate national time series (analyses in which I don't put much faith), you have to add trend or period variables to control for noneconomic factors. You see a pattern of high unemployment and more crime, but you have to look at the other things happening during the same periods. Marie Provine, National Science Foundation, Washington, D.C.: I'm a political scientist and temporary bureaucrat. Do drug crimes follow any pattern that is different from property crimes, or do you put them in the same category as property crimes? What would you hypothesize, and what do you find in that area? R.F.: The most striking thing about drug crimes is their elasticity of supply. It is very high and that's why I use them as an example. When a drug dealer is arrested, the gang or business that sells drugs will just recruit somebody else. The situation with drug crimes today is similar to the situation in Prohibition: People want a commodity that is illegal. Economists will say that it is very hard to reduce drug crimes because of their elasticity effect. I did no special analysis of drug crimes. Studies of the drug trade in particular cities have come out pretty strongly in support of the economics explanation. Drugs are probably the best case for an economics explanation. If people decided they didn't want these drugs, the demand would die off, and the people who were supplying them would have to make a living in some other way. Beatrix Hamburg, Cornell University Medical College, New York, New York: I'm a behavioral scientist. I was interested that you mentioned England. Also I was hoping that you could give us some policy implications of your talk. It is my impression that in Europe the acceptable unemployment rate has been much lower than in the United States. However, I suspect that there have been some changes in the acceptable rates by reason of the vicissitudes of their own economies. Have you done international comparisons? Should we change our conventional wisdom about acceptable rates here? R.F.: The crime rate in England has been going up in the past year or so--much to the surprise and unhappiness of the people. When you compare Europe with the United States, the property crime rates are very similar. The only place we as a country beat out the Europeans is in violent crime. Europe (including the United Kingdom) currently has higher unemployment than the United States. On the other hand, European countries have less wage inequality and the earnings of the people at the bottom 10 or 20 percent are quite a bit higher; they earn more in real terms adjusted for prices. They don't have as many billionaires as we have, but they take care of the poor, low-wage workers, and other such groups. As far as I can tell, the higher wages and higher unemployment offset each other, and most European countries end up with a crime rate similar to the U.S. rate. They have not had a drop in crime, as we have. Instead, it has gone up a bit in some places. So I don't think that we have anything to learn in that sense from the Europeans. The British do a lot of policing and activity that is space related: putting more police in place and giving locks to people who have been robbed a number of times. Property crime seems to be very localized. They have done quite a number of experiments getting the police and the anticrime strategies in the right place at the right time (a British specialty). But violent crime is different: You get into the issues of guns and other tools used by society that contribute to violent crime. As for policy, if you believe the economics story, you might favor extending the earned income tax credit so that it goes to young men without family responsibilities or raising the minimum wage to deter crime. Increasing the minimum wage hasn't cost any jobs, but at some point it will. Extending the earned income tax credit to these young men says to them, "Your wages are not high because you are a high school dropout or did not do well in school, but society is going to give you some extra money for being a hardworking person who earns a living legitimately." Of course, the costs of such a policy must then be considered. Vincent Schiraldi, Center on Juvenile and Criminal Justice, Washington, D.C.: You said that in Europe the wages for the bottom levels are higher and unemployment rates are higher than in the United States, so it is a wash. That might explain why we have similar property crime rates, but the United States has about six or seven times as many people locked up. In the property crime category, we should be doing a lot better than the Europeans, forgetting violent crime. R.F.: Yes, that is correct. It is a sad thing that we have to do so much more to maintain a rate of property crime comparable to theirs. I stand corrected. Wayne Miller, Bureau of Alcohol, Tobacco and Firearms, U.S. Department of the Treasury, Washington, D.C.: Boston has been held up as a model of effective community policing. Did your study look at Boston in terms of reducing violent crime and the economic impact there? R.F.: I did not study Boston, but I know about the studies of Boston. I talked both to the police and the police union people who were involved. Getting guns out of the hands of the kids was the priority. They said to the young guys, "No guns. If we catch you with a gun, we're going to throw you away forever. We're really going to be tough on you." They didn't say, "Go sell your drugs," but they said, "Do your normal business (which meant go sell your drugs). If we catch you, we catch you, but we are not going to look for you or target you in a particular way." There was a strong policing component and the support of the community, particularly the black community, but the white community as well. The police brought in the gang kids, and they would have a "lineup" with several police officers, a district attorney, and church leaders. Everyone sent the same message: "Don't you dare use guns. You will have no support in this community if you use guns." Black kids were targeted (it was profiling in a real sense). If I walked down the street and I was 18, white, and middle class, they presumably wouldn't hassle me. If I was a black kid in the neighborhood, they would hassle me and if they found anything suspicious--a bulge or something--they'd check for the gun. This strategy had the support of the community because the community was being terrorized. One kid got shot outside a school by marauding gangs fighting over something. There was a uniform statement that this society of adults was as one: You are not going to use guns. It turned out that a lot of the kids in the gangs agreed with the adults. Rather than, "I have a gun and you have a gun. I could shoot you and you could shoot me," now, it is "neither of us has guns." (Crime European style!) In Boston it was not just the community doing its part; it was very focused on the message: We want the guns out of the hands of the kids so we don't have the homicides. Everybody understood that. They didn't send the message, "We are going to stop the drug trade." There is so much money to be made in drugs, and that is where the elasticity of supply is great. A situation developed where all the interests (including the kids, in the end) were aligned. And that was the reason for the success in Boston. Boston did not have quite as big a decline in crime as New York, but it was one of the leading cities in terms of the decline of crime. References Bernstein, Jared, and Ellen Houston, "Falling Crime Rates in the 1990s: The Role of the Low-Wage Labor Market," February 2000, Washington, DC: Economic Policy Institute. Donahue, John J., and Steven D. Levitt, "Legalized Abortion and Crime," Stanford Law School Working Paper, Stanford, CA, June 24, 1999. Fagan, Jeffrey J., and Richard B. Freeman, "Crime and Work," in Crime and Justice: A Review of Research, ed. Michael Tonry, Volume 25, Chicago: University of Chicago Press, 1999: 113-178. Fagan, Jeffrey J., Franklin E. Zimring, and June Kim, "Declining Homicide in New York City: A Tale of Two Trends," Journal of Criminal Law and Criminology 88 (4) (Summer 1998): 1277-1324. Farrington, David P., Bernard Gallagher, Lyndia Morley, Raymond J. St. Ledger, and Donald J. West, "Unemployment, School Leaving, and Crime," British Journal of Criminology 26 (1986): 335-356. Freeman, Richard B., "Disadvantaged Young Men and Crime," in Youth Employment and Joblessness in Advanced Countries, eds. David Blanchflower and Richard Freeman, Chicago: University of Chicago Press for the National Bureau of Economic Research, 2000: Ch. 5. Freeman, Richard B., "The Economics of Crime," in Handbook of Labor Economics, Volume 3c (general series editors, K. Arrow and M.D. Intriligator), eds. Orley Ashenfelter and David Card, Amsterdam, Netherlands: North Holland Publishers, 1999: Ch. 52. Freeman, Richard B., "Why Do So Many Young American Men Commit Crimes and What Might We Do About It?," Journal of Economic Perspectives 10 (1) (Winter 1996): 25-42. Freeman, Richard B., and William Rodgers, "Area Economic Conditions and the Labor Market Outcomes of Young Men in the 1990s Expansion," Working paper no. 7073, National Bureau of Economic Research, Cambridge, MA, May 1999. Glaeser, Edward, Bruce Sacerdote, and Jos‚ Sheinkman, "Crime and Social Interactions," Quarterly Journal of Economics 110 (2) (May 1995): 507-548. Gould, Eric, Bruce Weinberg, and David Mustard, "Crime Rates and Local Labor Market Opportunities in the United States, 1979-1995," Paper presented to the National Bureau of Economic Research Summer Institute Workshop, Cambridge, MA, July 6, 1998. Grogger, Jeffrey, "Market Wages and Youth Crime," Working paper no. 5983, National Bureau of Economic Research, Cambridge, MA, 1997. Raphael, Steven, and Rudolf Winter-Ebmer, "Identifying the Effect of Unemployment on Crime," mimeo, University of California at Los Angeles, Los Angeles, CA, January 2000. Wilson, James Q., and Joan Petersilia, eds. Crime, San Francisco: ICS Press, 1994. Wilson, James Q., ed. Crime and Public Policy, San Francisco: ICS Press, 1983. Zimring, Franklin E., and Gordon Hawkins, The Scale of Imprisonment, Chicago: University of Chicago Press, 1991. Notes 1. The peak total crime index was 5950 in 1980. The crime index in 1998 was 4616. On the basis of data from the period January to June 1999, it will be about 4,154 in 1999, for a decline of 30 percent. See Federal Bureau of Investigation, Uniform Crime Reports, Washington, DC: U.S. Department of Justice, Federal Bureau of Investigation, November 21, 1999. 2. The logistic equation does not immediately give the elasticity of supply to crime because both the dependent variable and the independent variable are a bit more complicated than a regression of ln quantity on ln price. The logistic can be thought of as a log odds ratio regression, linking to independent variables the log of the ratio of the chance of going to prison to the chance of not going. The share of income from crime varies with the ratio of criminal to legal wages, but also depends on the amount of legal and illegal work. As a result, the coefficients in the table must be reduced modestly to obtain the elasticity. See Freeman 2000. 3. Fagan, Zimring, and Kim (1999) look at homicides in New York over a longer period and tell a more detailed story about "contagion effects" in violence. 4. Changes in the drug market may have also played a role here. ------------------ Exhibit 1: Employment/Population Ratio and Unemployment Rate, 1980-2000 Source: Council of Economic Advisers, The 1999 Economic Report of the President, updated. Exhibit 2: Median Weekly Earnings of Full-Time Wage and Salary Workers, 1980-2000 Source: Statistical Abstract of the United States, various editions. Exhibit 3: Uniform Crime Reporting Rate of Index Offenses per 100,000 Inhabitants Source: Tabulated from Maguire, Kathleen, and Ann L. Pastore, eds, Sourcebook of Criminal Justice Statistics 1996, Washington DC: U.S. Department of Justice, Bureau of Justice Statistics, 1997, NCJ 165361, Table 3.106, with 1996 update from the Sourcebook Web site: www.albany.edu/sourcebook. Exhibit 4: Property Crime Victimization Rate per 1,000 Households Source: Bureau of Justice Statistics Web site: www.ojp.usdoj.gov/bjs. Exhibit 5: Homicide Victimization Rate per 100,000 Population Source: Bureau of Justice Statistics Web site: www.ojp.usdoj.gov/bjs. Exhibit 6: Estimates of Illegal Wages Source: Fagan, Jeffrey, and Richard Freeman, "Crime and Work," in Crime and Justice: A Review of Research, Volume 25, Michael Tonry, ed. Chicago: University of Chicago Press, 1999: 225-290. Exhibit 7: Elasticity of Supply and Incapacitation of Crime Exhibit 8: Estimates of the Impact of Job Market Conditions on Crime Using Cross-Area Difference in Difference Models Sources: Gould, Eric, Bruce Weinberg, and David Mustard, "Crime Rates and Local Labor Market Opportunities in the United States, 1979-1995," July 6, 1998, National Bureau of Economic Research, Summer Institute, Cambridge, MA; Freeman, Richard, and William Rodgers, "Area Economic Conditions and the Labor Market Outcomes of Young Men in the 1990's Expansion," National Bureau of Economic Research, WP 7073; and Raphael, Steven, and Rudolf Winter-Ebmer, "Identifying the Effect of Unemployment on Crime," Journal of Law and Economics, forthcoming. Exhibit 9: Logistic Curve Estimates of the Effect of the Illegal Share of Income and Numbers of Crimes Committed in 1979 on Ever Being Incarcerated, 1979- 1996 Source: Tabulated from the National Longitudinal Survey of Youth, U.S. Department of Labor. ------------------ A Profile of Crime, Violence, and Drug Use Among Mexican Immigrants Presentation by William A. Vega Professor of Psychiatry Robert Wood Johnson Medical School University of Medicine and Dentistry of New Jersey March 15, 2000 Washington, D.C. We are a nation of immigrants and immigrant struggles are part of the American cultural mystique. Ironically, the treatment of immigrants in the media, public policy, and political discourse is often aggressive and harsh. The history of 20th-century America often was punctuated with periods of acute hostility toward immigrants. It began with rancor and pejorative descriptions of Eastern and Southern European immigrants as they forged their way into urban centers in the East and Middle West. Less noticed in the Eastern United States was the considerable emigration and resettlement of Mexicans into the Southwest, Far West, and Chicago during the Mexican Revolution of 1910-1920. These new urban and rural settlement areas became the templates for continuing Mexican immigration throughout the century, during which periods of particularly intense immigration alternated with periods of voluntary and involuntary repatriation to Mexico. Occasional outbreaks of overt anti-Mexican hostility also occurred, such as the Zoot Suit Riots in Los Angeles during World War II--a period when segregation of Mexicans and Mexican-Americans was common practice in the Southwestern U.S. In the mid-1950s more than 1 million Mexicans were deported under Operation Wetback. The most recent and vivid example of anti-Mexican immigrant hostility was seen in California in the 1990s during the tenure of Governor Pete Wilson. This period instigated anti-immigrant legislation to limit health, social, and educational services to immigrants, such as Proposition 187, a precursor to the abolition of affirmative action in California. Mexican immigration greatly increased during the final two decades of the 20th century. Problems with the Mexican economy, vigorous U.S. economic growth, and the strong demand for labor in U.S. agriculture, services, and construction industries were powerful stimulants for peak migration and immigration. In 1980 only about 2.2 million of the 8.7 million people of Mexican origin residing in the United States were immigrants. The number of Mexican immigrants doubled by 1990 to 4.3 million, and by 1997 it had accelerated to about 7 million immigrants.[1] These figures document that the number of recent immigrants has significantly increased. I estimate that the 2000 U.S. Census will enumerate about 20 million people of Mexican origin residing in the United States; 40 percent of them will be immigrants. Although this population continues to be disproportionately concentrated in California and Texas, the new immigrant streams are truly national, with Mexican immigrant populations now rapidly developing in the New York-New Jersey area and the Middle Western and Southern States. This increase in numbers of new immigrants has had, and continues to have, a powerful impact on public opinion and the criminal justice system. Media Images Versus Documented Realities It is clear that anti-Mexican feelings have played a consistent and important role in shaping immigrant policy in past decades, and public antipathy regarding the "dilution" of American culture and income transfers to immigrants is now a major public concern. Media images, including political ads, in recent years have portrayed an incessant stream of illiterate Mexicans swamping the border regions, overwhelming the limited resources of the U.S. Border Patrol and making a mockery of immigration policy. Coinciding with these images have been stories of rampant narcotics trafficking on the border, the use of immigrants to import illicit drugs, and Mexican immigrant communities across the United States being used as Trojan horses for the deployment of drug distribution networks. Mexican immigrants are also blamed for challenging local law enforcement agencies with gang activity, increasing levels of violence and theft, and overburdening health and educational institutions. Despite the media-fueled specter of invasion, the United States continues to offer pragmatic reasons for the continuing influx of immigrants. The integration of economic activity under the North American Free Trade Agreement, which went into effect in 1994, has not created disincentives to Mexican immigration in the face of the demographic explosion south of the border, the lack of jobs in many rural areas of Mexico, and the low wages for unskilled and semiskilled workers in the urban centers of central and northern Mexico. As Alan Greenspan, Chairman of the Federal Reserve Board, has remarked on several occasions, the most important long-range threat to U.S. economic growth is a shortage of workers, as U.S. population growth declines. Indeed, Mexican-Americans have among the highest rates of employment of any ethnic group. Although there is an apparent concentration of high-tech growth in the United States, this increasing national affluence has been accompanied by a demand for service and construction workers. Given that the economic disparity between the United States and Mexico is unlikely to change in the decades ahead, it is not too risky to predict a continuing high volume of Mexican immigration into the United States. Only a very serious stall in U.S. economic expansion is likely to offset this trend. Immigration studies have shown that new immigrants will follow the paths of previous family members to established ethnic enclaves and to employment, ultimately reestablishing family networks. With a pattern of accelerating Mexican immigration and settlement in all regions of the United States, it is worth focusing on their documented patterns of behavior rather than depending on the imagery from media reports of sensational incidents or contentious political debates about immigration control. My purpose is to present survey information of importance to criminal justice, law enforcement, and public health officials in formulating their policies and programs. This information is derived from a large field survey of Mexican immigrants and Mexican-Americans residing in urban and rural areas of central California conducted in 1996.[2] (Data gathering and analysis were supported by the National Institute of Mental Health and the National Institute on Drug Abuse.) The primary goals of the survey were to gather information about the prevalence of psychiatric disorders and substance abuse problems and to determine patterns of health services utilization. The survey, designed as an epidemiological study, uses a scientifically viable sampling strategy and household interviews to gather information about 4,000 people between ages 19 and 59. The respondents were representative of the population from which they were selected: low income and disproportionately low education. Immigrants averaged only a grade school education. The crime-related behavior patterns of Mexican immigrants with Mexican-Americans born in the United States are compared. This is self-reported information, gathered through the use of a fully structured interview and a high-quality survey process. Indications of Criminal Behavior Certain behaviors of interest to law enforcement were compared, including nativity (place of origin) differences in who carried a weapon (defined as a gun, knife, or club) in the 30 days before the survey, who had been arrested in the 5 years before the survey (other than those arrested by the Immigration and Naturalization Service [INS]), who had been in a fight serious enough to cause injury, who had used illicit drugs in their lifetime, and who had a diagnosis of lifetime drug abuse or dependence. Immigrants had lower rates than U.S.-born Mexican-Americans on every indicator. Males were much more likely than females to engage in these behaviors, and the differences among males were dramatic. For example, rates for U.S.-born Mexican-American males were about 30 percent greater than those for immigrants for arrests, 150 percent greater for carrying a weapon, 300 percent greater for fighting, 100 percent greater for any illegal drug use in their lifetime, and 300 percent greater for lifetime drug abuse or dependence. The criminal behavior profiles for women were distinctive because their overall rates were much lower than those for men and the rates for immigrant women were negligible. It is precisely for these reasons that the relative differences in drug use between U.S.-born and immigrant women were so striking. Immigrant women had very little illicit drug use, but U.S.-born women had a rate of 45 percent. The lifetime abuse or dependence rate for U.S.-born women was 700 percent higher than for immigrant women and 100 percent higher than for immigrant men. U.S.-born female arrest rates, although very low, were about five times higher than for immigrant women. Rates of fighting were similarly low for both groups of women. The pattern that emerged was very clear. Immigrants, whether male or female, had a much lower likelihood of engaging in these behaviors than did their U.S.-born co-ethnics. These comparisons beg the questions, Is there any evidence of change over time in these behaviors among immigrants? Do things get worse the longer immigrants reside in this country and, if so, for which behaviors? To answer these questions, the study compared those who had resided less than 14 years in the United States with those who had resided 14 years or more, as well as comparing them with the U.S.-born sample. Again, a stable pattern emerged. Immigrants residing in the United States 14 years or more had higher rates overall, a pattern reflected in almost every age group. This pattern was more general for men than women, because immigrant women had such low rates of these behaviors. What is striking about this information, however, is that immigrant men who had come to the United States as children had high rates of arrests and carrying weapons. Also, long-staying immigrants were much more likely to carry weapons even in middle age compared with shorter residence immigrants. Although immigrant women had very low arrest rates, women who entered the United States as girls also had higher rates of carrying weapons than those who entered as adults. (See exhibit 1.) The only instance in which immigrant women with less than 14 years residence had higher rates of negative indicators than long-staying immigrants was in regard to physical and sexual abuse by a current partner. (Of course, in this instance the women were victims, not perpetrators.) This outcome suggests that these households were stressed by their social adaptation to the United States, economic situation, and heightened marital tensions. Declining physical abuse rates suggest that these problems subsided over time. It is interesting that no similar pattern was found for verbal abuse or threats. Law enforcement practices also were assessed by examining the disposition of arrests among immigrants and the U.S.-born. To reiterate, the U.S.-born had higher overall arrest rates, and Mexican immigrant women had negligible criminal arrests. Among immigrants arrested, almost half were detained for driving under the influence of alcohol (DUI), whereas one-quarter of the U.S.-born were detained for this reason. The proportions that proceeded to prosecution were similar among immigrants and the U.S.-born (about 40 percent). About half of U.S.-born women were convicted, as were about two-thirds of U.S.-born men and three-quarters of immigrants. The higher immigrant conviction rates may be due to a greater likelihood of conviction for DUI. Incarceration rates were similar for the U.S.-born and immigrants (about 80 percent of those convicted). These comparisons do not suggest a criminal justice system bias favoring or disfavoring immigrants. The only issue of note was the relatively high arrest rates for DUI among immigrants. (See exhibit 2.) A second way to look at this issue is to compare U.S.-born rates with immigrant rates for persons with a diagnosis of drug abuse or dependence who were arrested. Again, there were no differences between immigrants and the U.S.-born, with about half of each having been arrested in the year before the survey. Although this information reveals nothing about fairness in law enforcement practices toward Mexican-born people as a group, immigrant status per se does not appear to inordinately increase arrest rates. Overall these results correspond with the analyses of two decades of institutionalization rates. Butcher and Piehl estimated that "if natives had the same institutionalization probabilities as immigrants, our jails and prisons would have one-third fewer inmates."[3] Therefore, they observed, immigrants were "assimilating to the (higher) criminal propensities of natives." Implications for Policymaking The seemingly inexhaustible demand for mobile, low-cost labor in the United States will continue to attract millions of Mexican nationals. Many will settle in this Nation, begin new lives, and start families. What do the results of this and other studies tell us about their impact on the law enforcement and criminal justice systems? --The INS faces immediate resource problems and policy issues regarding physically controlling the border, limiting production and distribution of fraudulent documents, and regulating employers. --Narcotics traffickers will continue using relatively small groups of individuals, including established Mexican-American gangs, to foster drug importation and distribution, and immigrant enclaves will be used for clandestine marketing. The problem in this instance is not primarily immigrants but the infiltration of the immigrant stream by traffickers to conduct their business with less risk. --Despite low rates of arrest, drug use, and violent behavior among immigrants, their greater numbers are increasing the burden of law enforcement, judicial, and correctional resources. This is an example of a low crime rate becoming a major problem when the population increases dramatically. --The major impact on the criminal justice system is long range: The problem of delinquency and drug use among Mexican-Americans will vastly increase in magnitude if current trends continue. People who are born in the United States or who enter the United States during childhood or adolescence will have much higher rates of delinquency, arrest, and substance abuse in their lifetime than adult immigrants. The intergenerational shift to higher rates was concentrated among U.S.-born Mexican-American women who were arrested and have substance abuse problems. Without major changes in the educational and income structure of Mexican-Americans, these statistics on crime and substance abuse can be predicted with a high degree of certainty. California juvenile arrest data show that, as early as 1993, Hispanic youths had 100 percent more felony arrests than African-Americans and Hispanic youths were involved in more than half of all status offenses reported in the State.[4] Perhaps the most powerful finding from the growing body of research about Hispanic adolescents and adults is that poverty is a much more significant factor in increasing criminal behavior and drug use within U.S. communities among Hispanics born in the U.S. than among immigrants. Long-range solutions must address core determinants such as education and academic achievement and the vitality, safety, and cohesiveness of low-income communities. Although school-based prevention programs to reduce drug abuse and crime are important, their effectiveness depends on improving key indicators of population and community prosperity. The profile of intergenerational economic mobility for Mexican-Americans indicates a substantial improvement in income between the immigrant generation and the first generation born in the United States.[5] However, there is no improvement in the second generation's median income. No doubt this stagnation is attributable in large part to the lack of secondary school completion among 35 percent of the Mexican-American population. The second generation is isolated in low-income areas, is at high risk for drug abuse and criminal behavior, and is more likely than Mexican immigrants to come in contact with law enforcement, criminal justice, and correctional systems as juveniles--and later as adults. Question-and-Answer Session Luis A. Payan, George Washington University, Washington, D.C.: These are interesting observations, but what do they mean? Have you done any thinking on a more philosophical level? What do they mean for the economy, for politics, for justice issues? At the end of your talk you began to address the causes: not enough Mexican-Americans go to school and on to higher education--perhaps they don't graduate. I'm sure there are other causes. Is it because this is a relatively violent society? Is it because they acquire these traits from society? And second, what about the solutions? What does it mean for us who are working in the different communities--justice communities, immigrant communities, and so forth? What are we to learn from this? W.V.: We are a violent society; we have the highest rates of violence of any Western industrialized nation. We have the highest rates of substance abuse, experimentation, and addiction of any society on earth. Immigrants move from a society that is very different and has minimal illicit drug use problems. Even though there is trafficking in Mexico, the levels of drug use are very low, except in the border regions, where trafficking is most intense and there is a high confluence from the two sides. I have thought a great deal about the question of whether it is possible to intervene in this process meaningfully, outside basic economic/educational determinants. I don't think you can do it without engaging the educational and economic systems, but is there something you can do above and beyond addressing the problems that create communities of vulnerability? Can you do something about the issue of weakened families and social networks? I have been asked this in forum after forum. I'm caught in the situation of judging whether this is an existential dilemma of people changing societies and changing cultures. Must they adapt to the new environment? Is there an iron law that says they must conform to American expectations and norms (which includes such things as higher levels of drug use and crime)? Or can we find a way to stop this process-- through carefully thought-out interventions at multiple levels--by buttressing these families' strengths, values, and methods of socializing children to be more law abiding adults? That is the next concrete step. The Substance Abuse and Mental Health Services Administration, National Institute on Drug Abuse, and National Institute of Mental Health experiment with those kinds of approaches. Some of the issues inherent in such approaches include the scope of interventions possible, the generalizability of these interventions, and the question of how to get them accepted and implemented throughout the Nation. Enormous resources would be involved, along with the need to integrate Federal Government incentives and local and State coordination to implement, disseminate, and maintain these programs over the long haul. We have yet to see a track record that proves it can be done in the United States (not just for Mexican-Americans but for adolescents from all groups). We need to develop interventions that work, are based on scientific evaluation, and can be implemented by people other than the experts who designed them. Resources to sustain these interventions must be available over decades because it takes decades to change a culture. Carol Wilder, U.S. Department of State, Washington, D.C.: In underdeveloped countries all over the world, people have left the rural, less densely populated areas and poured into urban areas. They leave behind the social constructs that help them obey societal rules. They are "at sea" in the cities and their crime rates soar, especially among the second generation. This pattern does not occur just among Hispanics moving into the United States and is not a result of the "evilness" of the United States. The breaking up of family and community constraints is, I believe, the prime determinant. We also need to look at poverty; we need to look at the two factors you talked about--education and community material well-being--so that people have opportunities and can break out of negative environments. I think that is much more important. I think your last suggestion of working to maintain the old culture, keeping what's good and helpful, is a losing battle. I think this other is a far, far greater determinant. W.V.: Research has definitely shown that the healthy kids are the bicultural kids, the kids who have mastered and positively identified with both cultures--especially Mexican-Americans in the United States. The issue of positive identity development for Mexican-American, African-American, and Puerto Rican children is fundamental. Each must perceive his or her color positively, not internalize self-hate, and not accept external definitions of self. Color variation should be normal and positive. I agree that there are many examples around the world where populations go through this process. However, if you look at the Mexican example (other than Mexico City), crime rates have not increased in the other cities the way they have soared among Mexican immigrants in the United States. This issue is becoming more confusing because many migrants return to Mexico from the United States and take back a lot of U.S. behaviors and styles. Mexico is beginning a rapid transformation in expectations for both men and women. Richard Stana, Administration of Justice Issues, General Accounting Office, Washington, D.C.: What is your opinion about the Los Angeles Police Department's "Rampart" scandal? Is it an isolated incident? Do you see task forces going "across the line" like that in other cities? What should local, State, and Federal law enforcement do to make sure that ethnic groups get a fair shake in the criminal justice system? M.V.: That breaks down into two issues. One has to do with the aggressive culture of law enforcement in Los Angeles, which has been well known for a very long time by those who have lived there. I grew up in East Los Angeles so I can tell you from firsthand experience. The extent and magnitude of the problem is different from place to place. Places like San Diego have done a very good job of overcoming some of those problems. Other places like Los Angeles, perhaps because of the sheer size of the police department, have had much more resistant enclaves. We see the same thing in New York City. Large police departments in general have difficulty implementing new goals and standards of conduct. It is difficult to maintain consistency in professionalism, especially in segments of the department that are given the "leading edge" in tough enforcement practices, such as gang and narcotics enforcement units. High morale and professional standards must be reinforced in these units especially. Additional support for drug courts, drug rehabilitation, and educational programs in the correctional system is needed to help offenders. Otherwise, juveniles will be released and involved in criminal activity again. A proposition was passed in California during the 2000 election that made offenders ages 14 and older criminally liable. This is a very serious problem that will disproportionately bring more and more Mexican-origin youths into the criminal justice system. They will fill the adult system as well because they will graduate very early in the criminal justice system and ensure a population for the adult prison system. Doris M. Provine, National Science Foundation, Washington, D.C.: Based on your arrest and dispositional data, it sounded like you said that the criminal justice system is not treating Mexican-Americans differently from the native-born. However, I noticed there were some interesting differences between the two groups regarding arrested persons who were illegal drug users. Did you conclude that there is less drug use among Mexican-Americans? Can you speak about who gets arrested and for what? Are there other differences and similarities within your arrest data? W.V.: Frankly, the arrests of immigrants are so dominated by DUI offenses (50 percent) that the other categories don't provide much of a comparison. The U.S.-born are more likely to be involved in traditional property crimes or crimes against a person than are the immigrants. When the DUI rate is that high, it completely monopolizes any profile. That is not necessarily a negative, because DUI arrests may keep people away from criminal behavior and offer the opportunity for rehabilitation. They also keep people out of situations that lengthen their criminal sentences and produce three-time losers. The difference in my data between foreign-born and U.S.-born arrestee rates is 30 percent. The people who study institutionalization say the difference in institutionalization rates between the foreign-born and U.S.-born is about one-third. These two sets of findings match very closely. Doris M. Provine: What about the drug war? Could it be that the drug war is more aggressive toward, say, African-Americans and other minorities? W.V.: I don't think you can make any direct connection between the impact of the drug war and the data I have presented. My data are not from the border or places where primary importation takes place, such as Tijuana, San Diego, Juarez, and El Paso. My data are from areas removed from the border, in which there obviously is illicit drug traffic taking place. But it's clear that the foreign-born population is not participating as consumers to the same degree as Mexican-Americans. This is different from saying that narcotics traffickers are not using these communities to hide out, to be invisible in setting up manufacture and transportation of, for example, methamphetamine. I think that is in fact going on. I think that communities are aware of these things, but they are not participating in them. Marianne Pieper, Office of Community Oriented Policing Services, U.S. Department of Justice, Washington, D.C.: You stated in your response to an earlier question that the most successful people had contact with both cultures. What is the impact of not allowing young children to speak in their native tongue (as in California elementary schools)--and not allowing them to learn English? What happens when we ask them to forget their native history? When we say, "You're not allowed to speak in your native tongue in school," does that have an effect? W.V.: In my mind, it has a tremendous impact symbolically. There's no question that the culture is going to be reinforced simply through population growth and the continuing immigration of Mexican-Americans. The media now offer the Spanish language on TV as well as on the radio, and last year was "Hispanic year" in the music industry. I think there is not a problem at the level of cultural icons and cultural reinforcement through the media. However, if you tell people, "Your language will no longer be taught," does it send a signal that they're worth less because they are from the culture represented by that language? On the other hand, I know from research I've done that the primary problem faced by adolescents who come into the country is language, a problem usually overcome in their first 4 or 5 years if they come as children. Mastering English quickly is to their advantage. This, however, begs a different question for a different seminar: Does simultaneously learning Spanish and English enhance or reduce the likelihood of learning effective English language skills and vocabulary? Victor Stone, Criminal Division, U.S. Department of Justice, Washington, D.C.: Could it be that your data reflect that the immigrant population is trying to not be visible, particularly because of the way our laws currently are structured? The INS may not arrest them, but that doesn't mean they don't fear that every traffic stop is going to turn into a deportation. Therefore, while they are immigrants, they are trying very hard not to be sent back. They came here to raise their standard of living, so they are very motivated to stay here. If an event scares them (even if it doesn't turn into an arrest or a conviction), they may disappear into another community to be less visible or go back to Mexico for fear that the system will deal with them unfairly. However, their children who are born here know they are U.S. citizens and can't be sent back; they don't have to remain invisible and they haven't experienced the different standard of living. I think this has been true for all immigrant groups, historically, in this country. The first generation born here doesn't know what the previous generation left, so the first generation is not as motivated to get ahead. Have you or do you plan to elicit that as a factor? W.V.: I think that is a very astute observation. It would be very difficult for me to say whether it's true. To some extent it probably is true. We also studied risk factors for depression among these populations in California. One of the most profound risk factors for depression we found among the U.S.-born (yes, the U.S.-born!) was fear of detention by the INS--fear of being asked to show papers, being taken away, or having to prove U.S. citizenship (or even worse, accidentally being sent to Mexico and having to make your way back somehow). So the cultural specter of enforcement affects both the U.S.-born and the foreign-born. However, only for immigrants is there danger that a contact with the criminal justice system could actually lead to deportation. This has been part of the culture of California from the mid-1990s to now. Notes 1. Schmidley, A. Dianne, and Campbell Gibson, Profile of the Foreign-Born Population in the United States: 1997, Current Population Reports, Series P23-195, Washington, D.C.: U.S. Bureau of the Census, 1999. 2. Vega, William A., Bohdan Kolody, Sergio Aguilar-Gaxiola, Ethel Alderete, Ralph Catalano, Jorge Caraveo-Anduaga, "Lifetime Prevalence of DSM-III-R Psychiatric Disorders Among Rural and Urban Mexican-Americans in California," Archives of General Psychiatry 55 (1998): 771-778. 3. Butcher, Kristin F., and Anne Morrison Piehl, "Recent Immigrants: Unexpected Implications for Crime and Incarceration," Working paper no. 6067, National Bureau of Economic Research, Cambridge, Massachusetts, 1997. 4. California Department of Justice, 1993 juvenile justice statistics, unpublished data, Division of Law Enforcement Information Center, California Department of Justice, Sacramento, California, May 1994. 5. U.S. Bureau of the Census, Population Profile of the United States: 1997, Current Population Reports, Series P23-194, Washington, D.C.: U.S. Bureau of the Census, 1998. Exhibit 1: Conduct Problems by Gender and Birthplace Exhibit 2: 5-Year Criminal History of Arrestees by Gender and Birthplace ------------------ Reducing Gun Violence: What Works, What Doesn't, What's Promising Presentation by Lawrence W. Sherman Professor of Human Relations Fels Center of Government University of Pennsylvania April 5, 2000 Washington, D.C. The year 2000 has become a crucible for gun control. This year, more than ever before, the Nation has moved toward a policy of making guns "safer," with only "safer" people having guns. The Smith and Wesson agreement,[1] State legislative initiatives, and the President's Federal legislative proposals have captured the public imagination with this gun safety strategy. The continuing tragedies of children shooting children have made the "safe guns, safe people" approach seem all the more necessary and potentially effective. The opposition to these strategies comes largely from the political right, which resists any new legislation as unnecessary. It frames the alternative as more enforcement of existing laws, which would be so effective as to make any new laws unnecessary.[2] Almost buried in the debate is the opposition from the left, which advocates the policies of all other nations with advanced economies: virtual bans on the possession of handguns, with tight registration and control of a limited number of long guns.[3] As the window of opportunity for new gun policy grows wider, the year 2000 also has become a crucible for science and its role in making public policy. For as the gun debate has escalated, research on the effects of various gun policies has been left far behind. To the extent that research has been cited at all, it has appeared in the usually suspect pattern of selective invocation of the mantle of science to support one argument or another. Far too often, there simply has been no research to cite on the major policy proposals. This presentation reviews the research we have, the research we do not have, and how we can use the research we have to reduce gun violence. Also, new legislation supporting the research desperately needed is suggested. For while many experiments in enforcement strategies can be conducted without new legislation, most legislative proposals cannot be adequately tested without actually enacting legislation. It is smart policy to treat new legislation as an experiment and design it to optimize what is learned from each new law. Two Kinds of Research: Epidemiological and Experimental Two kinds of research methods are covered in this review: epidemiological and experimental. It is the experiments, or what some call quasi-experiments, that form the basis of this presentation and the University of Maryland's 1997 National Institute of Justice (NIJ) Report, Preventing Crime: What Works, What Doesn't, What's Promising: A Report to the United States Congress.[4] The program evaluations reviewed for the report fall into the category of "impact" research, which measures the effects of programs and practices on measures of crime--in this case, violence committed with guns. By contrast, the epidemiology of gun violence--which traces the patterns of risk factors associated with its incidence--provides the research essential for an equally important part of policymaking: "imagination" research. By this I do not mean that the research results are fabricated--although that can happen. Rather, seeing patterns among the risk factors for gun violence can prod the imagination to invent new, more effective policies than any we now have or any we are debating. The Parable of the Screwworm The parable of the screwworm helps to make the connection between imagination and impact. One of the unsung heroes of our often-maligned Federal Civil Service, Edward Knipling, died in April 2000 after a lifetime of spectacular achievement.[5] As both an epidemiologist and experimentalist in the U.S. Department of Agriculture (USDA), he led the way in eliminating the screwworm in North America, as well as combating the tsetse fly in Africa and the Mediterranean fruit fly. His targets were a cause of great human misery as well as a major loss of food supplies. The screwworm sought out wounds on any warmblooded animal, including humans, and could cause death in a few days. Knipling's epidemiological research found that screwworm plagues were fostered by male worms mating many times but female worms mating only once. This finding prodded his imagination. If sterile males used up each female's one-and-only mating opportunity, he reasoned, there would be no offspring. If he could figure out a way to sterilize enough males, he would be able to drive the species into extinction. But when he proposed the idea to his superiors in 1938, they told him his idea was crazy. Fifteen years later, Knipling procured an old Army x-ray machine and conducted experiments showing he could sterilize male screwworms. He then released large numbers of them on Sanibel Island, Florida. Although the species quickly disappeared on the island, new worms from the mainland soon took their place. When the Dutch government offered the more isolated island of Curacao, Knipling released almost 2 million sterilized male worms over 3 months. This treatment completely eradicated the pest. At that point the USDA set up full-scale eradication programs in Florida and the Southwest, using giant sterilization factories to engineer the screwworm into extinction. Within 12 years, the pest was eradicated in the United States and most of North and Central America. Applying the Parable to Gun Violence Imagine what we might do with a similar strategy in reducing gun violence. We might notice epidemiologically, as Senator Daniel Patrick Moynihan has said, that all gun violence is caused by bullets. A State Governor or legislator might champion the hypothesis of ammunition control and pass a one-bullet-a-month law. A State official might cite the epidemiological evidence that death rates appear much higher from larger caliber handguns[6] and ban the sale of large caliber bullets altogether-- anything above a .22, for example. That same State might couple the restrictions on bullet sales with an ammunition buyback program, reducing the supply of bullets in the State. Then the State might make only modest reductions in gun violence relative to other States, largely from what is called the "Sanibel Island effect": the infiltration of bullets from other States by a thriving black market, just as we see today in States with strict controls on handgun possession. That experience, however, could ultimately lead to national ammunition control and a stunning drop in gun violence. All of this, of course, is speculation. What is not speculation is the fact that previous public health successes have always linked the policy intervention to the epidemiology of the problem. Polio vaccines have been given to children, not adults. Typhoid prevention was aimed at the water supply, not beer. Any effort to assess what works in gun policy must begin with the epidemiology of the problem, before any predictions of policy impact on the target population. For even if we have great successes with one selected target population, we may do little about the larger problem if the population we select is at low risk and the populations we ignore are at high risk. From this perspective, it is appropriate to use epidemiology to judge the "safe gun" strategies. These strategies seem aimed at middle-class gun owners who would use a triggerlock as conscientiously as they use their automobile safety belts. Even the built-in triggerlock of the Smith and Wesson agreement can be turned off and the guns rendered fully useable. In addition, a "personal gun" can be abused by the person authorized to use it. From an epidemiological perspective, the premise of safe-gun strategies is the corollary of safe-people strategies. If guns are safe because only safe people can activate them, a great deal hangs on the current definition of safe people. That boundary between safe and unsafe is vividly illustrated by firearms abuse by very young children, but it is not supported by epidemiological data. Epidemiology and What Works The epidemiology shows that current legal boundaries between people declared safe and unsafe for gun ownership fall very wide of any empirical measure of "safety." Since 1968, the major distinction has been between people convicted of a felony and all others. Additional bans on gun ownership by the mentally ill and people convicted of domestic violence misdemeanors have been hard to enforce, given the absence of needed online databases for screening gun buyers at the point of sale. In addition, although in approximately 18 States gun ownership by people convicted of certain misdemeanors involving violence, alcohol, or drugs is now barred, the Brady law and the battle over the instant background check have focused on the no-felony-conviction definition of safe people. Yet, by that definition, the majority of crimes with guns are committed by people who are legally safe, law-abiding citizens for purposes of current gun ownership policy. Samples of people arrested for using guns in crimes consistently reveal that the majority have no prior felony conviction. The Bureau of Justice Statistics reported that in 1992, two-thirds of felony weapons offenders had no prior felony conviction.[7] My review of data on people arrested for using a gun in a crime in Indianapolis in 1994 found an even higher proportion of nonfelons in the sample: 75 percent. In Prince Georges County, Maryland, my research found that half of the offenders arrested for using guns in robbery and assault in 1996 had never been arrested before for any offense, let alone a felony with a conviction. While people with felony convictions are indeed at higher risk than most people of committing a crime with a gun, that does not mean they commit most gun crimes. The key epidemiological fact is that most gun crimes would still occur even if every convicted felon in the United States were shipped to Australia (not just barred from legal gun ownership). This means two things. One, using a felony conviction to define "unsafe people" allows gun crime to happen at a much higher rate than it would be by using a more epidemiologically based definition. Two, the use of a gun in crime cannot be predicted simply from previous criminal history, and strategies other than restricting sales to safe people must be used to reduce gun violence. Dr. Garen Wintemute and his colleagues offer evidence for a more accurate definition of high-risk criminal records. In California, sales of guns to people with at least one prior misdemeanor conviction were six times more likely to be followed by a violent offense than sales of guns to people with no prior criminal history. Those who had two or more prior misdemeanor convictions on the day of the sale were 15 times more likely than the people with clean records to be charged subsequently with homicide, rape, robbery, or aggravated assault.[8] Yet, even if all misdemeanants were barred from buying guns, the United States would still have the highest rates of gun violence of all advanced countries, given that a majority of gun-using criminals have no prior convictions. The most important epidemiological fact is that gun violence is geographically concentrated in the areas of greatest inequality in the Nation--the hypersegregated poverty areas of inner cities. Half of all homicides occur in the 63 largest cities, which house only 16 percent of the population.[9] Most of those homicides are committed with handguns, often obtained illegally. Ample epidemiological data show that the greater the density of guns in a population, the greater the level of gun injury and gun death, other things being equal.[10] Thus the key epidemiological question for any new gun policy is whether it will increase gun density in areas with the greatest gun crime. The great epidemiological danger of the current safe-gun proposals is that they will create a large legitimate market for new, improved guns, leading to substantial increases in gun ownership and gun density. Moreover, according to tracing data from the U.S. Bureau of Alcohol, Tobacco and Firearms (ATF), new guns appear far more likely to be used in crime than old guns. The 1997 ATF analysis of crime guns traced in 17 cities found that the percentage of confiscated crime guns less than 3 years old ranged from 22 to 43 percent for juveniles and 30 to 54 percent for youthful adult offenders.[11] Some analysts have used the new-guns pattern to argue that a ban on new handgun sales would reduce gun violence, especially if accompanied by a law banning possession of handguns like the one in force in Washington, D.C. Others cite the predominance of high-caliber guns in homicides[12] and suggest that more selective bans on more lethal weapons could be attempted. Whatever the merits or failings of these ideas, they would at least respond to the key epidemiological facts: most gun crime is committed by people with minimal criminal records, who are unlikely to be hampered by safe-gun technology or current proposals for restricting sales of new guns. From Epidemiology to Experiments There is no way, of course, to apply such epidemiological analysis directly to predict the results of a proposed policy. It may be possible to say that the policy will miss the clearest risk factors. But only when a policy is implemented is it possible to assess its effects, both intended and unintended, good and bad. Ideally, such information would lead to revisions or refinements of the policy. Approaching new legislation in the spirit of experimentation would take this into account. Unfortunately, the ideological and political stakes in such legislation create a strategy of "ready, fire, aim," for which advocates must say they are convinced they are right before they enact it. They may indeed be convinced, but that conviction is based on faith rather than fact. Using facts more effectively to combat gun violence would require us to enact all new laws for temporary, preliminary assessment periods, accompanied by a rigorous and comprehensive evaluation, followed by a debate on the facts of that evaluation as well as on the theories behind each law. The Federal Government could even provide incentives for the States, as the laboratories of democracy, to conduct such legislative experiments. In the meantime, it is possible to review the available evaluation research, not just in terms of what works, but in terms of how much it works; that is, what proportion of gun violence would be affected by a national implementation of the program producing the same results. Reducing Gun Violence The 1997 University of Maryland report used a moderately high standard of evidence for deciding what works to prevent crime: at least two separate studies with a minimally adequate control or comparison group reporting crime prevention effects.[13] In addition, there could be no conflicting evidence at the same level of rigor. Where only one such study was available, we classified the program as promising. Similarly, two or more negative findings were needed to put a program on the list of what doesn't work, again without conflicting evidence. This standard yields a very short list: two policies that work, one that doesn't, and several that are promising (see sidebar). What Works: Gun Patrols and Background Checks The two programs known to work are epidemiologically focused on the high risks of gun violence. One focused on high-risk times and places and the other on high-risk people. The first program required uniformed police patrol of gun crime hot spots in areas with homicide rates many times above the national average. The second program required background checks before selling guns to ensure that buyers were "safe" (by some definition). Although the patrol strategy apparently can address a far larger portion of all gun violence than the background check strategy, the effects of background checks could be greatly magnified by matching the definition of unsafe people to the epidemiological data. Uniformed Gun Patrols. The first formal test of uniformed police patrols against guns was the Kansas City Gun Experiment in 1992. Police in a high-crime area worked overtime to increase gun seizures by 65 percent and found a 49-percent reduction in crimes committed with guns.[14] This NIJ study found no change in either gun seizures or gun crimes in a similar area several miles away. A modified replication of the Kansas City study was funded by NIJ in 1996 and carried out in Indianapolis. Two target areas either maintained or increased the level of gun seizures, while gun seizures dropped in a comparison area by 40 percent. According to the Hudson Institute's evaluation by Ed McGarrell and his colleagues, gun assaults, armed robberies, and homicides dropped by 50 percent in one area and 25 percent in the other, even as those crimes rose 22 percent in the comparison area and remained constant citywide.[15] (Note that the result for the second target area is a new finding, one that modifies an earlier reported conclusion that the strategy had only worked in one of the two target areas.) An evaluation of the strategy of uniformed police patrolling high gun crime areas looking for illegally carried guns was reported recently in the Journal of the American Medical Association. In this study, gun homicides in two Colombian cities, Bogota and Cali, declined by about 14 percent whenever gun carrying was banned, and police mounted special patrols using methods similar to those employed in Kansas City and Indianapolis.[16] Gun crime was not reduced as much as in the U.S. experiments, a fact that may be related to the extraordinarily high rate of guns seized per capita in the United States during the experimental periods. For example, the rate of seizure in Kansas City was 42 times higher than in Cali; no gun seizure data were available for Bogota.[17] The Boston Police Department's well-known reduction in homicides in the early 1990s was statistically related to a major increase in weapons arrests, in a pattern that mirrors the national trend of increased weapons arrests and decreased homicides.[18] The same pattern is found in New York City. The change in the ratio of weapons arrests to homicides is the key change around the time that homicide rates fell. As people carry fewer guns, fewer gun arrests are expected as well as fewer homicides. Plain-clothes patrols and drug enforcement recently have been associated with controversial cases of police-citizen encounters. The actions of these plain-clothes officers should not be confused with the results of uniformed patrol experiments. The Kansas City, Indianapolis, Colombia, and other data show the effects of what officers do in uniform, with full public awareness that they are police. In 1997 the Prince Georges County, Maryland, Police Department mounted such a program with a major increase in traffic arrests--all the more impressive, given its reduction in complaints of excessive force at the same time. There is no logical conflict between police (1) enforcing gun laws more frequently (sometimes misnamed "aggressive patrol") and (2) being polite to citizens and explaining what they are doing as they go along. Criminal History Checks. Background checks for gun buyers also are effective at reducing gun violence. The Brady law is often acclaimed as a success because it has stopped more than 300,000 people from buying guns illegally in a 5-year period nationwide.19 The true test of its success, however, is not the number of gun sales blocked but the number of gun crimes prevented. While the gun homicide rate has been dropping steadily nationwide since the Brady law became effective, the total number of guns sold, the number of federally licensed firearms dealers, and the reported level of crack dealing in some major cities have also dropped. Because we cannot separate the possible causal connections between gun homicides and the simultaneous changes of these and other factors, we must look at microlevel data for a more valid assessment of the effect of presale clearance strategies. Two impact evaluations meet the threshold of valid research designs using the University of Maryland Scientific Methods Scale. One study compared the subsequent crime rates of convicted felons who were blocked from buying a gun with the subsequent crime rates of people charged with (but not convicted of) felonies who were allowed to buy guns. If convicted felons are more dangerous people, we might expect them to have higher rates of crime even if they were barred from buying a gun. But according to the sample of 2,640 California background checks examined by Mona Wright and her colleagues, the 170 convicted felons who were prevented from buying new guns were 18 percent less likely to be charged with a gun offense over the next 3 years than the gun buyers who had felony arrests but no convictions. In other words, using arrested felons as a control group, gun crime by convicted felons appears to have been reduced by the background check policy.[20] A second impact evaluation using a different research design reached the same conclusion about the effectiveness of background checks. David McDowall and his colleagues found that when Florida adopted a mandatory waiting period and background check for handgun purchases, homicide rates dropped relative to controls.[21] Taken together, these two studies give background checks a secure status as a means of reducing gun violence. Although they do not necessarily reduce gun availability in the places where gun crime rates are highest, it is at least possible that background checks do so indirectly. Gun smugglers and straw purchasers, for example, may have found background checks too difficult to deal with and dropped out of the business. There is an immediate implication of placing background checks on the list of what works. That implication is closing the loopholes related to gun sales. If background checks reduce crime committed with guns sold in licensed gun dealerships, it is a reasonable hypothesis that they would do the same for gun shows and third-party gun transfers. About 4 percent of gun owners surveyed in 1994 had purchased their guns at gun shows, and 65 percent had purchased their guns in stores subject to the Brady law. Thus one-third of all gun sales are currently excluded from Federal law on background checks, either at gun shows, through classified ads, or in sales among friends and family.[22] Closing the gun show loophole alone may not do very much, but requiring all nonstore sales to be recorded and subjected to background checks could broaden the impact of the Brady law. What Doesn't Work: Gun Buyback Programs The program that is best known to be ineffective is gun buybacks, at least in the United States. In three separate, moderately strong scientific evaluations, there was no reduction in gun violence following the purchase by police of large quantities of guns. Richard Rosenfeld's evaluations of two separate gun buybacks in St. Louis examined a 1991 program that bought 7,500 guns and a 1994 program that bought 1,200 guns. Neither showed any reduction in gun homicides or assaults relative to the same offense types committed without guns.[23] A similar evaluation of a gun buyback in Seattle found no reduction in homicide and some evidence of an increase.[24] From an epidemiological perspective, buyback programs receive low marks for relating resources to risk factors. Nothing in the structure of gun buyback programs focuses the intervention on the risk. Guns are bought from anyone, regardless of where they live or whether the gun was readily accessible to people at high risk for crime. These programs are often justified rhetorically by saying that "every gun bought back is a potential life saved." Yet not all guns are at equal risk of being used in crime, a risk that varies widely by geographic area, type of gun, recency of manufacture, and criminal record of the gun owner. The ineffectiveness of the gun buyback programs is all the more important because this research has been ignored. These findings have been in the public record for some time. They were included in the 1997 Maryland Report to Congress and in the 1998 summary of that report. There is, to my knowledge, no contradictory evidence purporting to show that gun buybacks can be an effective policy for reducing gun violence. In addition, these programs are extremely expensive, usually costing hundreds of thousands of dollars. Yet as recently as the fall of 1999, another Federal program was launched to encourage local agencies to spend many millions of dollars on buybacks in public housing authorities. Such a program might conceivably have some effect on gun violence if it was limited to residents of the small percentage of all public housing projects nationwide that suffer gun violence problems. But based on the citywide program results, that seems unlikely. What's Promising: Gun Bans The available evidence suggests that several kinds of gun bans have been effective, although none of the findings has yet been replicated with rigorous field studies. One is a blanket or near-blanket ban on possession of handguns. New York City first enacted a near-ban on the purchase of handguns in 1911 with the Sullivan Law. While this law has not been formally evaluated, New York did have a lower homicide rate than many other big cities for most of this century. The law clearly alters the dynamics of the gun market, with strong evidence of the Sanibel Island effect: 85 percent of New York City crime guns traced by the ATF were imported illegally from outside the State.[25] Handgun Ban in Washington, D.C. When the District of Columbia passed an even more restrictive handgun ban than New York's in 1976, it created an opportunity for a detailed evaluation of its impact on gun crime. Colin Loftin and his colleagues examined the trends in gun homicides in Washington, D.C., and its surrounding communities before and after the change in the law. The 25-percent drop in firearms homicide that followed was not matched by a drop in other types of homicide, nor in an increase in homicide by other means. Nearby areas of Maryland and Virginia had no change in either firearms or nonfirearms homicides. The reduction in Washington, D.C., was sustained until the crack cocaine epidemic in 1987.[26] Yet it is possible that the ensuing increase in homicide that pushed Washington, D.C., to claim the highest murder rate in the country might have been even worse without the handgun ban having been in effect all those years. National Assault Weapons Ban. The 1994 national ban on the sale of newly manufactured assault weapons of specific manufacture and design was the subject of a national evaluation mandated by the Congress when it enacted the legislation. The NIJ evaluation by Jeffrey Roth and Christopher Koper found that requests for ATF traces of the banned weapons after use in crime dropped by 20 percent in the first year, steeper than the 10-percent drop in all homicides and in trace requests for all other gun types. Moreover, gun murders dropped 11 percent below projected levels in the 38 States that had not previously passed a similar ban but did not drop in States where such weapons were already banned.[27] There was also a reduction in the rate at which police were murdered with guns. Finally, it is worth noting that since 1934, U.S. restrictions on the ownership of fully automatic machine guns have been associated with the extremely rare use of such guns in crime. Summary This discussion of what works has focused on a limited but critically important criterion. We have considered not what works to reduce all crime or even to reduce violent crime. Instead, the standard for what works in this list is what works to reduce gun violence. Whatever the merits of such claims as "more guns, less crime," they are ultimately beside the point from a public health perspective. America's crime rates, apart from gun violence, are quite modest by international standards. Americans are less likely to be burglarized in Washington, D.C., than Australians are in Canberra, and other crime rates are now higher in England than in the United States. As Franklin Zimring and Gordon Hawkins have argued, crime is not the problem.[28] The problem is gun violence and the crippling injuries and deaths that it causes. Using standards for impact evaluations that are substantially lower than those used by the Food and Drug Administration to approve new drugs, we have still produced a very short list. Only two strategies have acquired scientific evidence that rises to the level of what works: uniformed police patrols in gun crime hot spots and background checks for gun buyers. Only one strategy has been certified as ineffective: the expensive gun buyback programs. The only promising strategies are bans on specific types of guns, which are of course politically volatile. Nonetheless, this list provides useful guides to action. If uniformed and polite gun patrols as well as background checks were significantly increased, less gun violence might quickly be achieved. How much less remains to be seen. It is important to note that far more Federal funds could be invested in these patrols in the 1,000 or so census tracts at highest risk, and far more gun sales could be subjected to background checks. Thus the available research points the way to major policy changes that could work wonders. Ironically, in a year in which the policy choices have been framed as more laws versus more enforcement, the research evidence suggests that both enforcement and legislation can make a difference. The choice between them is clearly false, for nothing makes them mutually exclusive. Police efforts to get illegally carried guns off the streets can only be aided by greater limits on who gets to buy new guns, which have been the fashion statement of choice for many young men without criminal records. In this crucible year, however, the list of what works excludes most of the proposals now on the table for reducing gun violence. Those proposals include: --External triggerlocks. --Internal triggerlocks. --Personal weapons, or "smart guns" that work for only one owner. --"Firearms fingerprinting" before sale to allow police to trace all bullets fired to the gun that fired them. --Serial numbers on more places on each gun. --A 10-shot limit on firing each gun. --A national one-gun-a-month purchase rule. This list, which is far from exhaustive, is testament to the imagination of policy analysts who have developed these plans. The extent to which they rely on epidemiological data varies widely. But the effects of these plans can only be properly measured by trying them out. Whether or not any new legislation calls for a mandatory evaluation (like the national assault weapons ban did), there will be more opportunities to learn what works. The Smith and Wesson agreement, for example, should create a major "natural" experiment, testing several of these proposals simultaneously with about 20 percent of the new handguns sold in the United States each year: --Using both Smith and Wesson guns made before the agreement and guns made by other manufacturers, the National Institute of Justice could compare the rates at which Smith and Wesson guns with internal, external, and no triggerlocks are used in crime, suicide, and accidents.[29] --Comparing the new Smith and Wesson serial number system to the old one may show differences in the rates at which police can catch gun crime offenders and possibly differences in the rates at which well-numbered and poorly numbered guns are used in crime. --The effects of a stronger trigger-pull pressure requirement can be measured in the rates of accidents involving young children. --The effects of requiring background checks at gun shows can be tested directly with the data from dealers following the new Smith and Wesson code of conduct. --The one-gun-on-purchase-day limit with a 14-day waiting period for each buyer to collect the rest of the unlimited number of Smith and Wesson guns can be used to test the effects of that rule on gun violence. If Smith and Wesson crime gun traces decline, the waiting period may have discouraged gun smugglers from buying them for transport into Sanibel Island States. Whatever opportunities these innovations may provide for experimentation, epidemiology must not be forgotten. The example of the screwworm must not be forgotten, nor such basic ideas as ammunition control. Gary Kleck, whose 1991 book on gun control policy indicts a multitude of ideas, dismisses ammunition control as well.[30] (Yet he conceded that if there were an end to ammunition, it would cause an end to gun violence.) He predicted it would not work solely because it is easy for skilled people to make their own ammunition. That ease presumes access to such elements as gunpowder, the sale of which could also be tightly restricted. Even if some people know how to make gunpowder, the obstacles to obtaining bullets would clearly rise under ammunition control. If they rise high enough, they would render useless many or most of the 200 million guns in circulation, without confrontation between police and citizens. It is always important to know, based on experiments performed, what works and what doesn't. But the list of what is known to work must never be confused with the list of what could work--if it could be tested. The list of experiments must be lengthened, and gun violence must be treated as the complex problem it is. For complex problems--whether natural ones like screwworms or human-made ones like bullets--can only be solved through systematic and long-term trial and error. Question-and-Answer Session Paul Blackman, National Rifle Association, Fairfax, Virginia: I have two questions. First, you mentioned that very few criminals use machine guns since the restrictions of 1934. Are you also suggesting that they don't use sawed-off shotguns, which were similarly restricted at the same time? Second, both the Kansas City study and the Washington, D.C., study used what might be considered inappropriate controls; that is, controls dissimilar from the places being studied--for example, Washington, D.C., was compared with the suburbs rather than the city of Baltimore, which had a homicide rate that fell more rapidly than Washington's. Can you explain why you still find those studies valid? L.S.: Sawed-off shotguns are different from, say, Thompson submachine guns from the World Wars. The potential for sawing off legally sold shotguns is clearly much greater than the potential for obtaining a Thompson, which is banned from commerce. My source on this is Gary Kleck, and I defer to his observations on it. On the second point, I think it is worth noting that you lump together the Kansas City and Washington, D.C., experiments with the same methodological point. The point about appropriateness of controls doesn't apply to Kansas City because the control area was as similar as possible: It was several miles away within Kansas City. Both areas had very high rates of homicide and driveby shootings. The major difference between them was population density; the target area was more dense than the control area. We looked at the potential for displacement of crime. Although gun crime went up somewhat in the immediately surrounding beats, it didn't do so significantly. That is what significance tests are about. The Washington, D.C., study of the ban on handguns by Colin Lofton and his colleagues comparing gun and nongun homicides had internal controls and looked at surrounding areas as potential for displacement (and also as possible controls). I think Tom Frazier might argue that Baltimore is as different a world from Washington as San Francisco is. There is no scientific basis for saying that it is a more appropriate control than any other big city in the United States. Perhaps what somebody should do is take a sample of cities with a population of more than 500,000 and check it out. That will add further fuel to this debate that has been going on since 1991, when the study was published in The New England Journal of Medicine. Frankly, you could pick at all of these experiments in one way or another. That is the nature of science, trying to figure out from each week's headlines, for example, whether hormone replacement therapy is a good idea for women over 50. You are left with the same questions in any case: How is the research done, and what can we conclude from it? All we can do is continue to do research and make the best of it. Jeremy Travis, National Institute of Justice, U.S. Department of Justice, Washington, D.C.: I'd like to note that it is gratifying that at a research forum we can have Handgun Control and the National Rifle Association sitting next to each other--and trading notes. This is wonderful! Vincent Schiraldi, Center on Juvenile and Criminal Justice, Washington, D.C.: You said there was a false dichotomy sometimes between the law enforcement approach and some of the bans on handguns or certain types of handguns, but you talked about how both are important. It strikes me that the approach used in Prince Georges County, Maryland, and New York City makes much more sense in an environment saturated with handguns than one in which handguns were banned. For example, I could maybe support squeegee men or turnstile jumpers being aggressively or assertively targeted in an environment in which many of them can legally obtain (or at least, easily illegally obtain) handguns. But where they couldn't, it might be different. I don't think it would be a false dichotomy that, if we could get control of those 65 million handguns, we wouldn't need to roust people from jumping turnstiles as if they were potential killers. I think that some of the goodwill reflected in public opinion polls represents this attitude: "Crime went down in my neighborhood. Yeah, search the guy jumping the turnstile. But if crime went down in my neighborhood without doing that, I wouldn't feel as good about it." Can you comment on that? L.S.: I think the major attraction of ammunition control for me is precisely the point you are making, that it would be a way to get around this enormous stockpile of guns in the United States that makes squeegee people or turnstile jumpers into potential killers. You only have to look at the number of homicides in New York City subways before and after they used frequent arrests of turnstile jumpers (again, you don't have to arrest people aggressively). Another NIJ study found that when people are arrested politely, they have a much lower repeat offending rate than if they are not arrested politely. In public discussions I think frequency of arrests and impoliteness of the arrest process have been totally conflated as "aggressive policing. That is, if you are making frequent arrests, you must be doing it in an aggressive manner. That may be the case in some cities; but it doesn't have to be. Law enforcement leadership can speak out about the tremendous importance of treating all citizens not as potential killers (although in a country with lots of guns--certainly in high gun crime areas--that has to be a part of the working assumptions), but as potential customers, like a major hotel corporation, where an officer says, "Have a nice day," after saying, "May I pat you down?" We have to focus on where the problem is. This is not unique to guns. In AIDS prevention, for example, we have a great deal of expenditure going to low-risk populations and not nearly enough to high-risk populations. That's partly due to the way the founders wrote the U.S. Constitution, giving, for example, 15 percent of the population control of half of the U.S. Senate, resulting in rural interests hostile to investing in cities even if that is where the problems are found. We will continue to have a very disjointed national debate because these policies look different, depending on where you sit and where you live. The U.S. Supreme Court has to continue to interpret law for the whole Nation on when the police can frisk and when they cannot. They have done some very interesting things this term about that, and I think that's fine. But the issue is not only how we enforce, but how much we enforce, where we enforce, and whether there is some way to get around that terrible problem that police reformer August Vollmer predicted when the automobile was invented, saying, "Oh no, now the police are going to have to confront all kinds of middle-class people, and we are never going to regain their friendship and loyalty." Imagine if we had a world without traffic enforcement. Michel McQueen, Nightline, ABC News, Washington, D.C.: I am curious how the uniformed patrols in gun and crime hot spots actually work. Is the goal confiscation, or is it prosecution? Do you make a distinction between juveniles and adults in these patrols? L.S.: The goal is prevention. And how it's achieved may not be as important as the fact that it is achieved with dignity and legitimacy by the police. Taking guns off the streets, deterring gun carrying, or deterring crime in general--all might require additional police in gun crime hot spots. (Incidentally, hot spots are very localized. It is not just a matter of saying "this precinct has a high rate." Within that precinct, crime mapping can tell you that 1 to 3 percent of the addresses in that precinct have the majority of the gun crime.) One of the problems we have with police is that they get bored easily. If they stay in a hot spot and nothing happens, they say, "See, I told you there is no need for me to stand here." They then go to some other low-risk place, which of course increases the risk that crime can happen at the street corner where it has happened most often in the past. Michel McQueen: How do you envision this working in practice? I'm thinking about the recent killings of public school students in Washington, D.C. There was a very emotional outcry about this. Public school students came forward at one of the public events and talked about their feelings of being unsafe. They talked about a heavy police presence, particularly in certain neighborhoods known to be saturated with guns. They see police there, but they don't see the police doing anything, so they feel unsafe. This cycle of paranoia leads to more gun carrying. I'm interested in more detail about how this actually works. Do the police just stop and frisk people? L.S.: You can't stop and frisk people without "articulable suspicion." What you can do, and they did in Kansas City, was to fully enforce traffic laws. Very often they found guns sitting on the seat in plain view. In Prince Georges County [Maryland], as Phil Pan of The Washington Post reported when the program began, they were fully enforcing traffic laws and also asking people for consent to search their trunks or glove compartments or other places, saying, "There has been a lot of gun crime in this area lately. For your safety and ours, would you mind if we checked out the other parts of the car?" Racial profiling has become a big issue in the New Jersey State Police. Racial profiling was not a policy in Prince Georges County. In fact, some comments made to the Post reporter at the time include, "I wish they'd done this a long time ago. This is going to make us more safe." There is tension between wanting the police to do something and not wanting them to do too much or to do things too rudely. I think that's what places such a tremendous burden on police in these areas and leads some people to say, "They simply can't be trusted; let's leave them out of it. Let's find other ways and not have heavy police presence because it creates this terrible risk, as in the Amadou Diallo case." We can look at the overall numbers and see what's been prevented. The New York City Police Department went from 77 killings of citizens in 1971 to 18 the year before Diallo was killed (and 13 the year after he was killed). The risk of police killing citizens has gone way down; the capacity to detect and get guns off the street has gone way up. Homicides are far below their 1971 levels in New York. When people tell me there is no progress, I'm now old enough to know that they are wrong, at least in some cases. We can make tremendous strides; we just have to keep holding on in the face of these bad events that drive policy and create opportunities to destroy good policy. We must look at the evidence in a more systematic and objective way to find out what will work best for everybody. Elizabeth Glazer, U.S. Attorney's Office, Southern District, New York, New York: Recently, particularly in light of the successes of Operation Exile in Richmond, Virginia, Federal prosecutors have been urged to do a much higher volume of felony possession cases, using Federal laws to put felons who are in possession of firearms in jail. Your numbers seem to show that felony convictions are not necessarily a good predictor of who's an unsafe person. In light of that, do you have any advice about strategies Federal prosecutors should employ to reduce gun violence? L.S.: There are two issues here. One, the 25 to 35 percent of gun crimes committed by convicted felons could have been prevented if those folks had been behind bars. Two, there may be a big chunk of gun violence that could be prevented by selective incapacitation of gun offenders under Project Exile, Operation Triggerlock, or the other Federal interventions. We can argue that there is more room for using sentencing policy with respect to gun crimes. There is some controversy in the literature about the effectiveness of mandatory sentencing policies, which is why I left them off the list. But what may be even more important is the complete disdain State judges hold for treating the offense of carrying a concealed weapon as a serious crime. The typical sentence for carrying a concealed weapon in most big cities is probation. That was what drunk driving was like in 1980, and we stopped treating it that way. If we took the risk of carrying a gun as seriously as we take the risk of driving a car drunk, we might find that we could deter gun carrying through sentencing policy as much as (or maybe even more than) through police patrols. Combining the two, in which the sentences would be substantial for carrying a gun along with the higher likelihood that the police would be able to detect a gun if you are carrying it on the street, could have two effects. One, it could reduce the rate of gun carrying; the other, it could increase the rate of violence in police encounters with citizens. If somebody knows that they are going to do time for carrying a gun, they might choose to shoot it out with the police officer. This is why we have to do experiments. We can't figure these things out in our heads. We have to test the policies and see what happens. Ted Gest, U.S. News & World Report, Washington, D.C.: Could you elaborate on your apparent dismissal of the "more guns, less crime" theory, which was an allusion to the thesis that more, legally obtained concealed weapons had some correlation or maybe causation with decreased violence? Have you determined whether that theory is flawed, whether it has no effect or negligible effect on gun violence? Should it be tested, or is the basic policy of easier access to legally concealed weapons flawed, based on your analysis of the Brady law? L.S.: John Lott's book More Guns, Less Crime: Understanding Crime and Gun-Control Laws makes me ask, what is the issue? Is it crime or gun injury? Even if we are willing to concede that the price of having a homicide rate equal to Australia's (which is 2 per 100,000, or 25 to 33 percent of the U.S. rate) is to have an Australian burglary rate, which is substantially higher than ours, I'm happy. Assume hypothetically that more guns deter burglars. If they also cause more gun crimes, why is that good? Although the focus of Lott's book is on the relationship between guns and crime rates, it has virtually no data on gun violence except for gun homicides. It is a national sample and does not look at the effects of gun-carrying laws in the few areas that have most of the homicide, making the analysis irrelevant to the policy issue of reducing gun homicide in the United States. Officer Paul Liquorie, Montgomery County Police Department, Rockville, Maryland: What findings do you have on projects with mandatory minimum sentencing for illegal possession of handguns, like Project Exile? L.S.: I'm not aware of any evaluation of prosecution at the Federal level and its impact on gun violence in a city like Richmond. I don't think a Richmond evaluation has been done. Again, my point: Let's not confuse what has been found to work (or not) with what could work (or not). We need to do a lot more evaluations. Notes 1. Dao, James, "Under Legal Seige, Gun Maker Agrees to Accept Curbs," New York Times, March 18, 2000, A1. 2. Bruni, Frank, "Bush Moves a Little Away From the N.R.A.'s Positions," New York Times, March 18, 2000, A8. 3. Zimring, Franklin, and Gordon Hawkins, Crime Is Not the Problem: Lethal Violence in America, New York: Oxford University Press, 1997. 4. Sherman, Lawrence W., Denise Gottfredson, Doris MacKenzie, John Eck, Peter Reuter, and Shawn Bushway, Preventing Crime: What Works, What Doesn't, What's Promising: A Report to the United States Congress, Washington, D.C.: U.S. Department of Justice, Office of Justice Programs, 1997, NCJ 165366. (Visit the Web site http://www.preventingcrime.org to access the report.) 5. Nagourney, Eric, "Edward Knipling, 90, Enemy of the Dangerous Screwworm," New York Times, March 27, 2000, A25. 6. Koper, Christopher S., "Gun Lethality and Homicide: Gun Types Used by Criminals and the Lethality of Gun Violence in Kansas City, Missouri, 1985-1993," Ph.D. diss., University of Maryland, 1995. 7. Greenfeld, Lawrence A., and Marianne W. Zawitz, Weapons Offenses and Offenders, Washington, D.C.: U.S. Department of Justice, Bureau of Justice Statistics, 1995, NCJ 155284. 8. Wintemute, Garen J., et al., "Prior Misdemeanor Convictions as a Risk Factor for Later Violent and Firearm-Related Criminal Activity Among Authorized Purchasers of Handguns," Journal of the American Medical Association 280 (1998): 2083-2087. 9. Computed from Federal Bureau of Investigation, Crime in the United States, 1993, Uniform Crime Reports, Washington, D.C.: U.S. Department of Justice, Federal Bureau of Investigation, 1994. 10. Reiss, Albert J., Jr., and Jeffrey A. Roth, eds., Understanding and Preventing Violence, Washington, D.C.: National Academy of Sciences, 1993. 11. U.S. Bureau of Alcohol, Tobacco and Firearms, Youth Gun Crime Interdiction Initiative: Crime Gun Trace Analysis Reports: The Illegal Youth Firearms Markets in 17 Communities, Washington, D.C.: U.S. Bureau of Alcohol, Tobacco and Firearms, 1997. 12. Koper, "Gun Lethality and Homicide." 13. Sherman, et al., Preventing Crime. 14. Sherman, Lawrence W., and Dennis Rogan, "Effects of Gun Seizures on Gun Violence: Hot Spots Patrol in Kansas City," Justice Quarterly 12 (1995): 673-693. 15. McGarrell, Edmund, Stephen Shermak, and Alexander Weiss, Targeting Firearms Through Directed Patrols, Indianapolis: Hudson Institute, 1999. 16. Villaveces, Andres, et al., "Effect of a Ban on Carrying Firearms on Homicide Rates in Two Colombian Cities," Journal of the American Medical Association 283 (2000): 1205-1209. 17. Sherman, Lawrence W., "Gun Carrying and Homicide Prevention," Journal of the American Medical Association 283 (March 1, 2000): 1193-1195. 18. Ibid.; unpublished data, Federal Bureau of Investigation Uniform Crime Reporting Program as analyzed by Matthew Holtman, University of Pennsylvania, 2000. 19. Manson, Donald A., Darrell K. Guillard, and Gene Lauver, Presale Handgun Checks: The Brady Interim Period, 1994-98, Washington, D.C.: U.S. Department of Justice, Bureau of Justice Statistics, 1999, NCJ 175034. 20. Wright, Mona A., Garen J. Wintemute, and Frederick A. Rivara, "Effectiveness of Denial of Handgun Purchase to Persons Believed to Be at High Risk for Firearm Violence," American Journal of Public Health 89 (1999): 88-90. 21. McDowall, David, Colin Loftin, and Brian Wiersema, "Easing Concealed Firearms Laws: Effects on Homicide in Three States," Journal of Criminal Law and Criminology 86 (1995): 193-206. 22. Cook, Philip J., and Jens Ludwig, Guns in America: Results of a Comprehensive Survey on Firearms Ownership and Use, Washington, D.C.: Police Foundation, 1996. 23. Rosenfeld, Richard, "Gun Buybacks: Crime Control or Community Mobilization," in Under Fire: Gun Buy-backs, Exchanges, and Amnesty Programs, ed. Martha Plotkin, Washington, D.C.: Police Executive Research Forum, 1995. 24. Callahan, Charles, Frederick Rivara, and Thomas Koepsell, "Money for Guns: Evaluation of the Seattle Gun Buy-Back Program," in Under Fire: Gun Buy-backs, Exchanges, and Amnesty Programs, ed. Martha Plotkin, Washington, D.C.: Police Executive Research Forum, 1995. 25. U.S. Bureau of Alcohol, Tobacco and Firearms, Youth Gun Crime Interdiction Initiative. 26. Loftin, et al., "Effects of Restrictive Licensing of Handguns on Homicide and Suicide in the District of Columbia," New England Journal of Medicine 325 (1991):1615-1620. 27. Roth, Jeffrey, and Christopher Koper, Impact of the 1994 Assault Weapons Ban: 1994-96, Research in Brief, Washington, D.C.: U.S. Department of Justice, National Institute of Justice, 1999, NCJ 173405. 28. Zimring, Franklin, and Gordon Hawkins, Crime Is Not the Problem: Lethal Violence in America, New York: Oxford University Press, 1997. 29. Dao, James, "Under Legal Seige, Gun Maker Agrees to Accept Curbs," New York Times, March 18, 2000, A1. 30. Kleck, Gary D., Point Blank: Guns and Violence in America, New York: Aldine deGruyter, 1991: 415. ------------------ SIDEBAR Reducing Gun Violence: What Works, What Doesn't, What's Promising What Works? --Uniformed police patrols in gun crime hot spots. --Background checks for criminal history to restrict gun sales in stores. What Doesn't Work? --Gun buyback programs in American cities. What's Promising? --Virtual bans on private handgun possession. --Bans on the sale and manufacture of new assault weapons. What's Worth Testing? --Any program addressing a known risk factor and tied to a careful evaluation, including: ----Bans on handgun sales and possession for all convicted misdemeanants. ----Bans on high-caliber handguns. ----The elements of the Smith and Wesson agreement. ----Ammunition control for all or some bullet sizes. ----Waiting periods for ammunition. ----National one-gun-a-month laws. ------------------ Reinventing Evaluation to Build High-Performance Child and Family Interventions Presentation by Heather B. Weiss Director Harvard Family Research Project May 3, 2000 Washington, D.C. The growing push for social program accountability across the private and nonprofit sectors and at all governmental levels is stimulating productive debates about the definition and role of evaluation. In addition, calls are being heard for trials of new approaches to knowledge development that emphasize organizational learning and continuous improvement. This paper examines the current transformation in the role and purposes of evaluation and argues for strategic investments in evaluation and knowledge development as part of larger systems of learning and accountability. One vision for a learning system is described, and its key functions and operations are illustrated with examples from ongoing efforts to invest evaluation resources to support learning and accountability. These examples illustrate an array of local and State efforts to build the capacity of organizations to obtain and use data that will inform and continuously improve practice and policy. The paper concludes with a brief discussion of the challenges this reorientation poses for those who fund knowledge development and evaluation as well as those who train evaluators. Growth in the Role of and Demand for Evaluation Since its origins during the War on Poverty in the 1960s, the field and profession of social program evaluation has grown substantially and been challenged repeatedly as Americans have increasingly demanded "proof" that programs work and merit the allocation of their taxes and other support. Recent polls about American willingness to invest in services for children, youths, and families, for example, show strong links between willingness to pay taxes and evidence that programs work.[1] By the 1980s, evaluation was firmly entrenched as a key ingredient in social problem solving and building public will to support social programs, and government and foundation funders invested in developing and evaluating model programs to ascertain their effectiveness. The logic behind this was to test whether a model worked under experimental or quasi-experimental conditions, and if it did, the assumption was that it could be "scaled up" elsewhere. Evaluations of multisite demonstrations were designed to address questions about implementation and effects in a variety of community conditions. Foundations conducted research and created demonstration programs assuming that the public sector would at least consider expanding successful interventions.[2] This view of the role and positioning of evaluation in social problem solving has come increasingly under question. As anyone familiar with social programs knows, unevaluated or "unproven" programs are routinely scaled up, while "proven" programs often fall victim to the "research, demonstrate, dilute, or adapt" syndrome, in which replications bear little resemblance to the original. This "model drift" and evaluation-based learning about the ways in which contextual factors (such as the richness of community resources in models that provide information and referral to other services) influence the implementation and effects of interventions have led to questions about the generalizability of research and demonstration findings to different populations and communities.[3] These realizations and questions about traditional research and development strategy led in turn to several recommendations for better learning from evaluation investments. First, some observers suggested that instead of evaluating only "flagship" research and development programs, it made sense to also selectively evaluate the "fleet" of community-based and -developed innovative programs.[4] Second, evaluators began to ask more differentiated questions--not just what works, but what works where, when, how, and why--to build a more nuanced and useful base of data with which to inform policy and practice. The development and increased use of meta-analysis (summarizing data and results from multiple evaluations across intervention areas) has also helped increase the yield and power of investments in the evaluation of social programs. By the 1980s, no one flagship exemplar was seen as doing the "evaluation duty" for all examples of its type, and public and foundation funders increasingly required some form of evaluation for new social programs and interventions. An example from early childhood programming illustrates these points. The Head Start program has typically relied on commissioned evaluations of a subset of its programs as well as evidence from other non-Head Start early childhood programs to make the case that it is a good investment. As States and localities have bought the argument for the investment in early childhood programs, they have had the choice of adding funds to the federally funded Head Start program or developing and supporting alternatives. When considering this choice, State and local officials increasingly ask for evidence that their local Head Start programs are effective and more worth the extra cost their than less expensive local program alternatives. Evidence from national studies is not regarded as sufficient. Similar issues have been raised for other Federal programs, such as the Even Start family literacy program. This demand for local data is causing Federal agencies to rethink their evaluation strategies. The Early Head Start evaluation designed in the 1990s has both a locally designed component and a common cross-site national component. Both Federal and State government funders of social programs have begun to rethink their evaluation strategies to consider the information needs of a broader range of stakeholders. New Challenges in the Era of Accountability The biggest and perhaps most far-reaching challenges now facing research and evaluation leaders, funders, and customers are a result of the reinventing government efforts of the 1990s, whose calls for accountability and results have permeated every sector. The Federal Government Performance Results Act (GPRA) has its counterparts in State and some local governments, and the call for results-based accountability (RBA) has penetrated the nonprofit and for-profit service worlds. The advent of performance accountability is transforming the discussion about the purposes of evaluation and knowledge development in critical ways. Since the 1960s, debates about evaluation methods have been important, but now issues about who gets and uses information and data for what purposes--such as learning, accountability, rewards and sanctions, performance management, and continuous improvement--are at the center of debate and innovation. The emphasis on accountability is also opening up and repositioning the process of obtaining data from external evaluators and getting them to the provider organizations, funders, and perhaps the public. The push for accountability and performance management is a complex and testable reform strategy designed, in theory, to produce better individual and community outcomes from social and educational investments. Throughout the 1990s, researchers have studied State-level efforts to develop results-based accountability within early childhood and child and family services and are now examining the transition to performance management in nonprofits. After a decade's experience in Federal and State agencies, communities, and organizations, the implementation of results-based accountability and performance management is still in its infancy. It is unclear whether the transition to fuller implementation is possible or if it will pay off in better outcomes, more effective allocation and use of resources, and increased capacity for innovation. The early work has been plagued by: --Insufficient resources (time, money, and staff). --Primitive tools and measures. --Burnout or impasses early in the process of defining workable performance goals and measurable indicators. --Little or no organizational capacity to obtain and use data. --Competing and overlapping accountability systems and indicators in executive, legislative, and funding offices. --Complexities and risks of determining the locus and consequences of accountability, particularly when agencies or individuals have only partial control of necessary services and resources. --Resistance and the view that, like the U.S. Department of Defense's program planning and budgeting system (PPBS) of the 1960s and its successors, this idea too will pass.[5,6] Although there is fear of "gotcha" accountability and a cessation of funding for programs or agencies that do not produce results, little evidence of this exists to date, except perhaps with performance contracting arrangements. The need for program evaluations as a component of accountability and performance management strategies is also being recognized as the limits of performance indicators for understanding, explaining, and improving performance become more clear. Finally, government agencies and nonprofits are having difficulty locating the sustained internal and external technical assistance they need to support this work. At a time when there is growing demand for assistance from evaluators and others, those with the necessary skills are in short supply. The accountability and performance management process can be seen as a spiral unfolding over time. The first loop involves laying out the mission, goals, objectives, and performance indicators. The next loop tracks progress on the indicators, celebrates the improvements, and determines why some indicators decreased or did not improve. The final loop involves the dialogues, reflections, and analyses needed to decide on the next step toward improving performance. As Letts, Ryan, and Grossman recently argued,[7] the challenge for service providers is no longer to develop an effective program, prove it works, and "put it on autopilot." Rather, the challenge is to create the organizational capacity to continue to learn, innovate, improve services, and demonstrate impact. Very few organizations or public agencies have completed even the first loop in the spiral. As the above discussion suggests, there are numerous technical problems in implementing performance management. But perhaps more important are the cognitive, organizational, resource, and capacity barriers that must be overcome to allow the second and subsequent spirals of accountability and learning to occur, as evidenced by significant organizational or other changes designed to affect outcomes. One official from a State with one of the most advanced RBA approaches summarized the situation by saying they are at the third point on a 100-point scale. There is also a growing realization that although more traditional research and development activities- -such as experiments with new models or policies--are necessary and important, they do not address the need for new approaches to obtaining and using data for performance management and accountability. All parties in this system--policymakers and funders, service providers, and evaluators--are challenged to reinvent evaluation thinking and practice to create and test new approaches. It is increasingly clear that without a system for learning and continuous improvement, accountability is not likely to lead to improved policies, services, or outcomes. The push for accountability will go further and be more productive if it is set within the larger framework and support of a learning system, one with a key strategic role for experiments and other evaluations as well as supports and expectations for both accountability and continuous improvement. The Learning System Alternative The accountability movement is firmly rooted, but its members are unsure how to move it forward. What do the stakeholders need to implement and test whether performance management and accountability can lead to improving outcomes and solving social problems? The answer that is proposed and will be discussed--the creation of a learning system focused on using an array of data for continuous learning and improvement--is timely and viable for several reasons. First, the increasing pressure for, and experience with, performance management and accountability at all levels is creating a will to move from an immediate to a longer term approach to accountability that emphasizes improvement, learning, and innovation. Recent experience also indicates the scale and nature of the support necessary to do so. Second, while evaluation and other research-based information are rarely used to inform policy or practice, the demand for this type of information is likely to grow if the pressure for accountability continues and the performance indicator "needle" does not move up on key social problems. So the choice is not between investing in performance management or evaluation; rather, it is how to better align them to improve outcomes. Third, research suggests a growing willingness in both the public and nonprofit sectors to focus on outcomes and results if (and it is a large "if") expectations about performance are reasonable and the necessary resources and support are available. Finally, at the core of learning systems are learning organizations. Drawing on private-sector work, Garvin defines a learning organization as "an organization skilled at creating, acquiring, and transferring knowledge, and at modifying its behavior to reflect new knowledge and insights."[8] To be successful learners, people in organizations must be willing and able to use data on a regular basis, assessing implications, and making changes accordingly. The experiences of a growing number of local and State-level public and nonprofit learning organizations (and partial learning systems) offer some evidence that the proposed reframing is possible and can lead to more effective services and policies (see "Examples of Learning Organizations and Systems"). From a Federal Perspective To stimulate thinking about learning systems, William Morrill and I recently laid out the framework described below.[9] This particular version of a learning system, developed from a Federal perspective, specifies the types of investments in knowledge development, processes, and related supports necessary to move major Federal programs toward development of a learning system and continuous improvement. Although foundations, State and local nonprofits, and public funders do not make the same level of investment in knowledge development and evaluation, the logic of this learning system and most of its steps are applicable to programs they fund. Their capacity to move toward learning, continuous improvement, and accountability arguably would be enhanced if their efforts were even minimally aligned with similar Federal ones. For example, national-level investments in key areas (such as indicator development, continuing compilation and dissemination of research and evaluation results, and strategic commissioning of evaluations to fill critical knowledge gaps identified by policymakers and practitioners) would greatly support the local and State-level work with performance management and learning. Whatever the level of application, it is recommended that the three stakeholder groups--policymakers and funders, service providers, and evaluators--work together to build the learning system. Collective building and operation of a learning system by these groups is essential for long-term improvement in outcomes. Stages of a Continuous Learning System In a five-stage version of a continuous learning system (shown in exhibit 1), the basic learning loop involves laying out the vision, goals, and performance measures; tracking progress, determining the reasons indicators changed; and discussing what additional information could help determine changes to be made to improve performance. Over time, the organization works through the five stages repeatedly, continually examining and modifying its behavior, model, or policies in accordance with the learning process--and presumably its outcomes improve. The learning loop can also guide the work of larger units, such as large programs or fields of service or practice (see exhibit 1). Stage 1. During stage one, the stakeholders design the whole learning system. They set the learning agenda, performance goals, and indicators. They also obtain resources and a joint commitment to learning and identify the research and evaluation questions and gaps. The learning agenda is keyed to an organization, agency, or field's long-term strategic plan. It specifies how the stakeholders will come together to implement all the stages of the learning system. Another key issue for joint discussion is how to use evaluation resources to help understand both how and why indicators change and which services affect them. This ongoing discussion enables more strategic targeting of evaluation to help improve performance. Because information penetrates policy and practice incrementally and iteratively, information must be accessible and flow continuously in a learning system to promote course correctives and innovation. It is important that the stakeholders discuss the accountability consequences--the content and timing of their expectations for progress in program indicators, as well as for learning and change in the learning system itself. Expectations should be reasonable and the learning system should factor in the private-sector experience of learning from failure.[10] The consequences for accountability should be clear at the outset for all, and there should be opportunities and incentives for risk taking, innovation, and experimentation with new services and policies. This discussion must also include consideration of the point in a continuous learning system at which it is clear that a program or policy simply does not work well enough to warrant continued support. Stage 2. In stage two, the stakeholders learn from experience and relevant research and incorporate these lessons into program and policy design. Activities include assembling resources, specifying outcome and process measures and the data necessary to support them, networking to share successful innovations and common problems, and identifying technical assistance needs and providers. In this stage the stakeholders must create the means to disseminate and synthesize relevant research and evaluation information not only at this point but throughout the whole learning system process. Attention must be paid to developing new thinking about dissemination to maximize the learning from research and evaluation. Dissemination strategies should attempt to initiate active discussion and consideration of the implications for practice, organizational and policy change, as well as stakeholders' access to timely information about the latest research and evaluations. The Cochrane Group in Great Britain[11] has created a massive database of experimental and other studies on health and other intervention areas; similar work is being brought to the United States by the Campbell Group[12] These are examples of the kind of effort needed to make high-quality information more available for stage two--and the whole learning system. No learning system will succeed without strategic investments in amassing and ensuring the accessibility of the information necessary for continuous improvement, learning, and accountability. Stage 3. In stage three, stakeholders engage in innovation, monitoring, and evaluation. Activities include continuously testing new ideas and approaches, designing the evaluation strategies to assess them, and then monitoring and assessing progress with performance measures, evaluation data, and clinical data. Stage 4. In stage four, the stakeholders learn from evaluation and comparisons with others and make course corrections. Activities include using monitoring and evaluation information to identify the changes and course corrections needed, benchmarking to examine progress of the program or field of practice or policy, and assessing and applying knowledge from relevant basic and applied research. The recent work of the National Academy of Sciences (NAS) on reading and literacy is a good example of the kinds of work required for a field in stages four and five in a learning system. NAS gathered representatives from the three stakeholder groups in a participatory process to examine the evidence on a very controversial topic--approaches to reading instruction--and to make recommendations for practice, policy, and new research. The discussion, reflection, and consensus reached by the NAS panel helped resolve the disabling phonics-versus-whole-language debate and ultimately may help improve reading outcomes.[13] Stage 5. Finally, in stage five, the stakeholders examine the lessons for program respecification and identify knowledge gaps for further research and experimentation. Activities include identifying subjects for research and transferring knowledge for continuous improvement across the network and service or policy field. This is also a point at which to consider the types of large-scale and experimental evaluations that would address knowledge gaps for large program areas or fields. Our research on public-sector RBA and nonprofit performance management underscores that neither they nor any learning system can succeed without resources--time, money, and personnel. In his history of social program evaluation, Levitan notes that although PPBS launched the modern evaluation profession, PPBS itself was a failure because of the lack of adequate financial and organizational resources.[14] Considerable work needs to be done to determine the costs of a learning system and how to pay for it in new and reallocated resources. The hypothesis is that a learning system would eventually yield greater returns and do so more efficiently from knowledge development investments measured in terms of better and more use of information and gradually improved social problem-solving capacity. All parties involved in building such systems should discuss what critical mass of resources is necessary and specify a timeframe of a decade or more for assessing whether their hypothesis is correct. This model of a learning system is presented to stimulate discussion and critique. Its viability is being tested; it will continue to be modified through research in public and nonprofit accountability. Researchers are examining what is needed internally in an agency or organization, as well as the necessary external support, such as easy access to relevant research and evaluation data. Ongoing research suggests that the changes in leadership and organizational culture, funders' expectations and support, and resources to shift to learning and continuous improvement are enormous. Nonprofit organizations that do not make the shift to the "accountability game" are in jeopardy, but it is as yet unclear that those that make the shift and put forth the effort required for learning and continuous improvement will succeed. Arguably, their success depends, in part, on the willingness of the stakeholder groups to create a system such as the one outlined here. This requires that policymakers and funders invest in learning, not just strategic planning, indicator development, and accountability. The continuous learning system outlined here is a starting point for discussion of what it means to invest in learning. Examples of Learning Organizations and Systems Professional service, policy and management, and evaluation communities are working together in innovative ways to generate information for learning and continuous improvement. Each example presented below shows how key elements of the learning system link to inform policy and practice. Each example also illustrates a continuous learning loop or cycle with the core features of a learning organization: ongoing use of indicators to track and assess performance; ongoing dialogue and reflection on the implications; and resulting change in the organization, program, or policy. The continuous nature of the process as well as the use--not just collection--of data distinguishes this work. The examples range from single organizations (Project Match) to local (Consortium on Chicago School Research and the San Francisco Youth Development Outcomes Project) and State (Minnesota's Early Childhood Family Education Program and Vermont's Early Childhood Steering Committee) service systems. In each case, evaluators have played major roles in developing the learning approach and carrying out the work with the other stakeholders. These, along with similar partnerships among the three stakeholder communities, are leading the way by bringing new approaches to learning and accountability to the public and nonprofit sectors. Although the examples are just the beginnings of fully developed continuous learning systems, they and similar activities signal that more researchers and evaluators are developing the skills and the tools necessary to facilitate learning and that the service and policy communities are seeking and using evaluation and performance management data to improve practice and outcomes in an ongoing and systematic way. Project Match Project Match, a welfare-to-work program in Chicago, illustrates how a local program has used its commitment to evaluation and to the performance goal of sustainable employment to develop an ongoing learning system.[15] Project Match analyzed data from management information systems (MIS) and program participation to respecify its intervention models and transfer the lessons learned to inform national welfare policy. After tracking and critically examining the patterns of program participants' involvement in education, training, and employment, the program managers fundamentally altered their service model to encourage more clients to first seek employment rather than education and training. They found that those who opted immediately for school or training were more likely to drop out than those who worked and then went back for more education. This finding was later supported by the results of several larger welfare-to-work experiments. More and more community-based welfare-to-work and other programs are incorporating key aspects of learning models into their self-assessment and public accountability. They are, for example, using a sequence of setting client and organizational performance goals, developing MIS and client tracking systems, and then using these data for problem solving, model respecification, and accountability. A major challenge for States and the Federal Government is how to support these local activities within a broader learning system that shares knowledge and successful innovations from other sites and sources, such as experiments. Consortium on Chicago School Research The work of the Consortium on Chicago School Research led by Bryk and colleagues shows how universities, in consultation with a range of stakeholders, including administrators, union representatives, and parents, can conduct research to improve schools, assess the progress of public school reform, and report the results to the public on a regular basis.[16] The foundation-funded consortium uses multiple methods (surveys, case studies, administrative records, classroom observations, and longitudinal analyses of test scores) and measures to generate information and provide feedback to guide the reform process. Its work has helped the Chicago school system focus on ways to support better learning outcomes, carry out self-analysis, and build program development for schools on probation and remediation. The research model emphasizes mutual learning among researchers, policymakers, and practitioners. Bryk suggests that although stakeholder engagement requires much more time, it increases the likelihood that the research results will influence the change process to improve outcomes. The consortium example raises challenging issues about the resources necessary to create fully developed learning systems. As noted earlier, resources are required not only to collect information but also to engage the key stakeholders in using and learning from data as part of a continuous learning process. Substantial philanthropic resources are an essential component in the consortium example. Attaining a better sense of the resources required to develop key components as well as complete learning systems will be a critical next step in assessing their feasibility more widely. Minnesota's Early Childhood Family Education Program State-level partnerships between researchers and the service community, supported by public managers intent on continuous improvement (and only a small evaluation budget), can produce high-quality programs. Michael Q. Patton, a pioneer in utilization-focused evaluation, and his colleagues have worked with Minnesota's Early Childhood Family Education (ECFE) Program over the past 20 years to build a learning system to support high-quality statewide implementation.[17] Program managers and the service community were coached in a self-assessment research process designed to study the effects of the program on parents. The findings led to a series of course corrections to improve children's developmental outcomes. With strategic use of a small research budget (a one-time foundation grant of $150,000 and a $10,000 yearly evaluation allocation from the State), the State staff, in partnership with local providers, now have many basic components of the continuous learning system in operation at a minimal level. While this case illustrates the possibility of implementing learning system ideas within a single program, it is a substantially greater challenge to develop them within a whole service sector. Vermont's Early Childhood Steering Committee Vermont's Early Childhood Steering Committee is addressing this challenge in its statewide effort to build a learning system to improve outcomes for young children.[18] The committee is responsible for conceptualizing, implementing, and evaluating a comprehensive and well-integrated system of early childhood services. Beginning with a strategic planning process to articulate a vision for improving children's lives, the committee examined barriers to getting there and ways to overcome them. This process relied on lessons from past State experience and relevant experiences of other States and on information about the success or failure of past experiments. The committee regularly calls in experts and consultants from within and outside the State and encourages testing new ideas through experimentation and innovation. Processes to collect, report, and use performance indicators and other data have been established. To improve school readiness, a key performance goal, the committee is strengthening its child care system and measuring child care quality. Consisting of providers, parents, policymakers, and evaluators, the committee meets regularly and uses the data to assess progress, understand why changes have occurred, and alter strategies accordingly. The committee reports to a State Team on Children and Families, which oversees the integration of social and educational services in Vermont. San Francisco Youth Development Outcomes Project One of the most comprehensive efforts to build a learning and accountability system currently under way is the San Francisco Youth Development Outcomes Project (YDOP). It is a partnership among the Community Network for Youth Development (a major youth service intermediary organization), six foundation funders, seven diverse youth-serving organizations, and national (the Institute for Research and Reform in Education and the Community Youth Action Project) and local technical assistance providers and evaluators. Following a community-based youth development framework grounded in developmental research and developed by Connell and Gambone,[19] the partners work with the youth-serving organizations to develop indicators of youth development and detailed organizational action plans to improve youth outcomes. The framework lays out nine areas of organizational practice that research suggests promote favorable results. Through a self-assessment process, the agencies examine their organizational practices, set targets for change, and track their progress. They are building agency capacity for the ongoing self-assessment necessary to support organizational improvement. Working with funders to develop a flexible system for outcome reporting tied to learning and continuous improvement, they pilot and evaluate this process so it can be shared with others. The YDOP project illustrates how an intermediary organization can create a partnership among providers, funders, and researchers and evaluators to develop and test the five-stage learning system illustrated in exhibit 1. Early lessons from this project suggest the key roles of the intermediary overseeing the process; of committed funders and agency leadership; and of the flexible, ongoing technical assistance coaches skilled in organizational development. Together the partners are creating agency cultures open to reflection and change as well as greater understanding on the part of funders of what it takes to build agency capacity for high-performance management and better outcomes for young people. Barriers to Implementing Learning Systems These five cases illustrate the core characteristics of a learning system at the levels of the agency (Project Match and YDOP), the delivery system (Consortium on Chicago School Research), and the State (Minnesota's Early Childhood Family Education Program and Vermont's Early Childhood Steering Committee), and suggest the supports and processes needed to build them. By Garvin's[20] definition of a learning organization, these cases are creating and transferring knowledge and modifying their own services, organizations, and policies to improve their outcomes. These and similar examples are "existence proof" that such systems are feasible in an array of conditions and that innovative evaluators are helping to reinvent and expand evaluation ideas and practices to promote accountability and performance management. It will probably take another decade to determine whether this accountability and learning strategy can be implemented at a larger scale and, in turn, improve outcomes. In the meantime, the limited experience to date suggests several factors are hindering further implementation of systems of learning, accountability, and continuous improvement. First are major resource and training issues for all involved: evaluators, service delivery staff and managers, and funders. Public and private organizations struggle to find evaluators with the skills they need, managers struggle with developing the organizational culture and processes necessary for learning and change, and policymakers and funders often have unrealistic expectations about accountability. RBA and performance management are not quick fixes, nor is the proposed transition to learning and continuous improvement; it remains to be seen whether they will attract the support and resources to be fixes at all. Second, learning systems with the potential to improve outcomes require investments in basic research, experiments, dissemination of promising and best practices, and State and local evaluation capacity--the supporting information stakeholders need to interpret indicator data, design new programs and policies, and the like. It is unclear how to create and fund these essential supports for learning and continuous improvement. The multitiered evaluation strategy of the Bureau of Justice Assistance, U.S. Department of Justice, which builds State and local evaluation capacity, disseminates evaluation findings, and creates partnerships among researchers and practitioners, is one of a number of promising and strategic Federal efforts to support learning systems.[21] The U.S. Department of Education and the Charles Stewart Mott Foundation's public-private partnership for the 21st Century Community Learning Centers is another Federal program that emphasizes a learning approach with the related supports to improve out-of-school-time services. Finally, as Paul Light noted in a recent report on the embryonic status of performance management in nonprofits,[22] it is important to begin with modest expectations, build capacity, and slowly raise the bar as the necessary resources and experience accrue. The sports imagery of the high-jump bar conveys the need for initial, reasonable expectations, as well as of eventual progress and performance improvement. The argument here is that 10 or more years into the accountability experience, it will be time to step back and look at the whole game--poles, bar, field, and runners. What are all the future settings for the bar? What will it take to enable public and nonprofits to successfully jump over it at each setting? The argument and example of the continuous learning system presented here challenges us to address these questions, while always asking whether this approach is, in fact, helping to improve outcomes. Question-and-Answer Session Don Murray, National Association of Counties, Washington, D.C.: We have been watching the Smart Start Program in North Carolina very closely. It is tailored to exactly what you are talking about: grassroots evaluation. Governor Hunt has put $300 million into 100 counties. Does intergovernmental collaboration fit in your scheme? North Carolina has demonstrated how crucial that is. Although I do not know how you would measure this, how do you engage key stakeholders and track the level of enthusiasm, which is very high in North Carolina? H.W.: Two years ago I was the final site visitor of Smart Start when it won one of the Ford Foundation/Kennedy School of Government Innovations in American Government awards. A critical piece of the work in evaluating Smart Start is being done by a group of people at the University of North Carolina. Their approach is not unlike what I have described. Although they could not conduct an experiment because the legislature would not let them, they have a good longitudinal quasi-experimental design. In addition, as they obtain various information, they are plowing it back in to all of their Smart Start programs to support continuous improvement. The Smart Start application began with each of those places, laying out the status of kids and families as well as services for kids and families. They continue to refer back to that to measure what is different as a consequence of these Smart Start programs. Miron Straf, Committee on National Statistics, National Academy of Sciences, Washington, D.C.: I wanted to make three points stimulated by your great talk. First, there is a concern, certainly among many academics, about the shift from program evaluation to performance measurement and accountability. The goal of program evaluation is not really accountability … la GPRA (Government Performance Results Act of 1993), because the objectives of GPRA often reflect political and normative judgments. Also, many people feel that program evaluation is really a science and performance measurement is more a technology or an art. The crux is to relate what we can measure (often process variables) to the outcomes. (Research is a science and should be an important part of our goal.) I am pleased that you are going to these continuous learning systems. We can learn a lesson from the engineers who have to deal with many complex processes. They have a technique called evolutionary operation, which does not seek to maximize their process (there is no ideal or best program that they seek) but does try to make things better. Our evaluation of health care services would be better if it moved along those lines. Finally, there is a big cultural problem: Evaluation is not thought of in advance when Congress or others in the administration authorize programs. We will have to look at welfare reform and the recent children's health insurance program. Not a dime was allocated for evaluation, and there was no pressure for it, even within the administration, in the early days. Often, evaluation is at best a hurry-up afterthought, with an excruciatingly detailed evaluation design written into the legislation. Bill Christeson, Fight Crime: Invest in Kids, Washington, D.C.: We are almost 800 police chiefs, sheriffs, prosecutors, and crime victims who support getting what works to policymakers so they will make changes and support interventions to prevent crime. I went to a very interesting lecture last Friday about the Baldridge Awards and total quality management (TQM) in the schools. They are able to bring corporate America into the schools; they already have the data management capabilities and they do not have a shortage of people who can help them with developing data systems. The evaluation process seemed to have become totally democratized--the teachers and the kids themselves, not the administrators or the program managers, are expected to monitor, evaluate, and make changes. For example, they monitored the referrals to the principal for kids riding on the buses. They put the monitoring results on the wall and said, "This bus has x referrals and this bus has y referrals. Find out what the red bus is doing right and what the blue bus is doing wrong, and work out a solution." It was amazing; the system itself was driving solutions and they were finding many of them. They were homing in on the behavior of problem kids and working at different levels to find solutions for them. Before, they would have gotten lost in the shuffle. I was very impressed with it. But I am not sure how well it will replicate. What are your thoughts since it has been adopted widely in the corporate world? H.W.: The crux of what you said is very important and it speaks to larger questions. What are we getting information for? Whom are we getting it for? What do we expect that information to do for us? We need to invest in supporting the kind of thing you just described. We are trying to have ordinary people get information for ordinary services so they can improve those services. It turns a lot of our conventions about evaluation inside out. The incident you just described in this bus is a perfect example. People get information (why is it happening?) and solve the problem. This is a good use of information for problem solving and we need to expect that those kinds of things will take place as part of good performance management. What you described is critically important. We publish a newsletter called The Evaluation Exchange. If people are interested and not on the mailing list, let me know. We are always looking for examples of the sort of thing you just described. Allow me to mention some recent issues. We have been promoting the notion of youth participation in evaluation of youth programs. It is not to say we do not need good experiments on youth programs. But, in addition, we need evaluation that includes youths. The National Academy of Sciences group that is looking at community-based youth programs held a workshop in San Francisco in January 2000. The head of the California Wellness Foundation, Gary Yates, announced the foundation would no longer fund programs for youths that do not include youths--on the governance board, in the evaluation, in some thoroughgoing way. If programs it funds deal with positive youth development and support youth transition to adulthood, youths must be at the table. I do not know whether it still is, but the California Wellness Foundation has been in the top 25 funders of services and initiatives for children and youths. It is a very big player in funding innovative approaches to youths in California. We have been tracking a number of evaluators who have included youths in their evaluations. We have written about it in the past several issues of The Evaluation Exchange. In a recent issue we profiled a youth-led evaluation of reforms of San Francisco's juvenile justice system. Their story is similar to the one you described in the schools: Youths came up with little changes that prevented incarceration and other serious consequences. There is a place for this kind of thing. It is not all we need, but it is a piece of what we need if ordinary services are to improve and produce positive outcomes. Donna Bownes, Office for Victims of Crime, U.S. Department of Justice, Washington, D.C.: I was previously with the Office of Juvenile Justice and Delinquency Prevention (OJJDP), where we administered the Community Prevention Grants Program (Title V), which underwent a Government Accounting Office study in addition to a national evaluation. The program used government resources, both technical assistance and funding, to develop a community self-assessment workbook and to train program managers throughout the State so they could develop capacity. We also automated data collection, forms, and so forth to tailor the program to meet its own needs. We are working to develop capacity at both State and local levels. H.W.: Do you have examples from that program of places that are actually getting and using the data for program improvement? Donna Bownes: Yes, Dr. Heidi Shaw took over my responsibilities, and she is your contact at OJJDP. Robin V. Delany-Shabazz, Child Abuse and Neglect Program, Office of Juvenile Justice and Delinquency Prevention, U.S. Department of Justice, Washington, D.C.: I want to make two observations, and one is to underscore this thread of a conversation around client involvement, which is essential in a constructive continuous learning system. You say the partnership has to be between the policymakers and the program evaluators--but the families as well as the youths are essential partners right from the outset. Second, we have been working with a number of different programs whose goal is to construct a continuous learning system. We have found it helpful to create a partnership between the technical assistance providers and the evaluators. The evaluators then work as members of the team with the program folks and with the technical assistance providers, which both builds capacity and measures the results. We found this was essential to creating the continuous learning involvement and system. This is occurring in programs like "Safe Kids/Safe Streets," "Safe Start," and "SafeFutures," where we have begun to develop a real partnership among Federal agency staff, local program staff, evaluators, and technical assistance providers. Gloria Laycock, Policing and Crime Reduction Unit, Office of Justice Programs, National Institute of Justice, U.S. Department of Justice, Washington, D.C.: I have a 12-month fellowship to look at the relationship between research, policy, and practice, which is what your talk has been about. I agree with 99.9 percent of what you have said. What is really interesting, though, is how far off the criminal justice community is from where you are. We are still arguing about methodology, randomized control trials, and similar things and not getting to the point. You have not mentioned the mechanism or the processes by which things are supposed to work. Evaluators go in and evaluate things and, quite frankly, they do not first conduct an evaluability assessment to see if the proposed intervention could possibly produce the intended outcomes. D.A.R.E.[Registered Trademark] is a good example. If you reviewed the program initially, you were left thinking, "This cannot work." So, I would like to see more emphasis on how to carry out the learning system. I also agree that research and development need to be closer because good programs come out of research. You do not do research on them, you develop them from the research. And finally, on that point, NIJ is very badly placed to do that. I think there are several reasons. One is they do not have enough money. Although NIJ's budget is in the millions of dollars, it is mostly earmarked for selected purposes; its discretionary budget is relatively small, and most of that is spent on funding investigator-initiated projects, which it has a responsibility to do. NIJ has no capacity to do what you want. That is something for Congress to consider. H.W.: I believe we need to test the propositions I have laid out, but I would underscore that the work Jim Connell and Michelle Gambone are doing in San Francisco is based on Mr. Connell's own developmental research. He has been a pioneer in developing logic models or theories of change-type evaluations with Carol Weiss and others. There is a logic model for their work, but Mr. Connell is also asking, "Can you get organizations both to get and use data and to make the changes that developmental theory tells us should be made?" I wrote a report with a colleague a year or so ago about the way the 19 biggest foundations in this country that fund child- and family-related programs spend their money. Foundations are under the gun; their boards are saying, "Show that your funding strategies are adding value, producing results, and so forth." They are then pressuring their grantees, who are struggling with this as well. The Charles Stewart Mott Foundation has entered into a public-private partnership with the U.S. Department of Education on their 21st Century Community Learning Centers initiative. The Mott Foundation is building a learning system with the out-of-school-time community. Money is an issue and some of the money is earmarked for training and technical assistance about evaluation and continuous improvement. A critical mass of learning system programs funded through a public-private mechanism would give us a good test of whether some of these things are workable or not. I urge people to be creative when thinking about funding. I can tell you from the foundation side that a number of them might welcome an overture to work in a public-private partnership on youth development or another area of common interest in the child, youth, and family arena. Jim Breiling, Adult Psychopathology and Prevention Research Branch, Division of Mental Disorders, National Institute of Mental Health, Rockville, Maryland: We have certainly found it highly desirable in the research models that we have developed to get feedback from the youths. In the teaching-family-group home model, which is arguably the most research-based, humane, and effective group home model, the feedback from the youths on consumer ratings of fairness, decency, and concern on the part of the teaching parents is highly related to program effectiveness and is built into the ongoing accreditation. Every year, you do not get accreditation by taking course credits; you get it by demonstrating your performance in implementing the model and receiving high consumer ratings. I also think the improvement from this ongoing refinement is very helpful and desirable. But from my perspective, it is on the edge; in my areas of concern, the big problems are not knowing about and implementing major research findings and models that would make major differences. Risly and Hart have shown meaningful and enormous differences in vocabulary development and IQ that after age 3 are not remediable. Playing music for the fetus may be nice, but it is a secondary, fringe variable. The key thing is that the parent interaction and drawing out the child's vocabulary make the difference between the 75 IQ of the average welfare child and the 100 or 119 of the professional parents' child by age 3 (which is not remediable afterward). It is too big a gap. Take serious and violent offending delinquency. We have models based on science that work amazingly well--they are highly cost-effective. But tweaking pure cultural models that bring kids together and violate the pivotal finding that progression into serious and violent delinquency relates to having kids together will not do it, any more than a physician giving milk rather than antibiotics for an ulcer will. We have the data on programs for violence against women. The group battering treatments are totally ineffective. Tweaking them will not improve them; we have to implement the heavyweight research models that make large differences and make sure they are refined. How do you get the parents to develop the vocabulary? How do you keep delinquent kids from getting together and how do you get them to associate with nondelinquent kids? How do you deal with the psychopathology of the serious batterers? I think we have to get people to work on those heavyweight variables. H.W.: I think several aspects of what you said are very important. On the early development issues, Catherine Snow and colleagues analyzed Manpower Demonstration Research Corporation data with welfare moms and found exactly what you described. It is not just that a mom is reading to a kid; it is how the mom elaborates on the book, the conversation the child and the mom have about the book. How do we get that information into the hands of the people that do the family literacy work in this country so that they are not simply giving a kid a book; they are creating a relationship between a mom or another caregiver, a kid, and a book, which is very different from giving a kid a book. First, the issue of how to get what is known into practice is critical. Second, how to keep it effective is also critical. Nothing works forever. Therefore, there must be room for continuous invention and testing in the system. We must start thinking about a knowledge development and use strategy, and leave piecemeal efforts behind. That is my charge to you: How do you make this happen? I can give you examples in a community, and I can give you examples within a State agency. Is it ever possible to bump this up to a broader application? I think a broader application could potentially bring with it the resources necessary to get the kind of learning system I'm talking about into play. Notes 1. Bostrom, M., "Children as a Political Issue: A Review of Current Public Opinion," in Effective Language for Communicating Children's Issues, Washington, DC: Coalition for America's Children, May 1999. 2. Weiss, H., and E. Lopez, "New Strategies in Foundation Grantmaking for Children and Youth," Cambridge, MA: Harvard Family Research Project, 1998. 3. Weiss, H., and W. Morrill, "Useful Learning for Public Action," unpublished paper, Harvard Family Research Project, Cambridge, MA, 1998. 4. Weiss, H., and R. Halpern, "The Challenges of Evaluating State Family Support and Education Initiatives: An Evaluation Framework," paper prepared for the Public Policy and Family Support and Education Programs Colloquium, Annapolis, MD, April 1989. 5. Schilder, D., Aiming for Accountability: Lessons Learned from Eight States, Cambridge, MA: Harvard Family Research Project, 1998. 6. Bohan-Baker, M., D. Schilder, F. O'Reilly, J. Smith, and H. Weiss, "Learning from Starting Points," Cambridge, MA: Harvard Family Research Project, 1998. 7. Letts, C., W. Ryan, and A. Grossman, High Performance Nonprofit Organizations: Managing Upstream for Greater Impact, New York, NY: John Wiley & Sons, 1999. 8. Garvin, D.A., "Building a Learning Organization," Harvard Business Review (July/August 1993): 78-91. 9. Weiss and Morrill, "Useful Learning for Public Action." 10. Garvin, "Building a Learning Organization." 11. Chalmers, I., G. Macdonald, J. Grimshaw, M. Clarke, and A. Oxman, "How Might the Cochrane Collaboration Facilitate International Collaboration to Prepare and Maintain Systematic Reviews of the Effects of Social and Educational Interventions?" Available online at http://campbell.gse.upenn.edu/papers.html, 1999. 12. Boruch, R., A. Petrosino, and I. Chalmers, "The Campbell Collaboration: A Proposal for Systematic, Multi-national, and Continuous Reviews of Evidence." Available online at http://campbell.gse.upenn.edu/papers.html, 1999. 13. Snow, C., M.S. Burns, and P. Griffin (eds.), Preventing Reading Difficulties in Young Children, Committee on the Prevention of Reading Difficulties in Young Children, National Research Council, Washington, DC: National Academy Press, 1998. 14. Levitan, S.A., "Evaluation of Federal Social Programs: An Uncertain Impact," Occasional Paper, No. 1992-2, Washington, DC: George Washington University, Center for Social Policy Studies, June 1992. 15. Herr, T., and R. Halpern, Changing What Counts: Rethinking the Journey Out of Welfare, Evanston, IL: Northwestern University Center for Urban Affairs and Policy Research, 1991. 16. Bryk, A., presentation to the Evaluation Institute of the Annie E. Casey Foundation, Baltimore, MD, May 1997. 17. Cooke, Betty, Changing Times, Changing Families, St. Paul: Minnesota Department of Education, 1992; Mueller, Marcia, Immediate Outcomes of Lower-Income Participants in Minnesota's Universal Access Early Childhood Family Education Program, St. Paul: Minnesota Department of Education, 1996. 18. Bohan-Baker et al., "Learning from Starting Points." 19. Connell, J.P., and M.A. Gambone, "Youth Development in Community Settings: A Community Action Framework," unpublished paper, Gambone & Associates, Philadelphia, PA, 1999. (A copy of the paper may be requested from Michelle Gambone, Gambone & Associates, 429 Fulton Street, Philadelphia, PA 19147.) 20. Garvin, "Building a Learning Organization." 21. Kirchner, R., "Building State and Local Evaluation Capacity: The U.S. Department of Justice's Evaluation Strategy," in The Evaluation Exchange: Emerging Strategies in Evaluating Child and Family Services 4 (1998): 10. 22. Light, P., Making Nonprofits Work: A Report on the Tides of Nonprofit Management Reform, Washington, DC: Brookings Institution Press/The Aspen Institute, 2000. ------------------ Exhibit 1: Continuous Learning System: Five Stages ------------------ Other Titles in the Perspectives Lecture Series Perspectives on Crime and Justice: 1996-1997 Lecture Series, Volume I, November 1997, NCJ 166609 What, If Anything, Can the Federal Government Do About Crime? by James Q. Wilson Why Can't We Make Prohibition Work Better? Some Consequences of Ignoring the Unattractive, by Peter Reuter The Legitimation of Criminal Justice Policies and Practices, by Mark H. Moore Child Victims: In Search of Opportunities for Breaking the Cycle of Violence, by Cathy Spatz Widom Crime, the Media, and Our Public Discourse, by Norval Morris Perspectives on Crime and Justice: 1997-1998 Lecture Series, Volume II, November 1998, NCJ 172851 Crime Control, the Police, and Culture Wars: Broken Windows and Cultural Pluralism, by George L. Kelling Race, the Police, and "Reasonable Suspicion," by Randall Kennedy The American Experience With Stimulants and Opiates, by David F. Musto A Decade of Experimenting With Intermediate Sanctions: What Have We Learned? by Joan Petersilia The Epidemic of Youth Gun Violence, by Philip Cook Perspectives on Crime and Justice: 1998-1999 Lecture Series, Volume III, November 1999, NCJ 178244 Getting Deterrence Right: Applying Tipping Models and Behavioral Economics to the Problems of Crime Control, by Mark A.R. Kleiman Men and Fathers in the Community, by Felton Earls Violence, Free Speech, and the Media, by Sissela Bok Dilemmas of Corruption Control, by James B. Jacobs To obtain copies, contact the National Criminal Justice Reference Service (NCJRS): e-mail askncjrs@ncjrs.org; call 800-851-3420; or write to NCJRS, P.O. Box 6000, Rockville, MD 20849-6000. Include the NCJ number. Copies may be downloaded from the NIJ Web site: http://www.ojp.usdoj.gov/nij (in Publications and Products).