A social scientist reviewing the research literature on a topic often obsesses over potential biases in the individual studies being examined. However, this bias in primary studies may also exist at the level of a review; and a biased collection of studies will lead to a biased review. Systematic reviews strive to minimize bias in a review of conclusions drawn across studies. The purpose of this paper is to demonstrate the importance of engaging in a systematic and thorough search process by highlighting the following issues: 1) review evidence on the existence of publication issues bias; 2) show through an examination of the references in completed Campbell Collaboration reviews that the main bibliographic databases relevant to criminal justice are inadequate as a single source for identifying studies; and 3) address the myth that relying solely on peer-reviewed journal studies are included in a review. Recommendations to the field are offered for the development of a study registry or registries to facilitate the process of identifying an unbiased sample of studies. Tables and references