Work Group on American Indian Research and Program Evaluation Methodology Symposium on Research and Evaluation Methodology: Symposium on Research and Evaluation Methodology: Lifespan Issues Related to American Indians/Alaska Natives with Disabilities April 26-27, 2002 Washington, DC Editorial Committee Jamie D. Davis, Ph.D. Jill Shepard Erickson, M.S.W., A.C.S.W. Sharon R. Johnson, B.A., CRC Catherine A. Marshall, Ph.D., CRC Paulette Running Wolf, Ph.D. Rolando L. Santiago, Ph.D. Funded by National Institute on Disability and Rehabilitation Research Office of Special Education and Rehabilitative Services U.S. Department of Education Grant No. H133B980049 and Indian Health Service Office of Clinical and Preventive Services Division of Behavioral Health With support from Child, Adolescent and Family Branch Center for Mental Health Services Substance Abuse and Mental Health Services Administration The contents of this monograph are the responsibility of the American Indian Rehabilitation Research and Training Center and no official endorsement by the U.S. Department of Education should be inferred. Northern Arizona University is an Equal Opportunity/Affirmative Action Institution. American Indian Rehabilitation Research and Training Center Institute for Human Development Arizona University Center on Disabilities Northern Arizona University PO Box 5630 Flagstaff, AZ 86011 (928) 523-4791 Voice • (928) 523-9127 Fax • (928) 523-1695 TDD http://www4.nau.edu/ihd/airrtc Suggested reference: Davis, J.D., Erickson, J.S., Johnson, S.R., Marshall, C.A., Running Wolf, P., & Santiago, R.L. (Eds.). (2002). Work Group on American Indian Research and Program Evaluation Methodology (AIRPEM), Symposium on Research and Evaluation Methodology: Lifespan Issues Related to American Indians/Alaska Natives with Disabilities. Flagstaff: Northern Arizona University, Institute for Human Development, Arizona University Center on Disabilities, American Indian Rehabilitation Research and Training Center. This report is available in alternate formats by contacting the Institute for Human Development at (928) 523-4791. ISBN #1-930563-132.iii Table of Contents Preface.......v Acknowledgements vii PART I INTRODUCTIONS 1. Introduction to the Monograph 1 Jamie D. Davis, Ph.D., Jill Shepard Erickson, M.S.W., A.C.S.W., Sharon R. Johnson, B.A., CRC, Catherine A. Marshall, Ph.D., Paulette Running Wolf, Ph.D., Rolando L. Santiago, Ph.D. 2. The Symposium: Introductory Remarks 5 Holly Echo-Hawk, M.S., Facilitator PART II PAPERS 1. A Brief History of and Future Considerations for Research in American Indian and Alaska Native Communities 9 Jamie D. Davis, Ph.D., and Kelly Keemer 2. Research in Indian Country: Challenges and Changes 19 Walter Hillabrant, Ph.D. 3. Cultural Competence Approaches to Evaluation in Tribal Communities 32 Paulette Running Wolf, Ph.D., Robin Soler, Ph.D., Brigitte Manteuffel, Ph.D., Diane Sondheimer, Ph.D., Rolando L. Santiago, Ph.D., Jill Shepard Erickson, M.S.W., A.C.S.W. 4. Community-Based Research and American Indians with Disabilities: Learning Together Methods that Work . 50 Catherine A. Marshall, Ph.D., Sharon R. Johnson, B.A., CRC, Elizabeth Kendall, Ph.D., Howard Busby, Ph.D., Robert Schacht, Ph.D., Calvin Hill, B.S. 5. Learning from and Working with Yup'ik Professionals 67 Jennifer Olson, Ph.D., Philip Olson, Ph.D., Teresa Pingayak, Katherine W. Sterling, M.A., Lenea Pierzchanowski, M.H.R. PART III CROSS-CUTTING CRITIQUES AND DISCUSSIONS 1. Focus on Research Methods 81 Spero M. Manson, Ph.D. 2. Discussion Guided by Dr. Manson's Comments .91 3. Summary and Overview for Day One .97 Holly Echo-Hawk, M.S., Facilitator 4. Focus on Cultural Issues in Research: Developing and Implementing Native American Postcolonial Participatory Action Research 98 Joseph B. Stone, Ph.D. 5. Discussion Guided by Dr. Stone's Comments 122 6. Focus on Implications for Policy and Practice 133 Velma Garcia Mason, Ph.D. 7. Discussion Guided by Dr. Mason's Comments 140 8. Final Discussion and Summary Comments 142 PART IV ABOUT THE CONTRIBUTORS TO THE MONOGRAPH About the Authors 151 About the Discussants 158 About the Facilitator 161 APPENDICES Appendix A: List of Symposium Participants 165 Appendix B: Participants in the AIRPEM Network 169 Appendix C: Dr. Stone's Chapter Review Coding Sheet 172 Appendix D: A Model for Conducting Research with American Indian Participants 176 Justin D. McDonald, Ph.D. Preface Research in Indian Country can be, and often is, challenging. Tribes are often sovereign political entities with specific powers of self-governance. This raises issues of protocol and the tribal leaders' acceptance of research agendas. Other challenges faced by researchers are rooted in cultural and language barriers. The American Indian population is highly diverse, speaking about 200 languages and dialects. While there are some beliefs and traditions that are common to many Indian nations, American Indians cannot be perceived as being homogenous. This monograph has been developed to address alternative and appropriate methods of conducting research with this population. Research with American Indians needs to be academically acceptable in the general research community as well as appropriate culturally and linguistically. In the quest to improve and enhance quality of life for First Nations people, there are environmental, societal, economic, health, and disability issues that need to be addressed, including a need for capacity building throughout Indian Country. Research and program evaluation experiences shared in this monograph are offered as viable methods of impacting these multifaceted issues and problems. Joyce Y. Caldwell Rehabilitation Program Specialist National Institute on Disability and Rehabilitation Research Acknowledgements The editorial committee for this monograph acknowledges with appreciation the significant contributions of time by participants who formed the Work Group on American Indian Research and Program Evaluation Methodology (AIRPEM)-colleagues who trusted our process even as we were developing it, beginning with our first meeting in March 2000. From that first meeting, participants in the AIRPEM network assisted in various ways and at various times in planning the Symposium on Research and Evaluation Methodology: Lifespan Issues Related to American Indians/Alaska Natives with Disabilities, held April 26-27, 2002, in Washington, DC. We also warmly acknowledge the contributions of our symposium discussants, Dr. Velma Mason, Dr. Spero Manson, and Dr. Joseph Stone, whose thoughtful and thought-provoking critiques of the papers included in this monograph directed our discussions at the symposium and will allow us to develop more fully our thinking and our practice. We are very appreciative of the contribution of the symposium facilitator, Holly Echo-Hawk; her skill in facilitating our work at the symposium was extraordinary, as was her demonstrated commitment to our efforts to better serve American Indians and Alaska Natives with disabilities through research and program evaluation. On behalf of AIRPEM, we thank the Indian Health Service and the National Institute on Disability and Rehabilitation Research for their financial contributions to the symposium. We also thank the Center for International Rehabilitation Research Information and Exchange (CIRRIE) for making possible the attendance of Dr. Elizabeth Kendall at the symposium. We very much appreciate the staff contributions of the Substance Abuse and Mental Health Services Administration. We acknowledge the contribution of Hιlθne Flamand, Graduate Assistant, American Indian Rehabilitation Research and Training Center, in researching references. Finally, the editorial committee acknowledges the valued assistance provided to us by editorial consultant, Dr. Barbara Du Bois. The editorial committee itself, however, takes responsibility for any errors or omissions in the monograph. Part I Introductions Introduction to the Monograph Editorial Committee Jamie D. Davis, Ph.D. Jill Shepard Erickson, M.S.W., A.C.S.W. Sharon R. Johnson, B.A., CRC Catherine A. Marshall, Ph.D., CRC Paulette Running Wolf, Ph.D. Rolando L. Santiago, Ph.D.. The purpose of this monograph is to report on activities of the Work Group on American Indian Research and Program Evaluation Methodology (AIRPEM), including its first symposium, held on terms of collegiality, to address our concerns and share our experiences through critique and discussion of papers prepared for this meeting. Three distinguished American Indian social scientists and social policy experts, Dr. Velma Mason, Dr. Spero Manson, and Dr. Joseph Stone, agreed to critique the papers and join us for our discussions, facilitated by Ms. Holly Echo-Hawk, an organizational behavior and management consultant. The Symposium on Research and Evaluation Methodology: Lifespan Issues Related to American Indians/Alaska Natives with Disabilities was held April 26-27, 2002, in Washington, DC; Appendix A contains a list of symposium participants. The informal collegial network now known as AIRPEM evolved in response to the desire to share information and resources among researchers, program evaluators, health and human service professionals, policy makers, and private practitioners who work with American Indians who have disabilities (see Appendix B). Simply stated, we needed each other, we found each other, and we look forward to continuing to learn from each other out of shared interests and concerns. Specifically, the purpose of AIRPEM is to explore, share, and document American Indian cultural considerations in relation to "best practices" in research and program evaluation. A Brief Summary of Cultural Considerations AIRPEM participants understand the importance of tribal consultation when proposing research in Indian Country and the importance of conducting research that benefits American Indian communities. We do not support "color blind" research-in other words, we believe that research focusing on the needs of American Indians with disabilities of all age groups is so important that their needs must be specifically addressed in the delivery of health and human services. A history of public data-gathering that is "color blind" has resulted in data through which American Indians can not be identified, nor their needs isolated from those of the majority. AIRPEM participants understand that not only do cultural factors regarding American Indians (considered as one ethnic minority population) affect the research process, but that the diversity among American Indian nations is also to be considered. Symposium Administration and Support While persons in the AIRPEM network represent a variety of governmental agencies and private organizations (see Appendices A & B), the American Indian Rehabilitation Research and Training Center (AIRRTC) offered to provide administrative support in organizing the symposium. Administratively based at the Institute for Human Development, Northern Arizona University, in Flagstaff, the AIRRTC has been funded since 1983 by the U.S. Department of Education, Office of Special and Rehabilitative Services, National Institute on Disability and Rehabilitation Research (NIDRR). During its almost 20-year history of conducting research nationwide to benefit American Indians and Alaska Natives with disabilities, the AIRRTC has developed considerable knowledge regarding best practices for conducting research that is both sound in design and sensitive to the cultural needs of the individuals, families, and communities with whom and in which we work. The AIRRTC program of research 1) focuses exclusively on an underrepresented and underserved population; 2) focuses particularly on significant social problems associated with American Indians with disabilities, such as unemployment, substance abuse, and being "invisible" in large data sets as well as in social service and health care systems; and 3) focuses not only on improving employment opportunities for American Indians with disabilities but also on alleviating problems of diagnosis of disabling conditions such as FAS/FAE (Fetal Alcohol Syndrome/Fetal Alcohol Effects). Financial support from the Indian Health Service and from NIDRR ensured that we could hold the symposium and, importantly, that we could produce a proposed monograph based on papers that had been prepared for critique and discussion at the symposium. Significant staff support from the Substance Abuse and Mental Health Services Administration (SAMHSA) ensured that the symposium would offer a successful interchange of experiences. Jill Shepard Erickson, M.S.W., Project Officer at SAMSHA, and Catherine Marshall, Ph.D., Director of Research at the AIRRTC, served as co-coordinators of the AIRPEM symposium. Symposium Outcomes From our first planning meeting in March 2000, AIRPEM participants hoped that the symposium would allow participants to solidify the purpose of AIRPEM, that is, the utilization of our collective experiences and knowledge to better serve American Indians with disabilities. We intended to make available a monograph documenting our experience and allowing us a base from which to explore and define future directions in research and program evaluation. On the basis of the prepared papers and invited critiques, our concluding discussion at the symposium led to standards for the monograph which are, we hope, reflected in this document. Ultimately, we hope that through appropriate and accurate needs assessments and program evaluations, communities in Indian Country can see an increase in dollars for community development and problem-solving service delivery. To that end, we hope that this monograph provides examples of research and program evaluation efforts that have appropriately involved American Indians with disabilities. Through this monograph, we share also the critiques of our efforts and some of the discussions that occurred over the day and a half in which we communicated together about our aspirations, positive experiences, struggles, and intentions for future work. For example, Dr. Spero Manson's comments led us to consider issues relating to the role of the scientist as an advocate in an era when indigenous communities are increasingly taking ownership of the research process, and some of the tensions involved in conducting research in Indian Country. Dr. Joseph Stone asked us to consider the impact of postcolonial stress on the persons whom we are asking to serve as research participants and to reflect deeply upon the implications of colonial and postcolonial trauma for understanding the social, political, spiritual, and health-related experience of contemporary American Indians and Alaska Natives (see Appendix C). Dr. Velma Mason's comments urged us to consider the needs of tribes and reminded us of an obligation to assist with building capacity for tribes to conduct their own research and program evaluation. Discussants also reminded us to remember our history, in particular the Native researchers who first drew attention to cultural issues in research (see, for example, Appendix D). We also need to acknowledge the history of the disability movement and the attempts of NIDRR-funded researchers such as H. Rutherford Turnbull and Ann Turnbull (1989), who sought to identify "principles of research on people with disabilities and their families" (p. 1) and whose consensus conference noted that "research should be sensitive to cultural, socio-economic, ethnic, life style, and life span pluralisms" (p. 2). Similarly, we need to acknowledge the work of rehabilitation re-searchers such as Pollard (1992) who, in summarizing ethical principles and practices in conducting research with diverse populations, called to our attention that such considerations apply when conducting research with "members of the sociocultural Deaf community" (p. 88). Each of the papers prepared for the symposium reflects the contribution of individual or team authors who hoped to learn by sharing and who hoped to participate in improving our science by offering their best efforts for critique. A Brief History of and Future Considerations for Research in American Indian and Alaska Native Communities provides a historical and contextual overview of research in Indian Country, tracing the evolution, impact, and implications of attitudes and practices in the social sciences in relation to American Indians and Alaska Natives, bringing us to the opportunities of the present moment. Research in Indian Country: Challenges and Changes continues the contextual theme and provides specific examples of factors which must be faced in program evaluation, highlighting particularly questions of ethics in social research in Indian Country. Cultural Competence Approaches to Evaluation in Tribal Communities shares the responses of communities to cross-site evaluation efforts in a national evaluation program-responses that highlight not only the diversity in Indian Country but also the creativity with which Native communities are adapting contemporary research and evaluation principles and methods to meet their unique cultural needs, resources, and strengths. Community-Based Research and American Indians with Disabilities: Learning Together Methods that Work reflects the almost 20-year history of the AIRRTC's research experience and emphasizes how we can learn not only from the U.S. indigenous cultural experience but also from the experience of indigenous cultures in other countries, underlining the critical significance of culture itself as a primary factor in conceptualizing and carrying out research. Learning from and Working with Yup'ik Professionals describes an intervention and program evaluation involving a specific group of Native people, highlighting these Alaska Natives' desire and ability to make use of contemporary program technology and procedures to fulfill their own learning goals. Some of our symposium discussions had no conclusion; these may be the most important discussions, after all, because they raise questions that others can learn from and creatively address. We hope students will continue the discussions in their classes, that researchers and program evaluators will continue the discussions in their work with American Indian, Alaska Native, and other indigenous communities, and that policy makers will enter the discussions. To that end, we plan to continue communicating and meeting, to hold a second AIRPEM symposium, and to grow the network of AIRPEM participants. As Pollard (1992) concluded, "It is by remaining abreast of ethical and cultural opinions and developments, demonstrating real concern for the issues raised, and employing research practices that are in accordance with one's best informed and reasoned judgments that ethical responsibility is fulfilled" (p. 98). Thoughts on Future Directions We intend to stay focused on the intersection of disability and the American Indian and Alaska Native experience. It is important to clarify that we are not looking for one model of research. We respect the needs of individual programs to conduct research and evaluation in accordance with their program objectives. Based on our final discussion at the symposium, specific questions we hope to address and tasks for AIRPEM participants include: • How does what we have discussed and learned apply to evidence-based evaluation? How can we identify and contribute to methods that demonstrate successful program outcomes? • What do we want policy makers to do with this monograph? • Develop and agree upon standards of research conducted with American Indians who have disabilities-standards that funding agencies would honor when awarding grants. • Develop a bibliography of publications which have addressed or demonstrated appropriate research and program evaluation methods in Indian Country. • Identify consumer, professional, and advocacy groups that might be interested in joining our work: for example, National Council on Disability, Inter-tribal Deaf Council, Consortia of Administrators for Native American Rehabilitation (CANAR), National Council on Independent Living (NCIL), Rehabilitation Engineering and Assistive Technology Society of North America (RESNA), The Association of Persons with Severe Handicaps (TASH), National Congress of American Indians (NCAI), and the National Indian Health Board (NIHB). References Pollard, R.Q. (1992). Cross-cultural ethics in the conduct of deafness research. Rehabilitation Psychology, 37(2), 87-101. Turnbull, H.R., & Turnbull, A. P. (1989). Report of consensus conference on principles of family research. (Available from the University of Kansas Bureau of Child Research; Beach Center on Families and Disability; 3111 Haworth Hall; Lawrence, Kansas 66045). The Symposium: Introductory Remarks Holly Echo-Hawk, M.S., Facilitator. We all have a lot in common. We all care very deeply about the communities that we come from and about American Indian and Alaska Native communities across the country. We all share deep concern about the well-being and future of Indian children and families. We are all very self-sacrificing. We are in this business because of our personal beliefs and our desire to make a difference. Our personal backgrounds, our training, our education, and our professional experiences all contribute to our desire to make a difference. There's another thing that we share in common, and that is that we are privileged in a very special way. We are privileged to be of American Indian or Alaska Native ancestry, or we are privileged to be invited into those communities and into those families. Let me share a story with you. When I was 10 years old my family had a Native American Church prayer meeting for me, and I remember that very profoundly even though I was a little girl. Decades later, when my son was about the same age, I had a Native American Church meeting for him at the same place. When we held the prayer meeting for my son, people came from all over-from Navajo, from different tribes in Oklahoma, and of course my family was there as well. The church members pray for a long time before they arrive, and they prayed not just for my son but for all the generations that come forward from him. When you're sitting there and you hear a roar of prayers in all different languages-Navajo, Pawnee, Sioux, and English-it's a very powerful and privileged experience. I wanted to share this with you because I think it's important to remember what we are here for-that is, for the future generations. I am privileged to be here with you all, because the work you do, the words you write, the research you conduct, the information you seek to gather from Indian people, is very special and should be very protected. You hold a tremendous amount of power, and your valuable tools of written information can either be very, very useful to tribal communities-or very harmful. I am fortunate to be here because this is a gathering of people who want to critically examine their work so that it is of the most use to tribal communities and to the generations that follow. The last thing I wanted to mention is one of the creeds that I live by, which is "First do no harm." So, with that introduction of myself, I thank you all for allowing me to facilitate your conversation for the next two days. Part II Papers A Brief History of and Future Considerations for Research in American Indian and Alaska Native Communities Jamie D. Davis, Ph.D. Kelly Keemer Abstract The authors explore the relationship between the historical treatment of American Indian and Alaska Native people and research practices in these communities. Forced acculturation, exploitation, and other injustices contributed to feelings of inferiority and mistrust of investigators as well as of the research process. Participatory research methods and tribal research codes of ethics have helped transform the research process for American Indian and Alaska Native communities to one that is beneficial for Native people as well as for researchers. Future research needs in American Indian and Alaska Native communities are discussed Historical views about individuals and communities shape the ways that researchers interact with people and their communities. The European settlers thought of Native Americans as savages in need of socialization. Accordingly, as scientists began investigating American Indians, there was little concern for the needs of these people and their communities. "Civilization" brought new diseases and social problems to these communities and, with them, an opportunity for researchers to impose Western scientific values to solve these problems for Native communities. More recently, significant strides have been made in involving American Indians and Alaska Natives as partners in the study of physical, emotional, social, and environmental problems. Such partnerships are beneficial for researchers as well as for American Indian and Alaska Native people and their communities. Although the focus of this paper is the indigenous peoples of what is now the United States and Canada, the need to develop true collaborations with indigenous people has no boundaries. It is in this spirit that we write this paper-with the hope that it will, in some small way, contribute positively to the health status of indigenous peoples and close the gap of health care disparities in these communities. Historical Perspectives on Research in American Indian and Alaska Native Communities-Lessons Learned Paternalism-Conducting Research on Native Americans The mistreatment experienced by American Indians for centuries has resulted in little tolerance for non-Natives who wish to conduct research in Native communities. American Indians and Alaska Natives have come to believe that researchers often do not recognize the rich diversity of each Nation and, instead, may catalog the more than 500 federally recognized tribes as equivocally Indian. Acknowledging the diversity of Indian peoples and thus the uniqueness of each tribe is fundamental to gaining the trust and respect of the community. Few researchers understand the influence traditional values and beliefs have on the lifestyles of American Indian and Alaska Natives; for the scientist, such values rarely have significance for research (Davis & Reid, 1999). For American Indian and Alaska Native people, cultural values represent a way of life. A relationship exists between the historical treatment of American Indian and Alaska Native people and the research methods used to study these individuals. A search of the Native Health History Database (http:// hscapp.unm.edu/nhhd) by the authors of this paper revealed that published research has been conducted on American Indians since the early 19th century. Written accounts of American Indians certainly existed well before this time but were primarily observations made by missionaries and military personnel (e.g., see Oliver, 1764). At best forced acculturation and at worst annihilation were imposed on the indigenous peoples of North America. The first reservation was established in Connecticut in 1638, officially signaling the beginning of relocation efforts-efforts that would later tear at the very fabric of the collective nature of all American Indian communities (Equity Center Infoline, 1999). Native communities, entire tribes in many instances, were destroyed. Following this period of destruction and initial relocation came the second phase of extensive relocation and forced removal of the early- to mid-1800s. Research written during this historical era reflects the Eurocentric beliefs about American Indians. One such example is the 1857 article authored by Dowler titled Researches into the Sanitary Condition and Vital Statistics of Barbarians. American Indians were considered savages incapable of caring for themselves, much less of engaging in scholarly activities such as conducting research. Although the many efforts of the U.S. Government (e.g., forced removal and distribution of disease-infected blankets) were unsuccessful at destroying the Native Americans, new strategies were employed. In 1879, the Carlisle Indian School was established in Pennsylvania. The motto of Carlisle was "Kill the Indian, save the Man" (Styron, 1997). This new method, assimilation, although different from forced removal and genocide, was no less destructive to the individuals and communities exposed to boarding school policy. An 1872 statement by George Grant, a Presbyterian minister who traveled across North America documenting his observations, was prophetic: As the Indian has no chance of existence except by conforming to civilized ways, the sooner that the Government or the Christian people awake to the necessity of establishing schools among every tribe the better. Little can be done with the old, and it may be two, three or more generations before the old habits of a people are changed; but, by always taking hold of the young, the work can be done (Styron, 1997). Research of this era supported this notion that western education was the only hope to reform the amoral practices of Native Americans. Elliott authored one such article, "Wild Babies," in 1878. Federal legislation mandated compulsory schooling for Natives. By 1887 there were 167 schools both on and off the reservations with an enrollment of nearly 15,000. In 1902, of the nearly 200 Indian schools, 25 were federally sponsored off-reservation. The number of boarding schools continued to increase throughout the early 1900s; such schools were located in over 15 states and territories (The Brown Quarterly, 2001; Equity Center Infoline, 1999). Although sympathizers with the "Indian condition" were increasing in numbers, research practices were paternalistic. Even though many American Indians experienced considerable injustices at the hands of educators and missionaries, the education these individuals received undoubtedly contributed to the eventual evolution of research practices in American Indian and Alaska Native communities. Interestingly, it was not until 1924 that all American Indians were granted U.S. citizenship (American Indian Lawyer Training Program, Inc., 1988). The paternalistic approach to interactions with and research on Native peoples continued into the 20th century. Research conducted in the 1920s through the 1940s included Mental Disease Situations in Certain Cultures-A New Field for Research (Cooper, 1934), Medical Observations Made on Zuni Indians (Flemming, 1923), and Studies in the Dietaries of Contemporary Primitive Peoples (Klatsky, 1948). During the 1950s the U.S. Government again decided it knew what was best for American Indian people. The years between 1953 and 1962 are known as the "termination experiment" (American Indian Lawyer Training Program, Inc., 1988, p. 13). These are the years of the Termination Acts. The federal government believed that over 100 American Indian tribes should be able to assimilate into the larger society without the benefits and services afforded through trust responsibilities (American Indian Lawyer Training Program, Inc., 1988). During this period it was not uncommon for researchers to collect data without the full knowledge and consent of participants and without respect for local culture and tradition (Hodge, Weinmann, & Roubideaux, 2000; Macaulay, 1994). At one time (and even in contemporary times), it was not uncommon for helicopter research (Hodge et al., 2000) to take place in Native communities. The term helicopter research describes the way in which investigators fly in to collect the data and fly out, having little or no interaction with the community. These researchers leave without obtaining community endorsement or considering the needs of the community, to publish findings that tell only of the health and health care disparities and problems found in the American Indian and Alaska Native communities studied. Often community members are unaware of the purpose of the study; benefits to the community are unknown. Such research practices have led to feelings of exploitation and inferiority when researchers try to gain entry into a Native community. This can result in an unsuccessful research project (Davis & Reid, 1999). Many tribal communities have become protective and, at times, prohibit researchers from their communities. Joining-Conducting Research with Native Americans Although research practices continued to neglect the needs of American Indian and Alaska Native communities, the evolution of anthropology and social science research, coupled with the changing political climate in the U.S., led to some changes in the way research was conducted in these communities. The Nixon administration passed the Indian Self-Determination and Education Assistance Act (Pub. L. 93-638) in 1975, supporting the autonomy of American Indian and Alaska Native communities (Kunitz, 1996). Although first developed in the early 1960s, the Indian Health Service (IHS) policies, procedures, and responsibilities for research activities were significantly updated in 1977, 1982, and 1987. These new policy documents set forth the guidelines and oversight for research activities, including the Protection of Human Subjects, the IHS Institutional Review Board (IRB), Area Research Committees, and the Director of Research (Indian Health Manual, 2001). In addition, a latent benefit of assimilation and forced education was emerging; American Indians and Alaska Natives were obtaining advanced degrees and beginning to influence the research process. Carolyn Attneave, a Delaware and Cherokee psychologist, is one such example. Her work on the importance of culture for families and communities helped change the way researchers thought about and conducted research (see, e.g., Attneave, 1982). During the 1970s, researchers began working with Native communities-that is, they began to involve Native communities in the research process, but still without true collaboration. Consideration for American Indian and Alaska Native community needs is evident in the research of this era. Several examples of research during this period are representative of this paradigm shift. Patrick and Tyroler's (1972) research, reported in Papago Indian Modernization: A Community Scale for Health Research, focused on community needs; Taylor (1975) consulted with tribal members in developing his Cherokee Tribal Health Evaluation Report, and Brelsford (1977) authored Ggwangkumtenek Sungcarluta, consulted with Alaska Natives, and incorporated Alaska Native language into his paper. Consultation is not collaboration. Foulks (1989) candidly discussed the mistakes made and the resulting misalliances in a study of alcohol use among the Inupiat. Although the investigators established a Steering Committee that consisted of local Inupiat leaders and a Technical Advisory Group (mostly made up of non-Native professionals), the investigators were unable to reach consensus about the content of the report in conjunction with the Steering Committee and the Technical Advisory Group. When a press statement was released about the results of the study the headlines read, "Sudden Wealth Sparks Epidemic of Alcoholism: What We Have Here is a Society of Alcoholics" (Foulks, 1989, p. 13). The community was outraged. A non-Native faculty member of the Inupiat University of the Arctic summed up this outrage: [T]he North Slope Borough Assembly has been the victim of a sophisticated hoax aimed at destroying the credibility and integrity of the Inupiat people. The research on alcohol abuse and the news coverage was the most demeaning and reprehensible sham. Instead of using Winchester and Remington rifles to destroy a people and a culture, as with the Indians in the 1880s, they bent words, numbers, and statistics to accomplish what was in effect a social and cultural genocide. These con artists hiding behind the guise of professionalism and religiosity, and acting as consultants to the North Slope Borough have dealt a devastating blow to the Inupiat people and their cultural heritage (Foulks, 1989, pp. 14-15). Such violations of trust by researchers in American Indian and Alaska Native communities negatively affected the willingness of these communities to participate in the research process (Norton & Manson, 1996). Even the most recent research sometimes overlooks the nuances important for conducting research in American Indian and Alaska Native communities. In an important and generally well developed study of recruiting American Indians and Alaska Natives into clinical trials (Hodge et al., 2000), the term subjects is still used to describe research participants. Such language minimizes the value of the individuals who partner with researchers. Collaborating-Conducting Research in a Participatory Fashion Participatory Research Research has quantified and described some of the realities of American Indian and Alaska Native people but has contributed little to improve this reality (Dickson & Green, 2001). Despite the problems of the past, American Indians and Alaska Natives realize the need for research and health promotion projects in their communities. Participatory research (PR) represents an active step in improving research so that the research benefits the communities studied. Simply stated, PR empowers people to become responsible for addressing the issues that affect their lives. Participatory research begins with the idea that people can and must benefit from the research conducted in their communities. It is important that both scientists and community members share equally in the research planning, implementation, evaluation, and results dissemination phases as well as in any resulting benefits (Davis & Reid, 1999). The principles of PR clearly define the roles of the partners who are collaborating for the benefit of the community. The ultimate goal of participatory research is to empower communities to assume ownership of the research process and to utilize the results to improve their quality of life (Macaulay et al., 1998). Only research that primarily considers the interest(s) of the community, not just the interest(s) of the larger society, can bring significant benefits to the community served. Researchers must approach the community in a respectful manner. A community meeting that explains, in layman's terms, the goals of the project to all interested community members will allow for questions to be answered and misunderstandings to be addressed. "In recent years, many tribes have formed their own Institutional Review Boards (IRB) to obtain some degree of control in the research process and to halt the perceived misuse and misinterpretation of data" (Hodge et al., 2000, p. 43). Tribes evaluate research projects to determine whether hypotheses are of interest or results will provide benefits to the tribe; researchers must gain the approval of the tribal IRB. "Obtaining consent from tribal governments may involve significant time, preparation, and expense, which must be considered in planning grant applications and project deadlines" (Norton & Manson, 1996, p. 858). If research is to be conducted at an HIS facility, approval from the IHS IRB must be obtained. "It is not necessary for the researcher to know all the cultural distinctions. Rather, it is important that the researcher be aware of cultural differences" (Hodge et al., 2000, p. 43). Researchers are not expected to be experts on the more than 500 American Indian tribes and Alaska Native villages, but should learn as much as possible about the local community and tribe they plan to study. Investigators may become involved in the community through attending tribal meetings and social gatherings. Learning more about the history and culture of the tribe as well as local customs and beliefs about illness will allow the researchers to conduct a more efficient study that will prove successful to both parties involved. Showing genuine concern as well as a willingness to learn and be a part of the community can foster trust. Once the tribe or community has consented to or approved a research project, guidelines should be established collaboratively between the community and researchers. A Code of Research Ethics was developed in partnership with the Native Mohawk community of Kahnawake in Canada (Macaulay et al., 1998). The document demonstrates new concepts in the sharing of power and decision making for both the researchers and the community. Guidelines for conducting PR entail developing a policy statement, discussing the obligations of all parties involved, developing a data control plan, and developing a plan for disseminating results. The Kahnawake policy statement ensures that the cultural values, perspectives, and wishes of the community are firmly incorporated into the research plan; they emphasize the need for community empowerment (Macaulay et al, 1998). The following is an excellent example of a policy statement: The sovereignty of the Kanien'keha:ka (the people) of Kahnawake to make decisions about research in Kahnawake is recognized and respected. The benefits to the community as a whole and to individual community volunteers should be maximized by the researchers. Researchers should empower the community to support community goals of health and wellness, to promote healthy lifestyles, improve self-esteem and to fulfill its traditional responsibility of caring for the Seventh Generation. (In Mohawk tradition, the Seventh Generation represents those as yet unborn) (p. 107). The obligations of the researchers include maintaining continuous consultation and collaboration on all aspects of the research, involving the community through active participation rather than passive acceptance, transferring new skills to the community during the research process, and helping to address any health or social issues raised as a result of the research. The obligation of community-based researchers is to maintain a long-term relationship of trust in their dual role as caregivers-educators and researchers. The needs of the community should retain priority in any decision. The obligations of the community include meeting regularly with the researchers, promoting the objectives of the project, offering advice for the development and interpretation of the study, controlling the data after the project ends, and providing dissenting opinions, if needed, at the time of publication. Community control of the data throughout the research process can help ensure appropriate use of the data (Macaulay et al., 1998). The community maintains control over the dissemination of the results in the PR model. The ultimate decision on how the results are to be used, including whether to publish in scientific journals, resides with the community. Results should be presented to the tribal council and community members in a form that is interpretable and meaningful as well as in a manner that may be used by service providers and administrators (Macaulay et al., 1998, Norton & Manson, 1996). The study of diabetes mellitus and atherosclerosis in a Mohawk community (Montour & Macaulay, 1988) provides insight about successful dissemination of results. The findings of this study pointed to the serious negative consequences of diabetes for the community. The results were presented to various community groups over several sessions as well as broadcast on the local community radio station. Immediately after the results had been distributed, the Health Department began to note an increase in the number of people being screened for diabetes and seeking counseling for dietary changes, weight loss, and exercise. The schools banned junk food and initiated a daily exercise period for all faculty and students. This study, and the way the results were presented, led to a positive change within the Mohawk community. Participatory Action Research1: LessonsLearned with Aboriginal Grandmothers A two-and-a-half year program was designed to examine the unmet health needs of older urban aboriginal women and to study participatory action research (PAR) as a research method and intervention for health promotion (Dickson & Green, 2001). The primary goal of the assessment was to strengthen the Grandmothers' sense of self-worth. The Grandmothers' initial response was that research was something done to them for the benefit of the outsiders. The project was a success because the researchers partnered with the Grandmothers. A sense of community was fostered between the Grandmothers and researchers. The Grandmothers' roles in the research process were highly valued; they designed interview guides, consent forms, work plans and contracts, and provided secondary analysis and verification for interpretation of the data. The final report consisted largely of the Grandmothers' words and was approved and released only after extensive review. "Faith in the long-term process is essential for all participants of PAR because social and political change does not occur readily" (Dickson & Green, 2001, p. 481). The project concluded that, given the opportunity, support, and tools, Native people have the capacity to conduct research that is meaningful to them and contributes to their personal and social change. Participatory research balances the needs and wants of the community with those of the researchers; without this balance trust may be broken and significant problems may arise. The give and take approach is one that allows community members and researchers to remain equal partners; if researchers make use of participants' ideas, time, and/or bodily fluids, they must give back skills, employment, and/or training (Davis & Reid, 1999). Communities will benefit from research by simply addressing relevant concerns of the tribes and other Native communities. "Researchers who demonstrate a long-term commitment to the communities that they propose to study, and who are willing to enter into the partnerships implied by such commitments, will receive greater cooperation and support of their work" (Norton & Manson, 1996, p. 859). 1 PAR goes beyond PR by insisting that researchers offer tribes the support and tools to implement (i.e., take action) the research findings that will enable tribes to develop effective and culturally appropriate intervention and prevention programs. A body of literature discussing the differences of PAR and PR exists and a review is beyond the scope of this paper (see Marshall et al., this monograph) Best Practices A number of research codes of ethics exist, specifically developed by American Indian and Alaska Native communities. Here we present, in chronological order, a brief description of three formalized and one informal ethics codes. Also included in this section is a brief presentation of "cultural case formulation" and its relevance for research. Macaulay et al. (1996) developed a formalized code of research ethics to guide researchers as well as community members throughout the entire research process. The development of this ethics code involved academic researchers, community-based researchers, and the community in defining the various roles, from the initial research question and design of the project to the dissemination of research results. Specifically, The three partners will work cooperatively and collaboratively in the design, implementation, analysis, interpretation, conclusion, reporting, and publication of the experiences of the project. Each partner provides ideas and resources that come from the experience, knowledge, and capability of its members. Together, through respect for each other, consultation, and collaboration, they significantly strengthen the project and its outcomes. All three partners of the project share an understanding that community-based research is a powerful tool for learning about health and wellness, while contributing to the health of the community in which it is being conducted (p. 39). The Model Tribal Research Code was developed by the American Indian Law Center, Inc. (1999). This 28-page document represents an extensive evaluation of the need for such a code for American Indian tribes and Alaska Native villages and presents information about the appropriate as well as the inappropriate uses of such a code. Specific aspects of developing a research code of ethics, including a discussion of policy statements, scope, process, and enforcement, are presented and discussed thoroughly. In addition, a Model Tribal Research Code template is included in the document as is a checklist to aid tribal health boards and others to evaluate effectively the appropriateness and usefulness of a research proposal. Though not a formal ethics code, the Strong Heart Study (Stoddart, Jarvis, Blake, Fabsitz, Howard, Lee, & Welty, 2000), a large-scale multisite study of cardiovascular health, represents an excellent example of appropriate recruitment of American Indians into medical research. Collaboration with the IHS began early in the pre-award stage of the project and participating communities were involved in the proposal evaluation process and the methodology development process. Participating tribal members developed posters and logos used to inform and recruit community members. Brochures were developed to explain the purpose and rationale of the study, the procedures that would be utilized, and a community-identified point-of-contact who could answer questions and schedule appointments for participation in the study. Community-based recruiters contacted potential participants. Mass mailings and local radio broadcasts were used to promote participation in the study. Recruitment strategies included benefit powwows and health fair participation. The success of this study is attributed to the.16 multimethod recruitment strategy employed (i.e., meeting the various needs of potential participants) as well as to the respect for community needs, cultural differences, and the flexibility used to meet the needs of the researchers and the research participants. The National Science Foundation Arctic Program developed Principles for the Conduct of Research in the Arctic (http://www.nsf.gov/ od/opp/arctic/conduct.htm). These guidelines were developed to "promote mutual respect and communication between scientists and northern residents" (p. 1). The 13 principles emphasize the need to assess scientific investigations in terms of the potential impact on and interests of Native people and their communities. The recommendations outline the need to obtain full and informed consent explained in terms understandable to the community; to consult with the communities throughout the research process; to communicate results in a manner responsive to community needs; to provide appropriate credit for community contributions; and to respect anonymity, local cultural traditions, cultural property, and sacred sites. Historically, the Diagnostic and Statistical Manual of Mental Disorders (DSM) has paid little attention to the role of culture for psychiatric diagnosis. The Fourth Edition of the DSM (American Psychiatric Association, 1994) breaks from this tradition and acknowledges the importance of culture for "the expression, course, treatment, and existence of psychiatric disorders" (Christensen, 2001, p. 53). Christensen discusses the relationship between cultural competence and psychiatric diagnosis and presents an example of cultural case formulation for an American Indian client. Cultural identity; cultural explanations of illness; the relationship among the psychosocial environment, culture, and levels of functioning; and cultural and social status differences between the individual and the clinician are components necessary to understanding the role of culture for the individual. Although not specifically designed for research purposes, these principles are relevant for researchers studying psychiatric disorders as well as researchers conducting nonclinical investigations. The Future of Research in American Indian and Alaska Native Communities Research practices have changed dramatically over the past 150 years. American Indian and Alaska Native communities demand that research benefit their people and that the research be conducted in a collaborative and participatory manner. Continuing to develop research that addresses the health needs of American Indian and Alaska Native people is of vital importance. The goal of this section is to foster thoughts about the future direction of research in these communities and for these people. Research that addresses the use of direct service referrals by providers, employers, schools, union, family members, and self-referrals is needed. Such research will help communities understand how to make better use of available resources. Augmenting knowledge about service utilization rates can increase awareness about who is not being reached and how better to include these individuals. This type of research has particular relevance for elders and individuals with disabilities. Stigma also influences utilization rates. Understanding what contributes to stigma in a particular community, and what can done to minimize the effects of stigma, can help the community develop more effective outreach programs. Encouraging employment of tribal members in tribal programs is necessary; however, important issues can occur when family members, friends, or neighbors provide services or work in these programs. Individuals may be reluctant to disclose information openly to community members if there are concerns about indiscreet use of personal information. For example, concerns about confidentiality may develop when family members work for health programs. Research may be able to help us gain a better understanding about such issues. Research often focuses on the problems or concerns faced by a community or group of individuals. Additional research focused on strengths is needed to help clinicians better utilize the inherent strengths of individuals and promote wellness. Considering the history of problems relative to conducting research in American Indian and Alaska Native communities, considerable disparities exist in what we know about these communities as compared with what we know from research about the general population. Children, women, aftercare, violence, health care, traditional medicine-nearly every facet of life for Native people is fertile ground for investigation to benefit the people and the Nations. Conclusion Research is essential for understanding the health and well being of American Indian people. The usefulness of research is diminished when the community's vitality is overlooked. Communities are made up of people-individuals who live, breathe, and love, individuals who feel pain as well as joy. Several tribal research codes of ethics are available to aid investigators in designing appropriate research (e.g., American Indian Law Center, Inc., 1999; Macaulay et al., 1998). Existing models, such as the participatory action research method (Dickson & Green, 2001), provide practical examples of effective research practices in American Indian and Alaska Native communities. Both researchers and community members can benefit by establishing effective research partnerships. References American Indian Law Center, Inc. (1999). Model tribal research code: With materials for tribal regulation for research and checklist for Indian health boards (3rd ed.). Albuquerque, NM: American Indian Law Center, Inc. American Indian Lawyer Training Program, Inc. (1988). Indian tribes as sovereign governments: A sourcebook on federal-tribal history, law, and policy. Oakland, CA: American Indian Lawyer Training Program, Inc. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Press. Attneave, C. (1982). American Indians and Alaska Native families: Emigrants in their own homeland. In M. McGoldrick, J.K. Pearce, & J. Giordano (Eds.). Ethnicity and Family Therapy (pp. 55-83). New York: The Guilford Press. Brelsford, G. (1977). Ggwangkumtenek sungcarluta. Rockville, MD: Indian Health Service. The Brown Quarterly (2001, Fall). The challenges and limitations of assimilation: Indian boarding schools, 4(3). Retrieved March 2, 2002, from http://brownvboard.org/ brwnqurt/04-3/04-3a.htm Christensen, M. (2001). Diagnostic criteria in clinical settings: DSM-IV and cultural competence. American Indian and Alaska Native Mental Health Research: Journal of the National Center, 10(2), 52-66. Cooper, J.M. (1934). Mental disease situations in certain cultures-a new field for research. Journal of Abnormal Social Psychology, 29, 10-17. Davis, S., & Reid, R. (1999). Practicing participatory research in American Indian communities. American Journal of Clinical Nutrition, 69(4), 755s-759s. Dickson, G., & Green, K. (2001). Participatory action research: Lessons learned with aboriginal grandmothers. Health Care for Women International, 22, 471-482. Dowler, B. (1857). Researches into the sanitary condition and vital statistics of barbarians. New Orleans Medicine and Surgery Journal, 14, 335-352. Elliott, H.W. (1878). Wild babies. Harper's Monthly, 57, 829-838. Equity Center Infoline (1999, Dec.). Focus on American Indian education, 7(1). Retrieved March 2, 2002, from http://www.nwrel.org/cnorse/infoline/dec99/5.html Flemming, H.C. (1923). Medical observations made on Zuni Indians. Nation's Health, 5, 506-508. Foulks, E.R. (1989). Misalliances in the Barrow alcohol study. American Indian and Alaska Native Mental Health Research: Journal of the National Center, 2(3), 7-17. Hodge, F., Weinmann, S., & Roubideaux, Y. (2000 Nov.). Recruitment of American Indians and Alaska Natives into clinical trials. Annals of Epidemiology, 10(8 Suppl), S41-S48. Indian Health Manual. (2001). Retrieved March 2, 2002, from http://www.ihs.gov/PublicInfo/Publications/ IHSManual/ part1/pt1chapter7/pt1chpt7.htm Indian Self-Determination and Education Assistance Act of 1975, Pub. L. No. 93-638, 25 USCS §§ 450. 88 Stat. 2203 et seq. (2002). Klatsky, M. (1948). Studies in the dietaries of contemporary primitive peoples. Journal of the American Dental Association, 36, 385-391. Kunitz, S.J. (1996). The history and politics of U.S. health care policy for American Indians and Alaskan Natives. American Journal of Public Health, 86(10), 1464-1473. Macaulay, A.C. (1994). Ethics of research in Native communities. Canadian Family Physician, 40, 1888-1890. Macaulay, A.C., Cross, E.J., Delormier, T., Potvin, L., Paradis, G., & McComber, A. (1996). Developing acode of research ethics for research with a Native community in Canada: A report from the Kahnawake schools diabetes prevention project, 57(Suppl 1), 38-40. Macaulay, A.C., Delormier, T., McComber, A.M., Cross, E.J., Potvin, L.P., Paradis, G., Kirby, R.L., Saad-Haddad, C., & Desrosiers, S. (1998 Mar-Apr). Participatoryresearch with Native community of Kahnawake creates innovative code of research ethics. Canadian Journal of Public Health, 89(2), 105-108. Montour, L.T., & Macaulay, A.C. (1988). Diabetes mellitus and atherosclerosis: Returning research results to the Mohawk community. Canadian Medical Association Journal, 139, 201-202. The National Science Foundation Arctic Program. Principles for the conduct of research in the Arctic. Retrieved March 2, 2002, from http://www.nsf.gov/od/ opp/arctic/conduct.htm Norton, I.M., & Manson, S.M. (1996). Research in American Indian and Alaska Native communities: Navigating the cultural universe of values and process. Journal of Consulting & Clinical Psychology, 64(5), 856-860. Oliver, A. (1764). An account of an extraordinary disease among the Indians, in the islands of Nantucket, and Martha's Vineyard, in New England. Philanthropic Transatlantic Royal Society of London, 54, 386-388. Patrick, R., & Tyroler, H.A. (1972). Papago Indian modernization: A community scale for health research. Human Organization, 31(2), 127-36. Stoddart, M.L., Jarvis, B., Blake, Fabsitz, R.R., Howard, B.V., Lee, E., & Welty, T.K. (2000). Recruitment of American Indians in epidemiologic research: The Strong Heart Study, 9(3), 20-37. Styron, E.R. (1997). Native American education: Documents from the 19th century. Retrieved March 2, 2002, from http://www.duke.edu/~ehs1/education/ index.html Taylor, J.E. (1975). Cherokee tribal health evaluation report. Rockville, MD: Indian Health Service. Research in Indian Country: Challenges and Changes Walter Hillabrant, Ph.D. Abstract Selected settings, circumstances, problems, and barriers to research in Indian Country are described and ways to mitigate or overcome these problems and barriers are discussed. The reemergence of Indian self-determination and self-governance, federal requirements for review of research by institutional review boards (IRBs), and the Office of Management and Budget (OMB) have greatly affected the ways research is conducted in Indian Country. Tribes are gaining increasing influence over setting research agendas and funding priorities, and have, at a minimum, the power to reject unwanted research on their lands. Increasingly, researchers are required to identify and justify to concerned tribes the costs, benefits, and risks associated with participation in a study. This paper describes selected settings, circumstances, problems, and barriers to research in Indian Country, and suggests ways to mitigate or overcome these problems and barriers. Examples of issues, problems, and approaches taken from actual research projects are presented in four text boxes distributed throughout this paper. Background Stakeholders of Research in Indian Country The distinction among sponsors, consumers, and stakeholders of research in Indian Country is central to the theme of this paper. The primary sponsor of research in Indian Country is the federal government, with research generally conducted for or by federal agencies. These agencies may conduct the research directly using their own staffs. More often, however, agencies fund the research, which is conducted, under contract, by other entities and organizations. The consumers of research conducted in Indian Country include the organizations conducting the research, the sponsoring agencies, Congress, universities, and other research institutions. In recent years, consistent with the principles of Indian self-determination and self-governance, sponsors of research in Indian Country have started to relate to Indian tribes and Alaska Native villages (and related organizations) as consumers of the research findings. Stakeholders of research in Indian Country include sponsors, consumers, tribes, Alaska Native villages, their members, tribal consortia, Indian organizations, and Indian people who are not members of a tribe. Because there are more than 550 Indian tribes and Alaska Native villages within the United States, national and regional associations, organizations, and consortia represent important stakeholders of research in Indian Country, especially research that includes or affects more than one tribe. These Indian organizations are often organized around a particular area of interest. Examples include the National Congress of American Indians (NCAI), the National Indian Health Board (NIHB), the National Indian Education Association (NIEA), and the American Indian Higher Education Consortium (AIHEC). In addition, most tribes and Alaska Native villages are members of consortia or regional organizations such as Bristol Bay Native Corporation and the Tanana Chiefs Conference in Alaska, the United South and Eastern Tribes (USET), the All Indian Pueblo Council (AIPC), and the Southern California Tribal Chairman's Association (SCTCA). 1 Throughout this paper, Indian Country refers to any lands owned by American Indian tribes, Alaska Native villages, and consortia of tribes or Native villages, and to locations where large numbers of American Indians or Alaska Natives reside. Unless otherwise indicated, Indian refers to members of American Indian tribes, Alaska Native villages, and their descendants. Federal Agencies Sponsoring Research in Indian Country Occasionally Congress mandates studies and other forms of research in Indian Country; however, the bulk of the research is procured by or conducted by federal agencies. Most of this research is guided by each agency's mission and goals; in addition, some research is conducted to meet the requirements of the Government Performance and Results Act (GPRA) of 1993 that requires strategic planning and annual program performance reporting for every federal agency. In passing GPRA, Congress sought improved accountability for program performance and better planning of federal programs. GPRA has resulted in increased evaluation and other research to demonstrate program results. Federal agencies that have conducted or sponsored the greatest amount of research in Indian Country include the Department of Health and Human Services (DHHS), the Bureau of Indian Affairs (BIA), the Department of Justice (DOJ), and the Department of Education (ED). Among these agencies, DHHS sponsors the greatest amount of research. Within DHHS, research is funded by the Indian Health Service (IHS), the National Institutes of Health (NIH), the Substance Abuse and Mental Health Services Administration (SAMHSA- which includes the Centers for Substance Abuse Treatment [CSAT], the Center for Substance Abuse Prevention [CSAP], and the Center for Mental Health Services [CMHS]), the Centers for Disease Control (CDC), the Administration for Native Americans (ANA), and the Administration for Children and Families (ACF). In the last 10 years, DOJ has become increasingly active in promoting research in Indian Country. Within DOJ, agencies active in conducting or funding research include the Office of Justice Programs (OJP), which includes the National Institute of Justice (NIJ), the Drug Courts Program Office, and the Office of Juvenile Justice and Delinquency Prevention. Educational research in Indian Country is funded by both the BIA and ED. Other federal agencies that sponsor research in Indian Country include the Departments of Labor (DOL), Housing and Urban Development (HUD), Commerce (DOC), and Transportation (DOT). Increasingly, but slowly, federal sponsors of research have begun to consult with tribes and tribal organizations in setting the agendas and guidelines for research to be conducted in Indian Country. Congress and federal agencies are soliciting input from tribes and tribal organizations with respect to research goals and research budgets. Some examples of agencies would be: National Indian Health Board http://www.nihb.org/inside_budget.htm National Congress of American Indians http://www.ncai.org/main/pages/legislative_updates/index.asp United South and Eastern Tribes, Inc. http://usetinc.org/resolutions.html National Indian Education Association http://www.niea.org/2001niearesos.pdf Tribes and tribal organizations track proposed legislation and provide information to congressional representatives and their staffs and to congressional committees. Federal agencies often require researchers to establish project advisory committees that include tribal officials and experts on related research in Indian Country. Nature of Research Conducted in Indian Country The full range of research activities occurs in Indian Country, including randomized experiments, clinical trials, field experiments, quasi-experimental research, retrospective and prospective studies, cross-sectional and longitudinal studies, survey research, feasibility studies, evaluation research (including outcome, impact, and process evaluations), ethnographic research, participant observation, and case studies. The types of research conducted in Indian Country are not distributed evenly across sponsoring agencies. Most of the randomized experiments and clinical trials are conducted under the auspices of the NIH. For methodological (e.g., sampling) and logistical reasons, most research takes the form of evaluations, particularly process evaluations, feasibility studies, and case studies. Sampling issues are especially problematic for research in Indian Country because of an implicit political dimension-the heart of sampling involves the selection of a subset of a defined universe or population. As a rule, it is not practical to define the universe population of interest as "members of all Indian tribes" or "all Indians" because there are over 550 federally recognized tribes, and there is great variation across tribes-variation in language, culture, location, economy, and a host of other factors. Thus, some researchers have discovered something that eludes many non-Indian people-American Indians are not a homogeneous group; rather, most are members of unique tribes. Compounding the sampling problem is the large American Indian diaspora-more Indians reside in cities and other off-reservation locations than on or near reservations and other tribal lands (Hillabrant, Romano, & Stang, 1992). Because of the great diversity across tribes in many domains, research findings obtained at one tribe, or on a group of tribes, may not be representative of other tribes. To plan and draw a representative (i.e., random) sample of American Indians and/or Alaska Natives residing on or near tribal lands would be a daunting task, beyond the means and resources of most research projects. Identifying a sample frame and drawing a representative sample from a single tribe or a group of tribes often presents problems that challenge the resources available to most research projects. Data commonly used to identify primary sampling units are often incomplete or unavailable. For example, many individuals may reside in dwellings without a published address or even a named street, lack Social Security numbers or telephones, may have multiple residences, and may be away from their primary residence for long periods of time fishing, hunting, trapping, and gathering/harvesting. Many of these challenges can be attenuated and managed through close collaboration with tribal officials and residents of the communities participating in the research. Of course, it is possible to draw random, stratified random, probability proportionate to size, and multistage samples of well-defined groups in Indian Country. For example, a prospective study of substance abuse treatment outcomes included a first stage selection of 22 treatment facilities probability proportionate to size (number of clients served the prior year) within strata (IHS Areas), and quota samples of all clients accepted for treatment at randomly determined start dates (Hillabrant, Earp, & Rhoades, 2001). In a retrospective study of youth residential substance abuse treatment facilities, random samples of client records were selected for review at each of nine facilities (Hillabrant, Earp, & Brutus, 1997). An assessment of the expected impact of a change in the rule governing eligibility for services from the IHS included a random sample of the HIS user population-all IHS beneficiaries using an IHS or tribal facility in the last 3 years (Hillabrant, Earp, & Brutus, 1996). In this study, the sample frame was defined by automated patient registration and utilization data. Emerging Tribal Influence on Research in Indian Country Over the last 25 years, tribes, Native villages, and consortia have taken an increasingly active role in controlling research conducted in Indian Country. This active stance reflects the increasing autonomy exercised by tribes in taking control over programs such as education, the provision of health care services, employment, training programs, and welfare programs formerly operated by federal agencies. With the enactment of the Indian Self-Determination and Education Assistance Act (ISDEAA) of 1975 (Pub. L. 93-638), tribes began to exercise ever increasing levels of control over programs, policies, and research affecting their members. The increase in tribal control of research has been abetted by federal regulations requiring protection of people participating in research (21CFR Part 56). These regulations require that all research involving human participants be reviewed and approved by an institutional review board (IRB). Now, some tribes have constituted their own IRBs, which review all research conducted on or near the reservation in which tribal members are participants in the research. Regardless of whether or not a tribe has instituted an IRB, tribal approval is required for any research conducted on tribal lands. The concern about protection of people participating in research, in turn, reflects revelation of unethical research in which participants were not informed about the risks involved or were deprived of treatment and care that could have saved their lives or ameliorated their conditions. Among the most publicized unethical research are 1) a study sponsored by the U.S. Public Health Service in which 400 African American men in Macon County, Georgia, were deprived of medication to cure syphilis and not told of their illness, putting others at risk of contracting the disease (Tuskegee Study); 2) a study conducted by the U.S. Army in which soldiers were given the drug LSD; and 3) studies where prison inmates and terminally ill patients received injections of cancer cells to study the response of their immune systems (Lemonick & Goldstein, 2002). Overcoming History of Exclusion from Control of Research In the 1800's, often after successful military campaigns and other conflicts with American Indians, the United States treated Indian people as wards of the federal government (Cherokee Nation v. Georgia, 1831; American Indian Policy Review Commission, 1997). In this trustee capacity, federal agencies, often acting through the BIA and, later, the IHS (with respect to health care), determined what research would be conducted, which tribes would participate in the research, and which tribal members would serve as research participants. While the legacy of this trustee relationship still reverberates today, tribes have increasingly expanded their sovereign control over programs affecting their members since the passage of ISDEAA in 1975. Because of a long history of exploitation and both benign and hostile neglect by the United States, many tribes and tribal organizations want to be involved in all aspects of research in Indian Country, including planning, design, data collection, reporting of, and dissemination of research findings. Tribes and tribal organizations are requiring that researchers 1) hire tribal members to assist in data collection and other research activities; 2) demonstrate how the research findings will benefit the tribe and tribal members; 3) guarantee that the research protocol does no harm to the tribe, tribal members, and the environment; 4) guarantee confidentiality or anonymity of research participants, tribal communities, and the tribe; and 5) publish results only after review and approval of the manuscript by tribal representatives (see e.g., American Indian Policy Review Commission, 1997). These requirements and limitations increasingly exercised by tribes can both benefit and impede research in Indian Country. 1. Hire tribal members. Requiring researchers to hire tribal members to assist in research activities can have several benefits. Such hiring increases employment and contributes to the tribal economy. Tribal members assisting with the research acquire skills and knowledge. Having tribal members participate on the research team is associated with other benefits, including: a. The research team is less likely to exploit research participants, expose them to unnecessary risk, or demean them. b. Researchers are less likely to employ or express invalid stereotypes, or to express, explicitly or implicitly, prejudicial perceptions, opinions, or expectations. c. Often, the quality of the data collected is improved because of enhanced communication and better rapport between the researchers and the study participants. On the other hand, employing tribal members on the study team may be associated with a variety of costs. Hiring, training, and using tribal members on a research team can increase both the time required to complete the project and project costs. Using tribal members to collect data can create special problems for study participant confidentiality, and change the demand characteristics of the research (Orne, 1962). Study participants may conceal taboo or socially undesirable expectations, beliefs, or behaviors from other tribal members; such expectations, beliefs, or behaviors might be less concealed from outside investigators whom the research participants are unlikely to encounter in the future. 2. Demonstrate how the research findings will benefit the tribe and tribal members. It is customary for researchers to describe the anticipated benefits (and costs) of the research, especially in preparing grant proposals and in submissions to IRBs and OMB. Prior to the emergence of tribal control of research, discussions of the anticipated benefits of research in Indian Country tended to be abstract, making reference to Indians in general rather than to the participating tribe or tribes. Requiring researchers to relate costs and benefits to the participating tribe(s) tends to clarify thinking and make assumptions and expectations explicit-results often beneficial to the proposed research, the researcher, the participating tribe, and other stakeholders. On the other hand, research in Indian Country, like research everywhere, is conducted in a social and political context. The perceived benefits and costs associated with a particular study may vary across stakeholders. For example, managers of tribal programs may perceive costs and benefits of a study differently from members of the tribal judiciary or legislature. Proponents of traditional values and approaches may differ from advocates of change. While involvement of tribal stakeholders with conflicting values, agendas, or perspectives can improve the robustness and acceptance of a study, such involvement can also mire the research in conflict, leading to disorganization or even abandonment of the research. Depending on their interests, stakeholders may embrace or repudiate research findings (see Vignette 1). Vignette 1. Repudiation of unpopular results: Evaluation of the IHS Fiscal Intermediary (FI) The IHS commissioned an independent evaluation of the performance of the FI in order to determine the degree to which the FI was achieving the goals and objectives specified in its contract. These responsibilities included 1) receiving bills and claims submitted by hospitals, clinics, and other providers outside of the IHS; 2) reviewing the claims to ensure conformance with treatment referrals and authorizations; 3) paying the valid claims and bills; and 4) producing reports of the services provided and amounts paid. The evaluation was commissioned in the context of strong and ongoing criticism of the FI by staff in IHS and tribal clinics and by tribal officials. The major complaint about the FI was that it failed to pay bona fide claims within the 45-day time frame mandated by its contract with the IHS. Tribal officials were especially frustrated because providers submitted unpaid bills and claims to collection agencies. In turn, collection agencies hounded tribal members for payment of the claims unpaid by the FI. Site visits to the FI and to IHS and tribal clinics conducted as part of the evaluation produced convincing evidence that the primary causes for the delayed payments were incomplete referrals and treatment authorizations submitted by clinic staffs. As a rule, when the FI received complete and valid information from clinics, it paid the claims in a timely fashion. The evaluation recommended a number of ways that the referrals and authorizations submitted by the clinics could be improved. The evaluation results were accepted by many stakeholders; however, some, hoping that the study results could be used to terminate the FI contract, repudiated the findings and ardently tried to suppress them (Hillabrant, Brown, Weller, & Earp, 1988). 3. Guarantee that the research activity does no harm to the tribe, tribal members, and the environment. The level of risk to study participants, the tribe, and the environment and the potential to do harm depend on the nature of the research, with medical clinical research often bearing the greatest risk of harm to participants. However, risk is not limited to medical research, and all research has costs, even if only opportunity costs-time and resources spent on the research activity could be spent on alternative activities. Opportunity costs are especially important in situations where there are significant unmet needs, as is the case in much of Indian Country. Critics of research, including members of tribal IRBs, have suggested that most research funds would be better spent on service delivery. Of course, such judgments should consider the relative costs and benefits of a research project. It is the responsibility of the research sponsor(s) and those conducting the research to present the costs and benefits clearly to the tribe, the IRBs, OMB, and the study participants. It is the responsibility of the tribe, IRBs, and OMB to determine if the benefits outweigh the risks, that the risks do not exceed acceptable limits, and that participants are not exposed to unnecessary risks (see Vignette 2). Vignette 2. Risks involved in a screening feasibility study Three tribes agreed to participate in a study to demonstrate the feasibility of a self-administered, multimedia, computer-assisted screening system to identify adolescents who 1) are victims of physical or sex abuse, 2) suffer from posttraumatic stress syndrome (PTSD), and 3) abuse alcohol and other drugs. The system was to be set up at tribal clinics providing primary care to tribal members; adolescents coming to the clinic for care would be recruited to participate in the feasibility study. All screening research includes the risk of two errors: 1) a false positive error occurs when the screening system indicates that the adolescent has been abused, traumatized, or involved in substance abuse when, in fact, no abuse, trauma, or substance abuse has occurred, and 2) a false negative error occurs when the screening system fails to detect abuse, trauma, or substance abuse when one or more, in fact, occurred. In addition, other risks associated with the proposed screening included 1) mental health/emotional crisis or problems associated with "hits" (correct detection by the screening system), and 2) adverse consequences should the health care delivery, child protection, or other systems fail to provide services needed by adolescents successfully identified by the screening. There are subtle aspects to the "do no harm" dictum for research in Indian Country. Many consumers of research in Indian Country may be ignorant of 1) the history of exploitation and discrimination faced by Indian tribes, communities, and individuals; 2) the circumstances of many tribes (e.g., remote location, weather extremes, lack of infra-structure); and 3) the legal and constitutional status of tribes. Without knowledge of these circumstances, such consumers are unlikely to appreciate the significance of research findings. While such factors are too numerous to list, several favorites of the author include: • The unique circumstances of Alaska. Many if not most of the people who have never visited Alaska do not understand that it is farther from Anchorage to half-way through the Aleutian Islands than from Washington, DC to San Francisco, CA. Most Alaska Native villages have a single, unpaved road with an airstrip at one end. While there may be 20 trucks and cars in the village, they can leave the road only in winter when the tundra, lakes, and rivers freeze hard. • Many residences in Indian Country lack electricity, telephones, and running water. • For over 100 years, to the present day, the United States has been unable to account for money and resources held in trust for tribes and individual Indians. • Tribes are explicitly mentioned in Article I, Section 8 of the U.S. Constitution (restricts to Congress the power to regulate commerce with the tribes), and citizenship was not conferred on American Indians until 1924, with the passage of the Indian Citizenship Act of 1924. • Starting in the late 1800s, some Indian children were forced to leave their families to attend BIA boarding schools, where non-Indians prescribed the language they spoke, the religion they practiced, and the material they studied. Growing up apart from their families, frequently subjected to what is now (and often then) seen as physical and emotional abuse, had multiple, cross-generational negative effects on these children, their families, and tribes. While some of these boarding schools still exist, forced separation of children from their families ended, for the most part, by the 1950s. Because many consumers of research in Indian Country are ignorant of these and other important data, researchers often need to provide background information so that the research results can be understood in context. 4. Guarantee confidentiality or anonymity of research participants, tribal communities, and the tribe. Often, tribes do not object to the identification of the tribe or of tribal communities in research reports. Other times, especially when research is conducted on sensitive topics, a tribe may insist that the research report not identify the tribe or communities participating in the research. Protecting the privacy of research participants and keeping their identity anonymous can pose a special challenge for small tribes and communities. If the program being evaluated is small, it can be almost impossible to maintain the anonymity of key informants who provide relevant information (see Vignette 3). Breaches of confidentiality and identification of tribal communities by researchers, despite their agreement not to do so, have been the source of frustration and reactions against research by several tribes (William Freeman, personal communication, June 30, 1999). Vignette 3. Ethical problem: Substance abuse treatment center director "using" As part of a national evaluation of substance abuse treatment outcomes, researchers visited 22 treatment programs in Indian Country. At one of the programs, a small facility with a few staff, several staff members told the research team that the program director had relapsed and was "drinking every day, but without becoming grossly inebriated." The staff said that while they did not directly observe the program director drinking, they noted changes in her behavior and claimed to smell alcohol on her breath. When asked why they did not report this problem to the appropriate tribal authorities, the staff said that there was no one at the treatment facility who supervised the director, and each feared the loss of his or her job if they were to report the problem. Some staff wanted the researchers to pass on the information to tribal officials, but other staff, fearing reprisals, asked the researchers not to pass on the allegations to others. It seemed likely to the researchers that reporting the alleged drinking to tribal officials could not be done without compromising the confidentiality of some of the participants (project staff). The researchers were unable to independently confirm or disconfirm the alleged drinking by the program director. Some staff said that the program director was able to "remain clean" during the two-day site visit. Alternatives considered by the researchers included ignoring the allegations made by the staff, confronting the program director with the allegations, or passing on the allegations to the office of the tribal chairman. The members of the research team decided that they would not pass on to tribal officials the allegations made by the staff; rather, they would contact the agency sponsoring the study for guidance. Subsequently, the researchers contacted tribal officials and discovered that the treatment program director had already resigned (Hillabrant, Earp, & Rhoades, 2001). 5. Publish results only after review and approval of the manuscript by tribal representatives. This requirement is often imposed by prudent researchers as well as by tribes. By obtaining review and feedback from the appropriate tribal representatives (generally persons who are knowledgeable about the research topics), researchers can often identify errors or omissions which otherwise might go uncorrected. Thus, such review can improve the quality of research conducted in Indian Country. Over the last five years, federal sponsors of research have begun to disseminate research findings to all federally recognized tribes, tribal consortia, and Indian organizations. Both the BIA and IHS maintain a "tribal leaders" database available to the public and used as a mailing list for disseminating research reports (see, for example, http://www.ihs.gov/AdminMngrResources_ index.asp). Increasingly, research reports are being published on the World Wide Web, available to tribes and the public at large, such as the report from Hillabrant, Rhoades, Pindus, and Trutko, available at the U.S. Department of Health and Human Services website (http://aspe.hhs.gov/ hsp/wtw-grants-eval98/tribal02/). Participatory, Self-, and Empowerment Evaluation Program evaluation is supposed to be an integral component of program administration and improvement. When the program goals and objectives are formulated or reformulated, the evaluation is to be designed to assess progress toward the goals and objectives, and the evaluation results are to be used to inform and guide reformulation of program goals, objectives, policies, and procedures. Too often, it seems, the evaluation plans are lost or ignored, the evaluation is perceived by program managers and staff as a distraction from their work, and the evaluation results do not have the desired impact on the program. In reaction to the underutilization or improper utilization of program evaluation, there has been a growing interest in conducting evaluation research in ways that foster improvement and self-determination of organizations, communities, and tribes; this approach has been termed participatory or empowerment evaluation (Fetterman, 2000). Empowerment evaluation seeks to help people help themselves and improve their programs through self-evaluation, reflection, and capacity building. Program participants conduct their own evaluations with the aid of an outside evaluator who often serves as a coach or facilitator. Because of their unique political, cultural, and historical circumstances, Indian tribes may find the empowerment evaluation approach especially useful. Approval Processes Obtaining the required approvals for research, especially in the areas of health and education, can be a complex, lengthy, and expensive proposition. Often, there are three levels of review: IRBs, OMB, and tribal; each is discussed below. IRBs Institutional review boards (IRBs) are authorized and required by federal regulations (Title 45, Part 46 of the Code of Federal Regulations [CFR]as well as 21 CFR Part 50, protection of human subjects; 21 CFR Part 56, institutional review boards). These regulations pertain to research sponsored by the Departments of Agriculture, Energy, Commerce, Housing and Urban Development, Justice, Defense, Education, Veterans Affairs, Health and Human Services, and Transportation, as well as to research sponsored by other federal agencies such as the National Aeronautics and Space Administration, Consumer Product Safety Commission, Agency for International Development, and the Environmental Protection Agency. The primary objective of IRBs is the protection of human research participants. This protection is all-encompassing, including physical and mental health. Interested readers are referred to the Office of Human Subjects Research in the DHHS National Institutes of Research (http://www.dvincitbt.com/ohsrsite). Fundamental to the protection of research participants is the concept of "informed consent." Informed consent means that the study participant understands all the risks and rewards associated with participation in the research. This understanding is to be conveyed to the research participants by the research staff in language that the participant can fully understand. The research participant must indicate his or her understanding of the risks and rewards associated with the research by signing an informed consent form, and the researchers are required to maintain the original forms while the research is being conducted. In Indian Country, there may be multiple levels of IRB review, including national (agency-wide), area or region, and tribal. While it is possible to submit applications simultaneously to national and area or regional IRBs, it is not uncommon for one group or the other to require the prior approval of the other before completing their review. Since the IRB may meet only once each quarter, obtaining the necessary IRB approval may take one year or more, especially when one of the IRBs requires changes in study protocols, procedures, or data collection instruments. Some IRBs require the principal investigator to make a presentation to the board and to answer questions. Prudent researchers include the costs of such presentations in their budgets and project timelines. Obtaining informed consent for research in Indian Country involves special challenges. The researcher must be sensitive to the ways that culture affects the participant's understanding and interpretation of the study. Some cultural components may seem relatively obvious, such as the need for presentation of the information in the participant's primary language; however, the difficulties in translation are often revealed by heated discussions among translators as to what expression or metaphor should be used to communicate key concepts or aspects of the research. Other cultural components that affect obtaining informed consent and other aspects of research in Indian Country might include: • Norms about the expression of disagreement or conflict; • Kinship systems, the perception of how individuals are related, and norms governing interaction among related individuals; • Childrearing practices and associated norms and expectations; and • Models of health and disease and associated norms, expectations, attributions, and perceptions. Cultural factors that are unknown to the researchers can wreak havoc with research-"it's what you don't know that can really hurt you." One approach to identifying cultural factors likely to affect a study is the use of an advisory committee that includes key informants from the community or communities participating in the research, as advocated by other authors in this monograph. Even with such an advisory committee, unexpected cultural factors can damage research. In an IHS-sponsored national evaluation of substance abuse treatment outcomes for American Indian and Alaska Native women, no data were obtained from one tribal treatment program because most women served by the program refused to participate in the study. Many of these women were angry because 1) they had been forced to enter into treatment by a judicial proceeding, and 2) men were not required (or allowed) to participate in the study. While these circumstances were not unique to the tribe in question, they interacted with the unique aspects of the tribe's culture pertaining to gender roles; the other 21 tribal programs in the study successfully recruited into the study women in similar circumstances (Hillabrant, Earp, & Rhoades, 2001). Another aspect of research participant protection is the maintenance of confidentiality of participant data. Researchers must demonstrate to the IRB that no one, except the researchers, will be able to obtain or infer information about individual participants collected during the research. Researchers working in the area of substance abuse can face special challenges to maintaining participant confidentiality. Because substance abuse can include the use of illegal drugs (e.g., marijuana, cocaine, heroin, or the use of prescription drugs without a legitimate prescription), law enforcement officials may request study data and, in particular, the names of research participants who use illegal drugs. Such requests can come in the form of a subpoena issued by a court. A Certificate of Confidentiality, issued by the National Institute on Alcohol Abuse and Alcoholism (NIAAA), authorizes researchers to withhold the names and other identifying characteristics of people who participate as subjects in the research project specified in the certificate. This authorization applies to all persons who, in the performance of their duties in connection with the research project, have access to information that would identify the subjects of that research. Persons so authorized may not, at any time, be compelled to provide the names or other identifying characteristics of the research participants encompassed by the certificate in any federal, state, or local civil, criminal, administrative, legislative, or other proceedings. The statutory authority for the certificate of confidentiality is section 301(d) of the Public Health Service Act (part USC 241 [d]). The applicable regulations are set forth at 42 CFR Part 2a. OMB Review The Paperwork Reduction Act of 1995 requires that the Office of Management and Budget (OMB) review and approve all federally sponsored research, other than grants, that includes more than nine individuals. The materials are submitted to the OMB by the funding federal agency; however, contractors and researchers often draft the "OMB submission package" for the client agency. The OMB package includes: • Justification of the research, including proposed use of the study findings; • Explanation of information technology used to promote efficiency and reduce the burden of the research participants; • Explanation of how participant confidentiality will be assured; • Estimates of burden on research participants and cost of the research; • Project schedule and publication plans; • Statistical methods to be used (generally includes sample design, research design, and planned analyses); • Notices describing the study to be published in the Federal Register. The process of preparing the OMB submission package, responding to questions raised by the OMB, and obtaining the required approval or clearance generally requires 6-12 months. In the past, some agencies have taken the stance that research conducted in Indian Country is exempt from OMB review because of tribal sovereignty. The position taken is that the research is being conducted on behalf of the tribes, and the tribes are not subject to the Paperwork Reduction Act. Such research has proceeded without obtaining an OMB clearance number and, presumably, without OMB's knowledge. Tribal Approval Whether or not a tribe operates an IRB, research on tribal lands requires tribal approval. If the tribe has delegated responsibility for review and approval of research proposals to an agency or office, the review process is greatly simplified. If no such delegation exists, the tribal council, business committee, or other group reviews the research proposal; such review can be a lengthy process. The approval process can be expedited, and the quality of the research improved, if the researchers identify and persuade stakeholders at the tribe of the merits of the study. An advocate for the research who works for or with each tribe can make the difference between a long review process ending in disapproval, and an expedited, successful review. For example, one tribe approved its participation in the national welfare-to-work (WtW) evaluation within four weeks and approved participation in the women's substance abuse treatment outcome study only after two years of deliberations (Hillabrant, Rhoades, Pindus, & Trutko, 2001; Hillabrant, Earp, & Rhoades, 2001). While the two studies varied across many dimensions, two critical factors seemed to account for the difference in speed of approvals: 1) approval of the tribal IRB was required in the treatment outcome but not in the WtW evaluation, and 2) the WtW evaluation had the strong support of key stakeholders (heads of tribal departments) whereas the treatment outcome study lacked a strong advocate at the tribe. Interestingly, the treatment outcome study was supported by some stakeholders; the substance abuse treatment programs supported the study, but corresponding support was lacking among stakeholders with the needed "influence." Sometimes stakeholders at tribes are enthusiastic about participating in a research project because they want to share successful approaches with other tribes. For example, in a study of family violence in Indian Country, one tribe had developed a model approach that included legislation identifying family violence as an offense and implementation of effective policies and procedures for tribal police and courts-mandatory arrest of perpetrators, use of male-female police teams responding to family violence complaints (Hillabrant, Brown, Colhoff, & Earp, 1994). In the WtW evaluation, a tribe had developed a model approach to child support enforcement (CSE) which involved close collaboration with states to identify noncustodial parents who were employed but were not providing support for their children and to garnishee the wages of persons who refused to comply with tribal court orders to provide child support (Hillabrant, Rhoades, Pindus, & Trutko, 2001). Longitudinal research and studies that involve data collection for six months or more can face additional problems including changes in tribal administration and changes in program staffing. Changes in tribal administration (e.g., new chairman, chief, governor) or newly elected tribal council members may view with suspicion research approved by a prior administration. Soliciting and obtaining approval of proposed research from a wide range of stakeholders can circumvent or minimize the effects of changes in administration. Effects of changes in program staffing on a study can range from minimal to catastrophic, depending on the nature of the research. Frequently, new staff must be trained to use or interface with the study protocol. Ethical Problems and Issues In general, research in Indian Country may have neither more nor fewer ethical problems and dilemmas than research conducted elsewhere. Still, issues such as cultural competence, relatively high rates of poverty and illness, and associated deficits in infrastructure (e.g., public transportation, plants and facilities, telecommunications) can exacerbate ethical problems and dilemmas. University graduate training programs teaching research ethics may cover federal regulations and procedures associated with IRBs and OMB as well as regulations governing the care of animals in research (Health Research Extension Act of 1985); however, it is difficult for such training to focus on ethical issues and problems associated with persons who might come into contact with the research but who are not participants. Also, it is difficult for ethics training to address problems where the researcher's actions must balance immediate costs against potential greater costs. This paper concludes with discussion of such a problem. Already discussed in Vignette 3 is the allegation of on-the-job substance abuse by a treatment program director. Another example involves the evaluation of a Head Start program producing exceptional benefits that was housed in a building that represented unacceptable risks to the children it served (see Vignette 4). The evaluators were confronted with the choice of reporting the violations, likely to result in the suspension of the program, or passively acquiescing in the continued exposure of the children to the risks associated with an unsafe building. Making judgments about ethics and values can be challenging to researchers working in Indian Country because of potentially conflicting roles and circumstances, such as: • If the researcher is not a member of the tribe participating in the study, he or she may be unfamiliar with critical values and expectations pertaining to ethical issues. • The sponsor of the research may have agendas, rules, and expectations that are different from or in conflict with those of the tribe(s) participating in the study. • The protection of study participant confidentiality may conflict with protection of study participants from abuse, family violence, or other threats. In such circumstances, it is prudent for the researcher to seek guidance from the project advisory committee, the research sponsor, and/or legal authorities without disclosing information that would violate the identity of the research participant(s) or violate the confidentiality of participant data. Vignette 4. Ethical problem: Closing an excellent Head Start facility. The national Head Start program conducts rolling compliance reviews of local Head Start facilities to ensure that the programs protect the health and welfare of the children served. One of the program requirements is that no Head Start facility be located in a building with major building code violations. Conducting site visits at several facilities on a reservation, researchers found, with one exception, that each facility met or exceeded all Head Start program requirements. The children attending these programs were clearly flourishing, and entire families were benefiting from the program. One facility, however, was located on the second story of a wooden building that had multiple building code violations, including insufficient numbers of fire extinguishers and smoke detectors. Program staff were aware of the code violations; however, there were few alternate facilities available on the reservation that could be used by the Head Start program, and tribal officials had not responded to requests for better facilities. The researchers did not want to jeopardize the operation of the clearly successful program. Options discussed included ignoring the code violations, urging program staff to continue their efforts to secure an alternate facility, and reporting the infractions to Head Start headquarters in Washington. After a brief discussion, the study team concluded that failure to act, in this case, would subject the children participating in the program to unacceptable risks. Consultation with Head Start headquarters resulted in an order that the tribe immediately suspend operation of the program at the dangerous facility. This suspension stimulated tribal officials to action. Within a week, the tribe was able to make available a building that complied with Head Start regulations and the program resumed operations. References American Indian Policy Review Commission. (1997). Final report. Washington, DC: U.S. Government Printing Office. Cherokee Nation v. Georgia, 30 U.S. 1 (1831). Fetterman, D.M. (2000). Foundations of empowerment evaluation. Thousand Oaks, CA: SAGE. Government Performance and Results Act of 1993, 67 Fed. Reg. 30772 (May 8, 2002) (to be codified at 12 CFR Parts 707, 792). Health Research Extension Act of 1985, Pub. L. No. 99-158. 99 Stat. 820 (2001). Hillabrant, W., Brown, A., Colhoff, B., & Earp, J. (1994) A case study of family violence in four Native American communities. Rockville, MD: Indian Health Service. Hillabrant, W., Brown, A., Weller, C., & Earp, J. (1988). Evaluation of the Indian Health Service fiscal intermediary (FI). Rockville, MD: Indian Health Service. Hillabrant, W., Earp, J., & Brutus, C. (1996). Impact of the final rule: Health care services of the Indian Health Service, 42 CFR Part 36. Rockville, MD: Indian Health Service. Hillabrant, W., Earp, J., & Brutus, C. (1997). Evaluation of the Indian Health Service adolescent alcoholism treatment centers. Rockville, MD: Indian Health Service. Hillabrant, W., Earp, J., & Rhoades, M. (2001) Evaluation of IHS-sponsored substance abuse treatment programs for American Indian and Alaska Native women. Rockville, MD: Indian Health Service. Hillabrant, W., Rhoades, M., Pindus, N., & Trutko, J. (2001). The evaluation of the Tribal Welfare-to-Work Grants Program: Initial implementation findings. Washington, DC: Office of the Assistant Secretary for Planning and Evaluation, Department of Health and Human Services. Hillabrant, W., Romano, M., & Stang, D. (1992) Native education at the turning point. In P. Cahape & Craig Howley (Eds.), Indian nations at risk: Listening to the people. Charleston, WV: ERIC Document Reproduction Service (No. ED343756). Indian Citizenship Act of 1924 8 (U.S.C. §1401) (1924). Indian Self-Determination and Education Assistance Act of 1975, Pub. L. No. 93-638, 25 USCS §§ 450. 88 Stat. 2203 et seq. (2002). Lemonick, M., & Goldstein, A. (2002, April 22). At your own risk. Time Magazine, 46-51. Orne, M.T. (1962). On the social psychology of the psychological experiment with particular reference to demand characteristics and their implications. American Psychologist, 17, 776-783. Paperwork Reduction Act of 1995, 44 U.S.C. 35(1995)..In the year 2000, American Indians and Alaska Natives (AI/AN) Cultural Competence Approaches to Evaluation in Tribal Communities Paulette Running Wolf, Ph.D. Robin Soler, Ph.D. Brigitte Manteuffel, Ph.D. Diane Sondheimer, M.S., M.P.H. Rolando L. Santiago, Ph.D. Jill Shepard Erickson, M.S.W., A.C.S.W. Abstract Disability research and evaluation in Indian Country are rare and have generally been viewed with suspicion by both rural reservation-based and urban tribal communities because designs, procedures, instruments, and interpretation and dissemination of outcomes have often been developed or selected without regard for potential cultural conflicts. This paper explores the implementation of a national evaluation in eight tribal communities that obtained grants to establish systems of care for children with serious emotional disturbances and their families. The congressional mandate to participate in the national evaluation has resulted in tribal communities taking a step into the world of research and evaluation. Some of the challenges and successes these tribal communities have experienced through their participation in the national evaluation are explored. In the year 2000, American Indians and Alaska Natives (AI/AN)1 represented nine-tenths of 1% of the general U.S. population (U.S. Census Bureau, 2002); however, children and adolescents (under 15 years) in this population represent a greater proportion of the total population (33%) as compared to 22% for the general U.S. population (Hodge & Fredericks, 1999). It is important to note that AI/AN children and adolescents have been identified as being at a higher risk for mental disorders, depression, substance abuse, dropping out of school, delinquency, suicide, and homicide (in particular through vehicular accidents) than other ethnic minority groups and the general U.S. population (Nelson & Manson, 2000). It has also been suggested in the literature that these AI/AN incidence rates will increase with age at an accelerated rate as compared to all other racial or ethnic groups in the United States (Barlow & Walkup, 1998; Dion, Gotowiec, & Beiser, 1998; Goodluck & Willeto, 2000; Middlebrook, LeMaster, Beals, Novins, & Manson, 2001). It is clear that AI/AN children and families are in need of social and mental health services; however, determining the actual nature of a behavioral, emotional, or developmental disability is fraught with challenges rooted in the sociological, cultural, and political histories of AI/AN people themselves and their experiences with psychological and evaluation research. 1 The term American Indians/Alaska Natives (AI/AN) is not meant to homogenize individuals from distinctly different cultures into a single ethnic group; rather, it is the current term used to represent more than 500 federally recognized tribes, including approximately 2.4 million American Indians and Alaska Natives. Each of these sovereign nations has distinct languages, religious beliefs, values, and cultural, social, and political histories. With apologies, the terms AI/AN and AI will be utilized throughout this manuscript as an attempt at brevity To further exacerbate the challenges evaluation researchers face, services and research in Indian Country (urban and reservation tribal communities) have been noted throughout the literature as being based in western psychological theory, which is contrary to the value structures and worldviews of these tribal communities. Issues such as tribal sovereignty, tribal government permission to conduct research, data ownership, isolation, cultural barriers, and methodological and dissemination issues (Ericksen, 1996; Manson, 1997; Mihesuah, 1993; Stubben, 2001; Trimble, 1977) continue to plague service delivery and research and evaluation efforts with this population. Today we continue to ask questions such as: "What services?" "Where should services be provided?" "How should service be provided?" "Who should provide services?" "Are services culturally appropriate?" "Are services effective?" "Who are the children and families being served?" "What are children and families experiencing?" and "How do children and families change across time?" This paper will provide a brief overview of psychological and evaluation research in Indian Country, describe service programs, and offer examples of how current tribal community recipients of a services and evaluation grant program, the Comprehensive Community Mental Health for Children and Their Families Program, are dealing with the challenges of conducting evaluation research in their communities. Using descriptive data collected through the national evaluation of this grant program, it will discuss how these and other findings may be influenced by historical, cultural, and other factors. Only a few tribal or regionally specific community studies (Beals et al., 1997; Cross, 1986; Novins, Duclos, Martin, Jewett, & Manson, 1999) and a handful of larger scale studies have been completed with the AI/AN child and adolescent populations (Beiser & Attneave, 1982; Cummings, Ireland, Resnick, & Blum, 1999; Dion, Gotowiec, & Beiser, 1998; Mitchell & O'Nell, 1998; Roy, Chaudhuri, & Irvine, 1970; Sampath, 1974; Shore, Kinzie, Thompson, & Pattison, 1973), raising at least as many questions as they attempted to answer. The largest and most recent study, which included 13,454 AI/AN children, concluded that "the connection to family remains a consistently powerful factor in the lives of these youth" (Cummings et al., 1999, p. 38). We will see this theme recurring throughout our discussion, revealing some of the many and significant meanings of family relationships in Indian Country and their implications for services, research, and evaluation. The few smaller regional or tribal-specific studies often cited in the literature did result in raising an awareness of the "handicapping" and assessment issues with the AI/AN child and adolescent population. For example, early on, researchers found that as many as 75% of AI/AN children living in boarding schools have experienced school-related social or emotional problems (Dlugokinski & Kramers, 1974; Kleinfeld & Bloom, 1977), and Ramirez and Smith (1978) noted that as many as 38% of American Indian children in Bureau of Indian Affairs (BIA) (non-residential) schools were handicapped. Dion, Gotowiec, and Beiser (1998) found that both non-Native children and their parents rated themselves higher on depression than did AI children and their parents. However, the teachers of AI children had a tendency to assign these children higher depression and conduct disorder ratings than non-Native children. Fisher, Bacon, and Storck (1998) attempted to address the methodological issues in urban-rural comparisons by examining teacher ratings in these settings, finding that although American Indian youth have higher levels of internalizing and externalizing behaviors (e.g., depression and conduct disorder) than Caucasian youth in the same rural community group, differences are much less general and pronounced than previous research suggested. Additionally, the disproportionate diagnosis of psychosis, mental retardation, and learning disabilities among AI/AN children, both in the hospital and public school systems, has long been noted in the literature (Fritz, 1976; LaFromboise & Plake, 1983; O'Nell, 1989; Roy et al., 1970). Though the research cited above certainly provides information that may be useful to many psychologists and service providers, the concept of "disability" has little meaning in AI/AN communities where there is typically no distinction among cognitive, emotional, physical (developmental delays) and spiritual concerns or illness (Adair, Deuschle, & Barnett, 1988; Barlow & Walkup,1998). Furthermore, the concept of illness (cognitive, emotional, physical, and spiritual) is often grounded in a relational worldview that is cyclical in nature and identifies the individual with an illness or disability as being "special" (Cross, 1986, p. 11) or as having been gifted with special abilities or personality characteristics. Therefore, words such as "disabled" and "handicapped" impose a worldview that conflicts with that of many AI/ANs. Those children and adolescents who experience serious emotional or behavioral problems, or both, are often considered to be passing through a developmental stage, and rather than intervene, parents and families tend to assume a "wait and see" approach. In order for program evaluation research to be effective and accurate, it must address underlying issues of the definition of, and expectations for, behavior and change among program participants. Systems of Care in Tribal Communities A first step toward identifying challenges that AI/AN children and families face, and toward identifying strengths and weaknesses of service systems available to them, is to understand clearly and comprehensively the roots of the key system from which these children emerge: the family system. Red Horse, Lewis, Feit, and Decker (1978) provide remarkable insight into the extended family system, identifying three primary differences between AI/AN families and White European or Caucasian families. The first difference they note is in the definition of extended family. The White European or Caucasian definition identifies the extended family as three generations living in the same household, whereas in AI/AN cultures it is defined as a village-type network construct which has a significant impact on behavior and socialization processes. Secondly, in AI/AN tribal communities this extended family structure transmits culture and conserves family patterns, which in turn contributes to identity development. Finally, according to Red Horse et al. (1978), the family promotes accountability in that it sets standards and expectations which then maintain the wholeness of the group through the enforcement of values. It is interesting to realize that we have come around full circle: from the establishment of reservations and the imposition of a nuclear family model which was used as an instrument to "civilize" tribal people and assimilate tribal culture, to modeling community mental health services after AI/AN systems of care (Medicine, 1981, p.14). Unfortunately, until recently, re-awakening and operationalizing tribal systems of care in Indian Country has lacked organizational, political, and financial support. Instead, as in years past, these systems operate quietly underground and often go unrecognized or untapped as resources. Scattered throughout the literature are "gems of wisdom" (Cross, 1986; LaFromboise, 1988; Medicine, 1981; Red Horse, 1980; Swinomish Tribal Mental Health Project, 1991; Trimble, Manson, Dinges, & Medicine, 1984), each of which has offered "road markers" for service development and delivery to AI/AN populations. Respected American Indian leaders in the field direct us to turn to the extended family unit to answer questions about service development. Bea Medicine (1981) summarizes the discussion of future directions in the field with, "We should be discussing the underlying orientations, beliefs, and kinship systems of a variety of Indian families" (p. 13). This suggests that we need to pave our services highway with those family-based cultural values, beliefs, and kinship systems that are specific to the tribal community and to those families who may depart from the "norm" in their community. The concepts of systems of care and the "wraparound process" (interagency services addressing emotional, physical, mental, and spiritual needs of the child and family, from therapy to respite care), were initially introduced by Stroul and Friedman (1986); however, as Debbie Painte (in Kendziora, Bruns, Osher, Pacchiano, & Mejia, 2001) noted, the wraparound concept was not a new concept in Indian Country but "is a re-visiting of our former village and clan and tribal structures" (p. 31). She further explained how the wraparound process represents a return to traditional ways. "This whole cultural erosion that we've had really has led to some of the . . . challenges that we face. We needed to find a way to rebuild those structures that we had for our families before. When we heard about Wraparound, it clicked. This is how we bring those interventions back. Those cultural ways that we had. This is the validation of our culture" (p. 31). Most, if not all, AI/AN individuals, families, and communities would concur. Federal funding began in 1993 for grant communities to develop "systems of care" for service delivery to children and adolescents with serious emotional disturbance and their families through the Comprehensive Community Mental Health Services for Children and Their Families Program. This program, supported by the Center for Mental Health Services (CMHS) in the Substance Abuse and Mental Health Services Administration (SAMHSA), has a strong commitment to program evaluation and to building evaluation capacity in grant communities. This federal initiative has supported the development of systems of care in 43 states, including eight AI/AN communities. The initiative includes a man-dated national evaluation component that follows congressional efforts toward program accountability. The congressional mandate that federally funded programs must be evaluated, as well as the program's investment in building community capacity for evaluation, have resulted in grant-funded tribal communities taking a step forward into the world of research and evaluation. The following section provides a brief description of the tribal communities funded by CMHS to develop systems of care. It will be followed by a description of the national evaluation developed for CMHS-funded grant communities and will include a description of evaluation activities in these communities that include efforts to implement the national evaluation in tribal settings, as well as efforts to develop community-specific evaluations that address unique characteristics of tribal programs not captured by the national evaluation. Among the eight tribal communities receiving CMHS funding to develop and implement systems of care since 1993, two communities had previously participated in CMHS Circles of Care planning grants (Sault Ste. Marie and Oglala Lakota). Together, these two federal initiatives offered financial support that, for the first time, provided opportunities to "reawaken" historically and culturally based systems of care within these tribal communities. The first American Indian tribe to receive funding by CMHS was the Navajo Nation, in 1994, through its K'ι project. In subsequent funding cycles seven additional tribal programs were funded: in 1997 Kmihqitahasultipon (Passamaquoddy Tribe, Maine), and Sacred Child (Inter-tribal in North Dakota, South Dakota, and Montana), in 1998 Mno Bmaadzid Endaad (Sault Ste. Marie Tribe of Chippewa Indians and Bay Mill Anishnabe tribal communities, Michigan), and With Eagles' Wings (Northern Arapaho and Eastern Shoshone, Wyoming), and most recently, in 1999, Yuut Calilriit Ikaiyuquulluteng (Yukon-Kuskokwim Health Corporation, 58 villages, Alaska), AK-O-NES (United Indian Health Services, Humboldt and Del Norte Counties, California), and Nagi Kicopi (Oglala Sioux, South Dakota). In 2000, Terry Cross and his colleagues conducted a series of focus groups and presented a number of common themes inherent in the structures of five of these tribal programs. They included the use of 1) extended family; 2) traditional teachings; 3) culturally specific approaches; 4) cultural restoration (mentors, crafts, language); 5) methods that promote healing of Indian identity and self efficacy; 6) methods that build connections to community, culture, group, clan, extended family; 7) methods that are based in spiritual beliefs and support systems; 8) approaches incorporating elders or intergenerational approaches; 9) individual and family skill building for living in two cultures; 10) traditional helping values (e.g., 24-hour staff availability); 11) conventional and cultural methods to recognize and treat historic cultural, intergenerational, and personal trauma; 12) approaches that strengthen or heal the community; 13) incorporation of a respect for diversity within the tribe and within the service array; and finally, 13) conventional services (individual and family therapy, and health care services) (Cross, Earle, Echo-Hawk Solie & Manness, 2000, p. 48). The eight tribal grant communities have other similarities, such as their organizational structures. The systems of care in some of these communities are grounded within programs offered historically by their respective tribal, social, and health programs and sponsored by their tribal governments (Navajo, Passamaquoddy, and Oglala Lakota). The other five programs (Sacred Child, Wind River, Sault Ste. Marie/Bay Mills, Humboldt, and YKHC) have an intertribal structure based on shared resources and geographic location, and have had an intertribal approach to the development of services and sponsorship for funding. They may be co-utilizing health clinic services under a single health care provider, such as United Indian Health Service (for Humboldt and Del Norte counties in northern California) and the Yukon-Kuskokwim Health Corporation, or have joined together to access funding because they are geographically close to one another, as are the Sault Ste. Marie Tribe of Chippewa Indians and the Bay Mills Tribe of Anishnabe (Ojibway), both located in the northeastern end of the Upper Michigan Peninsula, and the Northern Arapaho and Eastern Shoshone Tribes that share the Wind River Reservation in Wyoming. For the most part, many of these tribal communities share similar challenges based on their geographic isolation, unavailable or unreliable transportation, severe weather, large geographic distances in their service delivery areas, and having access to few if any community-based mental health services within a radius of forty to a hundred miles of their communities. This is in contrast to some of the other tribal grant communities that are seemingly surrounded by the general population and therefore have limited (though possibly not culturally appropriate) services for their children and adolescents. These eight tribal and intertribal programs are fairly representative of the diversity in reservation and urban service delivery settings across the country in terms of their levels of self-identified acculturation/assimilation. The differences between and within these tribal grant communities are clearly evident in both the service delivery structures and the arrays of services they have established and are demonstrating. Of great interest is the development of culturally appropriate assessment instruments and the use of unique treatment methods that are major contributions to the field. For example, the Navajo tribal grant community's service delivery structure is based on "K'ι [which] means to have reverence for all things in the universe and to maintain balance and harmony by acknowledging and respecting clan and kinship" (Cross et al., p. 32). Accordingly, they developed the position of a "traditional behavioral management specialist" that is grounded in their traditional clan system while utilizing a Navajo approach to counseling and to coordinating other traditional treatments or ceremonies as needed and as identified by the healers and desired by the families themselves. In contrast, both the service delivery structure and the types of services provided in the Passamaquoddy tribal community are focused on providing multiple levels of interventions. Younger children, families, and the community itself are provided with opportunities (services) to reintegrate Passamaquoddy culture into their lives. This in effect works toward reestablishing tribal values and traditions, ultimately assuring the continued survival of tribal culture whi