Scientifically and Culturally Rigorous Evaluation

Evaluations of tribal programs are strongest when they have both scientific and cultural rigor. Together, these types of rigor help to make sure results are valid, or accurate, for the research community and the community the program serves.[i] [ii] [iii]

Scientific rigor requires evaluations to use an appropriate evaluation design and systematic methods to answer evaluation questions. Cultural rigor requires evaluations to be inclusive of and responsive to local cultural practices. It attempts to ensure that information is gathered in appropriate and meaningful ways.[iii] For example, evaluators in a tribal community may get input from elders to develop the evaluation plan or use oral traditions, such as storytelling, to collect information. Evaluations without cultural rigor may fail to recognize and appreciate the strengths of the community and tribal program.[iv]

How TEI Supports Rigorous Local Evaluations

TEI builds the capacity of Tribal Home Visiting Program grantees to evaluate their programs in ways that are both scientifically and culturally rigorous.

Support for scientific rigor in tribal communities may include-

  • Translating research terms into everyday language so that program staff, advisory board members, and others who may not be familiar with research can provide input into the evaluation plan
  • Exploring evaluation designs that provide an alternative to random assignment, such as historical comparisons, naturally occurring comparison groups, and within-person comparisons
  • Providing training materials and resources for home visitors and other program staff to support high-quality data collection
  • Supporting the development of systematic data collection plans

Support for cultural rigor in tribal communities may include-

  • Using a community-engaged technical assistance process that encourages and allows time for gaining input from advisory councils, tribal leadership, staff, and community members
  • Encouraging grantees to develop evaluation questions that reflect the interests of their tribal organizations and communities
  • Honoring local cultural protocols and incorporating these into evaluation planning and methods
  • Exploring ways of evaluating cultural activities and measuring outcomes that are important to the community and local culture

RESOURCES

Read more about merging scientific and cultural rigor: A Roadmap for Collaborative and Effective Evaluation in Tribal Communities (PDF, 1.11 MB)

Learn more about how scientific rigor is defined by the Maternal, Infant, and Early Childhood Home Visting Program (MIECHV) program: Design Options for Home Visiting Evaluation: Evaluation Technical Assistance Brief (PDF, 267 KB)


FOOTNOTES

[i] Coryn, C. (2007). The holy trinity of methodological rigor: A skeptical view. Journal of MultiDisciplinary Evaluation, 4(7), 26–31.

[ii] Kirkhart, K. E. (2005). Through a cultural lens: Reflections on validity and theory in evaluation. In S. Hood, R. Hopson, & H. Frierson (Eds.). The role of culture and cultural context: A mandate for inclusion, the discovery of truth, and understanding in evaluative theory and practice (pp. 21–39). Greenwich, CT: Information Age Publishing.

[iii] Tribal Evaluation Workgroup. (2013). A roadmap for collaborative and effective evaluation in tribal communities. Washington, DC: Children’s Bureau, Administration for Children and Families, U.S. Department of Health and Human Services. Retrieved from https://www.acf.hhs.gov/sites/default/files/cb/tribal_roadmap.pdf

[iv] LaFrance, J., & Nichols, R. (2010). Reframing evaluation: Defining an indigenous evaluation framework. Canadian Journal of Program Evaluation, 23(2), 13–31.