TEI

TEI Publishes Tribal Home Visiting Grantee Evaluation Plan Profiles on OPRE Website

The Tribal Evaluation Institute (TEI) created a compilation of Tribal Home Visiting Program evaluation plan profiles that introduce and describe the grantees’ evaluation studies. The profiles are designed for evaluators, program implementers, and federal staff interested in program impact in complex community contexts and may be most useful for individuals thinking about evaluating tribal home visiting and/or early education initiatives. They can be used as resources for the development and implementation of future evaluations in tribal communities, as well as to inform evaluation-related policies and grant requirements for tribal funding recipients. Each grantee profile presents a description of the program, a summary of their evaluation plan, and information about their evaluation team.

You can view the published report on the OPRE website.


TEI Contributes to Special Issue of the Infant Mental Health Journal

Staff from the Tribal Evaluation Institute (TEI) project worked closely with Tribal Home Visiting grantees, ACF, and the Tribal Early Childhood Research Center (TRC) to contribute to five articles in the May/June 2018 volume of the Infant Mental Health Journal titled, “New Directions in Infant and Early Childhood Home Visiting: Lessons Learned from the Tribal Maternal, Infant, and Early Childhood Home Visiting Program.” The published articles include:

  • Building evaluation partnerships with tribal communities for home visiting (Ayoub, et al.)
  • Methodological considerations for home-visiting research in tribal communities (Kilburn, et al.)
  • Performance measurement in Tribal Home Visiting: Challenges and opportunities (Morales, et al.)
  • Measurement issues in home visiting research within tribal communities: Challenges and strategies (Rumbaugh Whitesell, et al.)
  • Approaches to the evaluation of cultural adaptations of home visiting in tribal communities (Meyer, et al.)

This journal issue represents the first time tribal home visiting efforts were the focus of a special issue.


Grantee Led Evaluations in Tribal MIECHV

Grantee-Led Evaluations in the Tribal Maternal, Infant, and Early Childhood Home Visiting Program:

A Compilation of Grantee Evaluation Plan Profiles

In 2018, TEI created a compilation of Tribal Home Visiting Program grantees’ evaluation plan profiles that introduce and describe the grantees evaluation studies. The profiles are designed for evaluators, program implementers, and federal staff who are looking to assess program impact in complex community contexts and may be most useful for individuals thinking about evaluating tribal home visiting and/or early education initiatives. They can be used as resources for the development and implementation of future evaluations in tribal communities, as well as to inform evaluation-related policies and grant requirements for tribal funding recipients. Each profile presents a description of the program, a summary of their evaluation plan, and information about their evaluation team. Grantees reviewed, provided feedback, and approved their Evaluation Plan Profiles. Grantees also received local approval for the dissemination of these profiles, as desired by their community.

Click here to access and download the compilation.


CITATION:

Morales, J., Roberts, E., Salvador, M., Lyon, K., Geary, E., & Buckless, B. (2018). Grantee-led evaluations in the Tribal Maternal, Infant, and Early Childhood Home Visiting Program: A compilation of grantee evaluation plan profiles, OPRE Report # 2018-107, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.


Evaluation Capacity Readiness Assessment Checklist

Tribal MIECHV Evaluation Capacity Readiness Assessment Checklist (2017)

The Evaluation Capacity Readiness Assessment Checklist (ECR) was developed to help Tribal MIECHV grantees prepare to evaluate their home visiting services. The ECR is a self-assessment tool focused on examining organizational and team readiness to engage in evaluation activities. The assessment process can help grantees determine the areas of their program and organization that may need to be strengthened to be fully prepared to plan, launch, and implement an evaluation. The ECR should be conducted before evaluation questions are developed, as the results can inform what is feasible and appropriate for the program and organization. While the ECR was developed for Tribal MIECHV grantees, it is applicable to any program seeking to evaluate their services and activities.

Print Friendly, PDF & Email

Download the ECR Tool

ECR Tool in PDF format

The purpose of the toolkit, intended audiences and how to use it. Download ECR Tool in PDF format (.docx, 377kb).

ECR Tool in Microsoft Word format

The purpose of the toolkit, intended audiences and how to use it. Download ECR Tool in Microsoft Word format (.docx, 377kb).


Data System Improvement Toolkit - Module 5

Number 5

Module 5: Optimizing Your Current Data System

Strengthen your existing data system by learning to conduct assessments, identify enhancements, and organize your overall improvement project. Tools marked with an * below can be modified.

MODULE CONTENT

Module Introduction

This text shares two steps for customizing a data system and provides in-depth descriptions of tools found within the module

5.1 Guide to Data System Self-Assessment

This guide describes data system assessment, when it should take place, and what core components to examine. Read before accessing tool 5.2. (Guide)

5.2 Data System Assessment Checklist

Use this checklist to facilitate a conversation about your data system’s requirements, strengths, and areas for improvement. (Checklist/assessment)

5.3 Data System Improvement Charter

This sample charter describes objectives, goals, timelines, and benefits of a data system improvement project. (Template)


Data System Improvement Toolkit - Module 4

Number 4

Module 4: Displaying and Reporting Data

Access tools for creating and refining data dashboards to monitor services and prepare reports for funders. Tools marked with an * below can be modified.

MODULE CONTENT

Module Introduction

This text provides basic information about data dashboards and describes tools found within the module.

4.1 Data Reporting Map*

Use this tool to determine where data points are pulled from in existing reports and how calculations are made. Programs may find it useful to enlist addition support from a technical assistance provider or other consultant. (Template)

4.2 Understanding System Integration

This guide describes different types of system integration and provides an overview of the planning process. (Guide)

4.3: Guide to Data Dashboards

This guide serves as an introduction to tool 4.4. It lists common dashboard elements and considerations for customizing dashboard data. (Guide)

4.4 Example of a Data Dashboard

This sample dashboard presents a “real-world” example of concepts outlined in tool 4.3. (Example)


Data System Improvement Toolkit - Module 3

Number 3

Module 3: Protecting Data Ownership and Privacy

Get up to speed on managing and protecting data—and how to address these issues with systems vendors and developers. Tools marked with an * below can be modified.

MODULE CONTENT

Module Introduction

This text details two steps for owning and protecting client information and lists tools found within the module.

3.1 Understanding Data Ownership

This guide describes core elements of data ownership, including the use of ownership agreements and governance planning. Use with tool 3.2 (Guide)

3.2 Example of Data Ownership Contract Language*

Structured around the core elements outlined in tool 3.1, this document provides sample contract language for data ownership agreements. (Example)

3.3 Guide to Data Privacy and Confidentiality

Read this guide to better understand what data to protect, what regulations to comply with, and how to work with system vendors to ensure privacy. (Guide)

3.4 Considerations for Cloud Versus On-Premise Software and Storage

This tool explains the differences between cloud-based and on-the-premises storage and software options in areas such as cost, system function, ownership, and security. (Guide)


Data System Improvement Toolkit - Module 2

Number 2

Module 2: Documenting and Improving Data System Processes

Consistency is key when improving data systems and documenting related processes. Learn how to streamline data entry, quality, and storage. Tools marked with an * below can be modified.

MODULE CONTENT

Module Introduction

This text outlines three steps for strengthening data system processes and describes tools found within the module.

2.1 Guide to Data Mapping

This short guide includes a step-by-step overview of data mapping. Review it, along with tool 2.2, before working with a vendor to map your data system. (Guide)

2.2 Data Map Template*

Once you’ve read tool 2.1, use this customizable template to create a visual map of the data in your system. (Template)

2.3 Guide to Data Cleaning

This document provides a brief overview of data cleaning, including specific tasks for program staff, supervisors, and managers to complete before and during data reporting. (Guide)

2.4 Example of a Case File Checklist*

Supervisors can use this customizable checklist when reviewing case files to see if assessments were performed completely and on time. Use with tool 2.5. (Example and Template)

2.5 Example of a Missing Data Report*

This customizable template features a sample missing data report to help programs summarize results from their case file reviews. (Example and Template)

2.6 Data System Business Process Maps*

Process maps help users better document data collection and entry related to intake, service provision, and reporting. They can be customized with the help of a technical assistance provider. (Example and Template)

2.7 Best Practices When Converting to a Paperless Data System

This brief guide presents best practices and common challenges when moving from a paper-based data collection and entry system to a paperless system that uses laptops or tablets. (Guide)


Data System Improvement Toolkit - Module 1

Number 1

Module 1: Choosing a System and Working with a Vendor or Developer

Is it better to use a commercial off-the-shelf (COTS) product or develop a new, customized one? Make an informed decision and translate it into action. Tools marked with an * below can be modified.

MODULE CONTENT

Module Introduction

This text defines terms commonly used when selecting a data system and provides in-depth descriptions of tools found within the module.

1.1 Decision Guide: Is a COTS System Right for You?*

This table of questions can help you determine whether a COTS or custom data system would better meet your program requirements. (Guide and assessment)

1.2 Scan of COTS Systems*

This tool provides a summary of commonly used COTS products, including those used by tribal and state MIECHV grantees. (Summary)

1.3 Request for Proposals – Timeline Guide

This template identifies the major activities associated with developing a request for proposals. Visit tool 1.5 for information on vendor and developer activities. (Template)

1.4 Request for Proposals – Timeline Template*

Once you’ve identified your timeline using tool 1.3, enter your dates into this customizable template to create a personalized timeline chart. You can also add columns to note staff responsibilities, additional information, etc. (Template)

1.5 Request for Proposals – Template*

This template helps teams looking to hire a system vendor or developer. Complete the template by adding dates and information from tool 1.4. (Template)

1.6 Example of a Data System Software License Agreement

This tool walks you through the key elements of software user agreements, which often contain important, non-negotiable language about vendor expectations. (Example)


Data Systems Improvement Toolkit

Data System Improvement Toolkit

The Data System Improvement Toolkit is designed to help tribal program grantees develop or improve data systems. Program and evaluation staff who have experience with and oversee changes to their existing data system will find this toolkit informative. If you are interested in using the toolkit but do not have experience with data systems, please contact your technical assistance provider assigned through your federal grant.

How to Navigate the Toolkit

The Data System Improvement Toolkit contains five modules that walk users through the steps of strengthening their data system. Each module includes the following:

  • Introduction, including goals and terminology
  • Overview of the tools
  • Copies of the tools, including modifiable versions when applicable

Click on a module below to access its content. You can also download the full toolkit in PDF, including a more in-depth introduction.

How to Use the Toolkit

The five modules in this toolkit represent steps in a cycle. However, data system improvement is ongoing with no defined beginning or end. The following questions can help programs choose the right starting point for their needs:

  1. Does your system currently meet your requirements or do you need something new?
  2. What challenges most impact your ability to serve families?
  3. What improvements do you need to focus on most, in keeping with available time and resources?

Some programs will find that using one module or even one tool can address their needs. Others may find it helpful to access each module in the order they’re presented. For many, the right approach lies somewhere in between.


Data Systems Cycle graphic

MODULE OVERVIEW

Module 1: Choosing a System and Working with a Vendor or Developer

Is it better to use a commercial off-the-shelf (COTS) product or develop a new, customized one? Make an informed decision and translate it into action.

Module 2: Documenting and Improving Data System Processes

Consistency is key when improving data systems and documenting related processes. Learn how to streamline data entry, quality, and storage.

Module 3: Protecting Data Ownership and Privacy

Get up to speed on managing and protecting data—and how to address these issues with systems vendors and developers.

Module 4: Displaying and Reporting Data

Access tools for creating and refining data dashboards to monitor services and prepare reports for funders.

Module 5: Optimizing Your Current Data System

Strengthen your existing data system by learning to conduct assessments, identify enhancements, and organize your overall improvement project.