Association for Public Policy Analysis & Management

The Federal Home Visiting Program is one of the flagship evidence-based social policy initiatives of the Obama Administration. The extent of measurement required to trace the program’s implementation is virtually unprecedented. Multi-faceted efforts are in place to measure the program’s evolution to date for research and evaluation, for quality improvement and for performance accountability purposes. This presentation focuses on the evolution of the performance measurement system for home visiting.

First, we will discuss the measurement framework implemented at the inception of the program. Provisions in the statute that were notably prescriptive about how improvement of performance would be assessed over time informed the original design. The priorities and expertise HRSA and ACF brought to the table, as well as challenges these two agencies faced in sharing administration of the program also shaped the approach. Second, we will summarize the experience of collecting and analyzing annual performance reports submitted by grantees during the first three years of implementation. Finally, we will summarize the process of re-designing the performance measurement system based on the initial experience of local implementing agencies, grantees and federal staff. 

 Each grantee defined and operationalized measures associated with similar reportable concepts (such as maternal depression or family self-sufficiency) differently in this first version. Baseline and comparison periods to assess improvement for a given topic also varied across grantees. This made measurement complicated to administer and to review. This variation also limited the ability to compare performance across grantees and the production of a national picture of overall program performance. 

As such, HRSA and ACF are undertaking a re-design to simplify and standardize the performance measurement for home visiting grantees. Setting the measures and providing support to collect and analyze grantee reports from scratch and in short order was a significant accomplishment. But after the three-year assessment of improvement required by statute was completed, the two agencies had an opportunity to make performance measurement more robust and less burdensome for grantees and federal staff alike. Reducing the number of reportable measures and standardizing them across grantees were fundamental drivers towards this two-fold aim. Grantees and other stakeholders such as home visiting model developers were supportive of this aim.

 HRSA and ACF sought to reach agreement among stakeholders and experts on the specific features and metrics of such a re-designed performance measurement system. This effort started with a series of listening sessions with expert groups to specify a much shorter list of core indicators for the home visiting program. We were fortunate to build on the work of the Data for Performance Initiative the Pew Home Visiting Campaign had launched earlier with the participation of grantee representatives, other stakeholders and experts. The purpose of this initiative was to establish a core set of process and outcome measures for home visiting as a field.   The presentation will primarily focus on the joint public-private work towards developing consensus around a more effective and efficient performance measurement for home visiting and the lessons learned along the way.