Association for Public Policy Analysis & Management

Policy-making and management has successfully embraced the scientific standards using evidence over the recent years. Whilst evidence for policy making and evaluation includes a variety of methods not limited to quantitative analysis, systemic reviews from journals, program evaluation, policy monitoring, meta-analysis, narratives and observational studies; randomized control treatments and quasi- experiments have become the “gold standard” for social programs, including for public health interventions.  

Quantitative methods such as randomization and quasi experiments are beneficial in minimizing bias and evaluating large-scale health interventions, but they are not sufficient as standalone to adequately inform the public health decision making, irrespective of statistical significance or generalizability. This is because of multiple reasons. First, when health policies are implemented at a large scale, the impact and outcomes are but linear. They are mired with multiple and complex causal pathways and equifinality. Second, an intervention could be effective in a given situation but may be ineffective somewhere else, presenting a huge challenge to provide comprehensive recommendations and modifications. Third, there is divergence between what research shows is effective and the policies that are enacted and enforced. Fourth, they lack plausibility and adequacy evidence, which highlight confounding factors and provide additional insights about the association between the intervention and the outcome. Instead of a silo-approach, evidence based policies, especially on public health, must draw from not only quantitative studies but also complement statistical predictions with qualitative research revealing verifiable and credible evidence to influence policy deliberations, priorities, and possible solutions.

With the background of four cash conditional maternal healthcare policies in India, this paper explores the potential of using this multipronged approach to review these policies based on evidence from multiple sources. India accounts for 50,000 maternal deaths, highest in the world. Quantitative evidence is drawn from randomized District Health Survey III and systemic review of the policy literature on the maternal health policies to understand the policy effectiveness. Insights are also drawn from the meta- analysis of ethnographic and anthropological literature as well as a small-scale primary qualitative survey in a cluster of nine remote tribal villages in India to draw upon the policy processes, policy content, and policy outcomes. The findings from the three sources affirm the difference in insights and reinforces the duality in evidence-based methodologies. However, when taken in conjunction, each evidence enriches the other, explicating not only the scale and the strength in effectiveness of the policies, but highlighting the causal mechanisms that are the strongest, and exposing the conflicts and shortcomings in the policy design and implementation that should be reviewed for subsequent modification.