What Can We Learn from Nutrition Impact Evaluations? Lessons from a Review of Interventions to Reduce Child Malnutrition in Developing Countries

Evaluation SummaryWhat Can We Learn from Nutrition Impact Evaluations?High levels of child malnutrition in developing countries contribute to mortality and have long-term consequences for children#xE2;#xAC;"s cognitive development and earnings as adults. Recent impact evaluations show that many different interventions have had an impact on children#xE2;#xAC;"s anthropometric outcomes (height, weight, and birth weight), but there is no simple answer to the question #xE2;#xAC;SWhat works?#xE2;#xAC; to address the problem. Similar interventions have widely different results in different settings, owing to differences in local context, the causes and severity of malnutrition, and the capacity for program implementation. Impact evaluations of programs supported by the Bank, which are generally large-scale, complex inter-ventions in low-capacity settings, show equally variable results. The findings confirm that it should not be assumed that an intervention found effective in a randomized medical setting will have the same effects when implemented under field conditions. There are many robust experimental and quasi-experimental methods for assessing impact under difficult circumstances often found in field settings.The relevance and impact of nutrition impact evaluations could be enhanced by collecting data on service delivery, demand-side behavioral outcomes, and implementation processes to better understand the causal chain and what part of the chain is weak, in parallel with impact evaluations. It is also important to understand better the distribution of impacts, particularly among the poor, and to document better the costs and effectiveness of interventions. High levels of child malnutrition in developing countries are contributing to mortality and present long-term consequences for the survivors. An estimated 178 million children under age 5 in developing countries are stunted (low height for age) and 55 million are wasted (low weight for height). Malnutrition makes children more susceptible to illness and strongly affects child mortality. Beyond the mortality risk in the short run, the developmental delays caused by undernutrition affect children#xE2;#xAC;"s cognitive outcomes and productive potential as adults. Micronutrient deficiencies#xE2;#xAC;#x1D;vitamin A, iron, zinc, iodine, for example#xE2;#xAC;#x1D;are also common and have significant consequences. Progress in reducing malnutrition has been slow: More than half of countries are not on track to achieve the Millennium Development Goal of halving the share of children who are malnou-rished (low weight for age) by 2015. The food price and financial crises are making achievement of this goal even more elusive. The World Bank has recently taken steps to ex-pand its support for nutrition in response to the underlying need and the increased urgency due to the crises. WHAT DO WE KNOW ABOUT REDUCING MALNUTRITION? The increased interest and resources focused on the problem of high and potentially increasing rates of undernutrition raises the question, what do we know about the causes of malnutrition and the in-terventions most likely to reduce it? The medical literature points to the need to inter-vene during gestation and the first two years of life to prevent child malnutrition and its consequences. It suggests that investments in interventions during this window of opportunity among children under 2 are likely to have the greatest benefits. Recently published meta-analyses of the impact evaluation literature point to several interventions found effective for reducing undernutrition in spe-cific settings. However, there are limitations to the generalizability of those reviews#xE2;#xAC;" findings, particularly in the context of large-scale government programs most likely to be supported by the World Bank. The reviews tend to disproportionately draw on the findings of smaller, controlled experiments; there are few examples of evaluations of large-scale programs, over which there is less control in implementation. In reviewing a large number of studies, interventions, and outcomes, they tend to focus on average impacts. They generally do not explain the magnitude or variability of impacts across or within studies. Very few address the programmatic reasons why some interventions work or don#xE2;#xAC;"t work, nor do they assess the cost-effectiveness of interventions. Objectives of the ReviewThis paper reviews recent impact evaluations of interventions and programs to improve child anth-ropometric outcomes#xE2;#xAC;#x1D;height, weight, and birth weight#xE2;#xAC;#x1D;with an emphasis on both the findings and limitations of the literature and on understanding what might happen in a non-research setting. It further reviews in greater detail the experience and lessons from evaluations of the impact of World Bank-supported programs on nutrition outcomes. Specifically, the review addresses four questions. First, what can be said about the impact of different interventions on children#xE2;#xAC;"s anthropometric outcomes? Second, how do these findings vary across settings and within target groups, and what accounts for this variability? Third, what is the evidence of the cost-effectiveness of these interventions? Finally, what have been the lessons from implementing impact evaluations of Bank-supported programs with anthropometric impacts? While there are different dimensions of child nutri-tion that could be explored, the report focuses on child anthropometric outcomes -- weight, height, and birth weight. These are the most common nutrition outcome indicators in the literature and the most frequently monitored by national nutrition programs supported by the World Bank. Low weight for age (underweight) is also the indicator for one of the MDGs.Methodology and Scope Forty-six nutrition impact evaluations published since 2000 were systematically reviewed. These evaluations assessed the impact of diverse interven-tions#xE2;#xAC;#x1D;community nutrition programs, conditional and unconditional cash transfers, early child devel-opment programs, food aid, integrated health and nutrition services, and de-worming. All of the evaluations used research designs that compared the outcomes among those affected by the project to the counterfactual#xE2;#xAC;#x1D;that is, what would have happened to a similar group of people in the absence of the intervention. About half used randomized assignment to create treatment and control groups, while the remainder used matching and various econometric techniques to construct a counterfactual. Among the 46 evaluations, twelve assessed the im-pact of World Bank-supported programs on nutri-tion outcomes in eight countries. While the broader review relies on the analysis of the published impact evaluations as the main source of data, for these twelve evaluations project documents and research outputs were reviewed and World Bank staff, country officials and the evaluators and re-searchers who conducted the studies were interviewed.FindingsA wide range of interventions had a positive impact on indicators related to height, weight, wasting, and low birth weight.There were a total of 10 different outcome indica-tors for the four main anthropometric outcomes. A little more than half of the evaluations addressing a height-related indicator found program impacts on at least one group of children, and this was true for about the same share of interventions aimed at improving weight-related and wasting (low weight for height)-related indicators. About three-quarters of the 11 evaluations of interventions that aimed at improving birth weight indicators registered an impact in at least one specification, including five out of seven micronutrient interven-tions. There was no clear pattern of impacts across interventions#xE2;#xAC;#x1D;in every intervention group there were examples of programs that did and did not have an impact on a given indicator, and with varying magnitude. Evaluations of the nut