Results of rigorous evaluations show that one of the most cost-effective ways to boost primary school attendance in western Kenya is to treat children for intestinal worms, a health-related intervention.1
Findings like these earned economists Esther Duflo and Abhijit Banerjee of the Massachusetts Institute of Technology (MIT), and Michael Kremer of Harvard University the 2019 Nobel Prize in economics. As founder and co-director of MIT’s Abdul Latif Jameel Poverty Action Lab (J-PAL), Duflo was principal investigator on a number of projects supported by the Eunice Kennedy Shriver National Institutes of Child Health and Human Development (NICHD).
Evaluating Social Interventions Like We Test New Drugs
Rigorous evaluation of anti-poverty programs is at the heart of Duflo and J-PAL’s work, supported by NICHD. To assess the impact of social programs, they set up treatment and nontreatment (control) groups from the outset, “the way we test drugs,” Duflo explained.
In Kenya, government and nonprofit organizations tried a number of initiatives to remove the obstacles that kept children from attending school—including cutting school fees, distributing uniforms, and serving free school meals, she said.
When researchers used carefully controlled field trials to examine ways of increasing school attendance, they found that school-based de-worming in Kenya (requiring two treatments totaling $1 per child per year) had the biggest impact per dollar spent.
While intestinal parasites are not life threatening, they leave children anemic, lethargic, and listless; the researchers found that sick children miss school frequently.
The de-worming program was rolled out to schools in the western region over three years. The Kenyan schools that received the de-worming program the first year were chosen randomly from an alphabetized list.
The incremental rollout and random way they chose which schools got the program first facilitated a rigorous evaluation of the de-worming program. This design allowed researchers to estimate the program’s impact by comparing attendance among children in schools with the de-worming program and those without it.
Collaboration Between Researchers and Local Anti-Poverty Groups Crucial
These types of evaluations require careful coordination among researchers and those implementing the anti-poverty interventions. But the results can build support for effective programs and better target the limited resources available to fight poverty, Duflo argued.
“There are many ideas floating around, but the evidence is spotty on the most effective ways to have a social impact,” she said. Based on J-PAL’s evaluation results, the de-worming programs were expanded in Kenya and initiated in parts of Uganda and India.
But randomized evaluations are not perfect; J-PAL staff reports have pointed out that this approach is not appropriate for evaluating every type of social intervention, and the results are context-specific. For example, school-based de-worming in India had much less impact on school attendance than it did in Kenya. In settings where intestinal parasites are less widespread, such as parts of Latin America, other interventions are more appropriate.
Evaluation Shows Improved Household Stoves Performed Better in the Lab Than in Homes
Evaluation of a project promoting improved household stoves underscores the strengths of this type of evaluation.2 Tested in a laboratory, these stoves cut indoor air pollution, improved health, required less fuel, and reduced greenhouse gas emissions. But researchers found different results when they examined the stoves in a community setting.
The field trial worked this way: Households in cities in India competed in a lottery for both the improved stoves and installation help from a well-respected nongovernmental organization. Duflo and her colleagues tracked the stoves’ impact for four years, examining households with and without the new stoves. They found some evidence of lower levels of smoke inhalation among household members with the new stoves in the first year, but “no effect over longer time horizons.” Lung function and health did not improve, nor did fuel consumption decline.
“Households failed to use the stoves regularly or appropriately, did not make the necessary investments to maintain them properly, and use ultimately declined further over time,” they reported.
Writing in National Geographic NewsWatch, Duflo and her colleagues argued that their “one discouraging study should not set the world back in the effort to eradicate [indoor air pollution].” On the contrary, they wrote, “it should renew our enthusiasm to search, with open minds, for appropriate solutions.”
In Duflo’s view, this sort of rigorous evaluation in a real-world setting often challenges conventional wisdom but ensures that “policy decisions are based on scientific evidence.” The lessons learned from research designed in this way can “inform future programs, increase the effectiveness of existing programs, and mobilize additional resources,” she said.
This article is an update of a piece originally published by PRB in 2013.
References
- Michael Kremer and Edward Miguel, “Primary School Deworming in Kenya,” Abdul Latif Jameel Poverty Action Lab, https://www.povertyactionlab.org/evaluation/primary-school-deworming-kenya.
- Rema Hanna, Esther Duflo, and Michael Greenstone “Up in Smoke: The Influence of Household Behavior on the Long-Run Impact of Improved Cooking Stoves,” MIT Department of Economics Working Paper No. 12-10 (2012).