A prize for evidence-based policy | Analysis
It is a rare and joyous moment when three of your most influential advisers and mentors win a Nobel Prize together – and today is that day! Abhijit Banerjee, Esther Duflo, and Michael Kremer have just received the Nobel Memorial Prize in economics for their pioneering work in the use of randomized field experiments to study the effectiveness of policies and programs at improving human well-being.
Abhijit Banerjee is an extraordinarily broad and deep development economist, with seminal contributions in several areas. Abhijit’s special genius is in how he has reinvented himself multiple times as a scholar to continue producing top-quality new research at a pace that would put those half his age to shame. In the eighties and early nineties, when data was limited, the most insightful ways to think about development were theoretical and his earliest classics were pure theory papers. These covered topics such as how people learn new facts and ideas, and the relationship between wealth distribution, occupational choice, and development. With the onset of personal computers and greater access to better quality of data in the mid-nineties, his work evolved to combining elegant applied theory with data to study fundamental questions in development economics such as the impact of land reforms on agricultural productivity. Finally, with the increasing use of randomized experiments in development economics (more below), he has written highly-influential empirical papers that study the impacts of specific policies to improve development outcomes including education, health, and reduction in poverty.
Abhijit’s transition to more empirical work and randomized experiments in particular, was in turn influenced by his association with Michael Kremer, when they were both on the faculty at MIT in the mid-nineties. Similar to Abhijit, Michael’s early classics and professional recognition came from his theoretical work. However, Michael had also spent three years before his Ph.D. teaching in rural schools in Kenya and had formed an early appreciation for how little data and evidence there was to test various programs and policies that claimed to improve the lives of the poor. When he won a MacArthur “genius” award, Michael used these funds to start running randomized field experiments in Kenya to test the effectiveness of specific policies to improve education and health outcomes.
The key idea behind a randomized control trial (or RCT) is that a program is first provided to some participants chosen by a random lottery (a “treatment” group) while others get the program later and serve as a “control” group. Since these groups are identical on average and treatment is assigned by lottery, comparing outcomes over time allows researchers to credibly study the impact of specific policies. While RCTs have their limitations, they are often considered the “gold standard” of evidence, and are the standard method used in testing and approving new medicines around the world. RCTs were not a new idea, but Michael’s main contribution was to systematically start using them to evaluate development interventions. As he once mentioned to me, “a new medication goes through the rigour of an RCT even when it only affects a few hundred thousand people; so it’s crazy that we spend billions of dollars on policies and programs affecting hundreds of millions of people with nothing close to the same level of evidence on effectiveness or lack thereof”.
Finally, the person perhaps most responsible for the RCT revolution in development economics is Esther Duflo – the youngest ever winner of the Nobel Prize in Economics. A force of nature, Esther was a student of Abhijit’s and Michael’s at MIT and co-founded the Jameel Poverty Action Lab (or JPAL). Under her leadership, JPAL has, in just fifteen years, transformed the field of development economics from being mainly theoretical to predominantly empirical and informed by high-quality evidence on the causal impact of programs and policies. JPAL now has over 180 affiliated researchers, and has nearly a 1000 completed and ongoing RCTs in sectors ranging from education, health, credit, savings, and governance. This research has in turn influenced the scaling of policies affecting over a hundred million people.
The Nobel prize is a recognition of not just the scholarship of these three pioneering researchers, but also of their broader impact. These include institution building, public communication of research findings, training of an entire generation of younger researchers and policy practitioners, and the impact that the JPAL way of research has had on the economics profession at large.
The value of this approach is well illustrated by JPAL research on education, where many “obvious” programs often have no impact and other less obvious ones can have very large effects. For instance, RCT-based evidence has shown that providing students with free textbooks may have no impact on learning because the students are not able to read. In contrast, a school deworming program can have large positive impacts on school participation and long-term outcomes at a very low cost. Similarly, adding an extra teacher to a school may not have nearly as much impact on learning as matching instruction to the learning levels of children.
One reason for the failure of well-intentioned policies is that they are often designed by elites who do not face the same constraints as the populations the policies are meant to serve. Thus, a policymaker may be full of good intentions in providing free textbooks, but may not have even considered the possibility that the majority of students are so far behind grade-level standards that a textbook does not alleviate the binding constraint to learning, which is the mismatch between the level of instruction and level of student learning. Similarly, while adding a teacher may reduce class sizes, it may be of limited use if the teacher focuses on completing the textbook and syllabus while student learning levels are far behind. In contrast, supplemental instruction by modestly paid volunteers with no formal training, may be much more effective because it targeted instruction at the learning level of the student. In a related vein, school-health programs often fall through the cracks due to lack of coordination between education and health departments, but may have much higher returns in terms of outcomes. The scaling up of supplemental instruction that aims to “Teach at the Right Level” and of school deworming programs around the world can be directly traced to this body of evidence.
Thus, while the individual studies may seem like they tackle “small” questions, their value is in disciplining theory and ideas with data and evidence. Taken together, the portfolio of high-quality evidence built by JPAL over the years in various sectors provides a rich understanding of complex sectors. In particular, this body of evidence shows that there is large variation in the impact and cost-effectiveness of policies and thus, shifting public (and donor) spending from less to more cost-effective programs and policies can significantly improve outcomes within a given budget.
The broader message for India (and the world) in this prize is that solving complex development challenges requires careful attention to data and evidence. Companies and products face a “market test” where they receive rapid feedback on whether a product is working or not. In contrast, it is possible for governments to spend taxpayer money badly for a long time without facing the consequences. This magnifies the importance of evaluation of major government programs, and improving the quality of public expenditure. The economic slowdown is a good occasion to appreciate and institutionalize a more systematic focus on data, evidence, and value for money in policy.
Karthik Muralidharan is the Tata Chancellor’s Professor of Economics at UC San Diego and a Board member and Education Program Chair at JPAL