Selecting the most effective nudge: Evidence from immunisation study
Immunisation is a highly cost-effective way of improving child survival. However, over two million children around the world die each year from vaccine-preventable diseases.
Despite large investments to increase access, full immunisation coverage for BCG, polio, DPT, and measles remains low in several countries, including India. As of 2016, only 62% of Indian children were fully immunised against these diseases as per the National Family Health Survey. A large fraction of children receive the first vaccine, but do not complete the full schedule, reflecting high initial motivation but difficulty completing later vaccination visits.
Previous research suggests that offering incentives, sending reminders, and appointing “immunisation ambassadors” in the community can build parental demand for vaccination and increase child immunisation. Yet these programmes have rarely been evaluated to examine impact at scale, compare different variants of each strategy, or test these policies in combination with one another.
To test how different policy combinations affect immunisation rates in an at-scale experiment, a team of researchers led by J-PAL founders Abhijit Banerjee and Esther Duflo randomised the provision of incentives, targeted reminders, and local immunisation ambassadors across seven districts (covering 2,360 villages) in India’s Haryana state. The study combined the experimental results with machine learning techniques to ascertain the most effective and cost-effective packages.
Haryana’s immunisation record
Despite large investments in vaccine delivery, immunisation rates are particularly low in some districts of Haryana. In the study population, about 86% of children aged 12-23 months received at least three vaccines at baseline. According to parents’ reports, however, only 39% were fully immunised and less than 20% had received the measles vaccine before 15 months. To complement the Universal Immunisation Programme (UIP), the government of Haryana was interested in innovative interventions to increase parental demand for immunisation.
Testing different strategies
The study examined several combinations of incentives, targeted reminders, and local immunisation ambassadors, and randomly assigned participants to receive each combination. These combinations resulted in a total of 75 different “policy bundles”.
Incentives: Family caregivers at 70 randomly selected Public Health Centres (PHCs) received mobile phone credits each time they brought their children to get immunised. The amount either remained the same throughout the immunisation schedule or was increased for the last two vaccines in the schedule. Some caregivers received total mobile phone credits worth ₹250 while others got ₹450.
Targeted reminders: Caregivers were randomly assigned to receive text and voice call messages reminding them that their child was due to receive a specific vaccine. Researchers randomly varied the fraction of caregivers receiving reminders in the catchment area of each subcentre: Either zero, 33, or 66%.
Local immunisation ambassadors: Within each village, 17 randomly selected individuals were asked to nominate people who they trusted to provide good advice on matters of health and agriculture ("trusted individuals") and those who were good at relaying information (“information hubs”). The respondents also identified a third group of ambassadors - the trusted information hubs. These individuals were good at relaying information as well as trusted to provide good advice. The six people nominated most frequently in each village were recruited as ambassadors for the programme. The ambassadors received one text message and one voice calls every month asking them to remind their friends, family, and other community members of the value of immunisation and, in villages with incentives, remind them about the incentives. Some villages in the study were randomly assigned one of these three types of ambassadors. And some villages in the sample had randomly selected individuals from the census as ambassadors. These villages were then compared with the ones that had no immunisation ambassador.
The most effective combination
Information hubs improved full immunisation rates, while SMS reminders alone had no impact. On average across incentive and non-incentive villages, in communities where “information hubs” (individuals identified as being good at spreading information) acted as immunisation ambassadors, full immunisation of children increased by 26%. In contrast, SMS reminders sent directly to a much larger set of parents had no impact on their own, compared to villages with no SMS reminders.
The most effective policy bundle was a full package that combined local immunisation ambassadors selected by the community, incentives that increased in amount across the immunisation schedule, and SMS reminders to caregivers about the next scheduled vaccine. This combination increased full immunisation rates by 55% relative to the comparison group.
The results suggest that local immunisation ambassadors amplified the effect of the other interventions, possibly by diffusing information about the incentives widely and explaining the content of personalised reminders. In this combination, high and low incentives are equally effective as long as they are combined with other interventions. In contrast, incentive values that remained the same throughout the immunisation schedule had no impact in any combination.
The policy that led to the largest increase in full immunisation for every dollar spent was the combination of information hubs or trusted information hubs with SMS reminders. It increased the number of immunisations per dollar by 9.1% relative to the comparison group receiving standard UIP services because the combination of the two interventions was both effective and inexpensive. In contrast, policies involving any amount of incentives were less cost-effective than the status quo on average.
What are the policy lessons?
The general conclusion that can be drawn from the study is that it is important to test at scale what combinations of interventions and what specific version of the policy might be most effective before widespread adoption of a policy.
A combination of inexpensive and effective interventions can both increase immunisation and reduce the cost per fully immunised child compared to the status quo.
Policymakers can benefit from leveraging community members to accelerate information diffusion and increase the effectiveness of other policies.
(The study has been authored by Abhijit Banerjee, Department of Economics at Massachusetts Institute of Technology and Director, J-PAL; Arun G. Chandrasekhar, Department of Economics at Stanford University and J-PAL affiliate; Esther Duflo, Department of Economics at Massachusetts Institute of Technology and Scientific Director, J-PAL South Asia and others.)