This interview on the 80,000 Hours podcast with Rachel Glennerster (chief economist at the UK’s Department for International Development and former executive director of J-PAL) is a great overview of some major issues in Global Development (as of 2018) and particularly how a practical academic thinks about, among other things:
- Aid, economic development and smart spending;
- Effective use of data to evaulate and design programs and influence policy ;
- Use of Randomised Controlled Trials;
- The differences between small-scale trials and making interventions work at scale;
- Good governance.
Recommended. Here’s a (long) snippit I found helpful about how she thinks about Randomised Controlled Trials (RCTs):
Nathan Labenz: Moving to RCTs in general and the state of debate around how much we should rely upon them. You mentioned that it’s kind of a 50/50 split right now, in today’s work. Do you think that’s an appropriate split? Do you think that it should be all RCT’s? What do you think is the right balance as we try to figure out what is obviously a very complicated world?
Rachel Glennerster: I think it’s really important to say that all of us who have worked on randomized trials have never suggested that this is the only methodology that you should use. Sometimes it’s held up as a straw person that we go around saying, this is the only methodology, people criticize us for saying it’s the only methodology, but nobody who’s done RCTs has ever thought that that’s the right approach. I think the right way to see things is you have a toolbox of ways to answer questions, and the right tool depends on the question that you’re asking.
I think we need good descriptive work to understand what the problems are. A lot of development programs just fail because they’re trying to solve a problem that doesn’t exist. They’re just solving the wrong problem. The first really important thing you’ve got to do is really understand what the issue is in any given area. If we’re worried about girls not going to school because of menstruation, well, let’s start by finding out whether they actually don’t go to school more when they’re menstruating. That’s a really basic, obvious thing. But we actually need more work on that kind of understanding the context, understanding the problem, is really important first step.
When I started doing agricultural work in Sierra Leone, and the first thing we did was work with the government to do a really detailed analysis of what are the problems for smallholder farmers in Sierra Leone? Not RCT, just descriptive. It turned up all sorts of interesting facts that people weren’t aware of. I think that’s really important, I think, then doing an RCT is useful for answering a really specific problem, a really specific question. But I think the best RCTs are the ones that test a theory. They test something that’s more generalizable than just does this program work? Its asking a question about human beings.
Here is an example. I did a project looking at how to improve immunization rates in India, which was fantastically effective. It started with a first assessment of what are the health problems in this area? Only 3% of kids in this part of India were getting fully immunized. Given that immunization is one of the most effective things that you could do, that rate is just appallingly low.
There were a number of theories about why that could be, and a lot of people said, “Well, people don’t trust the doctors, they don’t …” Well, not doctors because you rarely get doctors in rural India or rural anywhere, but nurses and clinics. So, they don’t trust the formal health system.
There was also a question of, so the clinics are often closed, so is that the problem? Is it that when you go and take your kid to the clinic, it’s often closed? Is it nurse absenteeism that’s the problem? Or is it just a behavioral economics thing that you’re happy to get your kid immunized, but you’ll do it tomorrow?
We read all this behavioral economics and we said, “Well, maybe we should look at that.” But we also wanted to test these other ideas. One arm made sure that without fail, there was someone to immunize your child and another arm did that, but also provide a small incentive. So, yes, we were testing a program but we were also asking a more fundamental question, which is, why don’t people get their kids immunized?
What we saw in the data is a lot of people got their kid immunized with one immunization, but they failed to persist to the end of the schedule. Which already, that’s just descriptive data and it starts to tell you, it’s not that they distrust the system or that they think that immunizations are evil, because they’re getting their kid one immunization. It’s more question of persistence. Now, fixing the supply problem increased the number of people getting the first shot, and the second shot, but again, it failed to fix this persistence problem. Where the incentive effect worked, was it helped people persist to the end.
That tells you that one of the big problems was this persistence problem. It tells you a lot about why immunization isn’t happening. Now, that project was completely impossible to scale. We were handing out lentils and the middle of Rajasthan where nobody showed up. It was just you would never … This was like economist designing logistics. It was a disaster. We learned a lot but you would never want to actually do a program like this.
A colleague of mine did something similar, where another program we had just done where we ended up improving teachers attendance by having cameras that were wrapped in sellotape and signed. Again, the logistics was a nightmare but it tested a theory. Once you have that, you can think about what’s the implementation issues? How do we implement this at scale because you better understand the problem. You want to use an RCT when you can test a specific problem and get an answer to why something is an issue. It’s an important question, you can answer it well, and it has broader implications. But you also need to use other types of methodology when your question is of a different kind.
The 80,000 Hours Podcast – Ten Global Problems: Rachel Glennerster on Global Poverty [Transcript]