Is the American Dream fading? Economist Raj Chetty explores secrets of upward mobility
Chetty just won Harvard's George Ledlie Prize, for research that uses big data to study what causes diminished opportunities. His findings have global import.
He’s the William A Ackman Professor of Economics at Harvard University. But that descriptor doesn’t really capture the scale of Raj Chetty’s achievements.

The 44-year-old earned his PhD from Harvard at 23. By 28, he was among the youngest tenured faculty members in the history of that university. He’s been awarded a Padma Shri and the MacArthur genius grant, the John Bates Clark medal (given to an economist under 40 whose work is judged to have made the most significant contribution to the field), and, earlier this month, Harvard’s prestigious George Ledlie Prize, awarded for research deemed valuable to mankind.
Widely considered a potential Nobel laureate, Chetty focuses on the science of economic opportunity. His work stands out for its use of big data. In a wide-ranging conversation with Wknd on Tuesday, he spoke about his roots, his research, the conclusions he’s reached. He also touched upon the fading American Dream (also the title of a study he conducted in 2016), and offered advice to Indian policymakers.
* Can you take us back to your roots and tell us about your association with India and how that influenced you?
So, I was born in Delhi, grew up in Hauz Khas and went to school at St Columba’s near Connaught Place till fourth grade. I came to the US when I was nine. Going back one generation, my parents grew up in villages near Madurai in Tamil Nadu. And this is related to my research a little bit and things that I am interested in. Families that are low-income. My grandparents had a lot of kids, certainly looking for opportunities and the American Dream. My parents came to the US eventually. My mom became a doctor. My dad did his PhD at University of Wisconsin Madison. They ended up going back to India. My dad was at the Indian Statistical Institute and my mom was at the All India Institute of Medical Sciences and they wanted to raise their kids in India. So my sisters and I grew up there for a while. And then they ended up coming back after Indira Gandhi was assassinated. My dad was in the finance ministry and worked with people like Manmohan Singh. At that point, a lot of things changed. My mom is a pulmonologist and I had quite a bit of childhood asthma and of course Delhi, with its pollution, is not the best.
* It’s got a lot worse.
Yes, yes. My mom maybe was anticipating that and well, we ended up coming back.
* So you grew up in an academic environment…
Yeah, absolutely. I am the last one in my in my family to publish a paper. My mom and dad, and my two elder sisters are academics. My sisters are professors at Emory University.
We grew up in a family where I remember at age four or five my dad bringing home at that time a computer, the Commodore 64. I was learning some very basic programming and learning how computers work. We were always learning things.
I feel like a lot of the experiences I had as a kid, I now see through my own big data research. One of the main findings is that what you are exposed to as a child, and in particular if someone who looks like you is following a particular career pathway, has a tremendous impact on kids following in those pathways. And we think that’s leading to a lot of intergenerational persistence of income and so on. So yes, it certainly had an impact on my career choices.
* We’ll get to your substantive work in a bit, but you were writing papers when you were in high school, were tenured faculty at Harvard at 28. What do you attribute this to?
Honestly, I think a lot of it is just having the right opportunities and the right exposure. In a lot of people who have excelled in their field, there is a similar trajectory. Lots of prominent scientists grew up in a family where one or both parents was a scientist or interested in science. Many star tennis and basketball players, since a very young age, were exposed to that sport either because a parent was doing the same thing or a relative.
I think it almost rewires your brain in a certain way if you are, from a very young age, exposed to this. It becomes almost second nature.
I remember at age eight or nine helping with some editing of grant proposals for my parents. You are just in that environment and I think it makes you think in a certain way. And then of course I had opportunities to go to excellent schools such as St Columba’s, and here in the US as well, then to Harvard and so on. Of course hard work matters, but my view is that opportunity is a major factor.
* You started out in science and got drawn to economics. Within economics, the work you do, particularly on inequality of opportunity, is not something many economists seem to work on. What led the shift, and drew you to this theme?
When I was in high school, I worked in a microbiology lab that was focused on electron microscopy and understanding things at a cellular level. I could tell from working in that lab that my interest and skill was more in the statistics side of it, the data analysis side of it, less than the careful natural experiments, which are of course extremely important.
Then when I came to Harvard, I had a couple of, I think, really great opportunities. First I took an introductory intermediate economics microeconomics class with a professor named Andrew Metric. And he was just a phenomenal, very inspiring teacher.
Then, within a few weeks, I had a sense that I wanted to explore economics and statistics and things like that. So I emailed many professors saying that I was interested in working for them as a research assistant. I didn’t have any experience, but I said “I can learn quickly”.
One professor, Martin Feldstein, replied. He had been Ronald Reagan’s chief economic advisor and was a very prominent economist at Harvard. And I ended up getting an opportunity to work with him and really enjoyed that.
By the end of that first year, Marty said to me, “You have lots of ideas. I am happy to pay you as a research assistant but I actually think a better use of your time is to work on your own work and I will meet with you regularly.” Of course, as an 18-year-old, it was incredible to hear that from someone of his stature and gave me a great boost in confidence.
I was interested in the big questions. You asked why opportunity? I would say lots of people are interested in opportunity, even if they may not call it that. Economists who work on growth, who work on disparities, inequality, all of those things connect to opportunity in my view. I think, traditionally in economics, there has maybe been less of a focus on differences across groups or people’s chances of rising up or how things are changing over time. I don’t think it’s because people think those issues are uninteresting. They have partly been constrained by not having the data they need in order to be able to ask these kinds of questions.
I think part of my interest also comes from my own background — seeing the opportunities my parents had and how that shaped the opportunities we have, for the intergenerational effect; seeing the difference between India and the US, wondering why am I lucky to be in this position of having all these opportunities when it seems like my cousins, and lots of other people who are very talented don’t have access to the same opportunities. That seemed like an important issue to try to address.
* So your substantive research conclusions, of course, are deeply insightful in themselves. But I want to first ask you about the methodology that you bring to your research. How does big data come into play?
The analogy I would make here, which I find helpful, is the discovery of the compound microscope in biology, which allowed people to literally disaggregate and look at things at a much finer level, a cellular or subcellular level, in a way that was not feasible before we had those microscopes.
Big data allows us to first disaggregate, say, a national picture into much, much finer sub-units. Once you are able to break the data down in that way, you can make very sharp comparisons asking, say, why is it in this part of New York or this part of Washington DC, we are seeing kids doing really well, but in another part of Washington DC, outcomes look much worse even for children with comparable backgrounds, same parent income, same race and those kinds of comparisons.
Now there is one further challenge in economics that big data does not solve in and of itself, which is that, unlike in natural sciences, we can’t run lab experiments. There also, it turns out — and I think we figured this out over time — big data can be quite useful because with very large scale data, you can essentially start to find experiments, natural experiments within the data you have.
* You rely on census data and on tax records, but also creative and innovative data sets that people don’t think about. Could you walk us through the types of data you rely on?
In simple terms, the way I think about big data is essentially data that is not being actively collected through surveys, which is the traditional way that we collect data in the social sciences. Even today, there are major surveys in India like the National Sample Survey or in the US, the current population survey. Obviously that can generate quite valuable information, and for over a hundred years, has been the basis of whatever empirical economic analysis we have been able to do.
But increasingly, in the digital economy, there are many sources of data that are collected not for the purpose fundamentally of scientific analysis, but just for the purpose of other transactions. And the private sector is where I think there is much more progress to be made.
To give you one example, we have set up a collaboration over the past few years with Meta, the company that operates Facebook. And we were able to use the large-scale Facebook data, which again obviously was not fundamentally collected for research purposes. It’s a dataset used to link people and used for the company’s operations, but that data network has proved to be incredibly valuable in studying social capital, measuring social capital, understanding how it relates the economic mobility. And similarly, we have worked with datasets from credit-card processors, payroll companies.
The key characteristic of it is that it is generally very large-scale, covering oftentimes the whole population or a big chunk of the population. And the key aspect of it is that it’s not fundamentally designed for research purposes, it was collected for some other purpose.
And then you work on it through creative and careful work. This is where the methodological innovation matters. You can’t just take that data off the shelf and use it for a research analysis. You have to do a bunch of things to make it suitable for research to make sure it’s representative of the population you are trying to study. For example, if you look at credit-card data, you’ll see big spikes in spending on, for instance, certain holidays or during the Super Bowl in the US. So how do you filter out that kind of noise and extract the signal that you’re trying to study? Those are all issues that arise when using these kinds of datasets.
* So let’s talk about some of the broad patterns based on this methodology. The one big conclusion that your work has thrown up is that the American dream is fading. Why do you suggest that?
First, what is the American Dream? It is just the idea that, through hard work, anyone can go on to have a better standard of living than their parents. This, of course, is a fundamental thing around the world that people care about. And traditionally people have seen America as the land of opportunity, the place where you can actually achieve this.
In a paper published in the journal Science a few years ago, doing some careful data work — it’s not trivial to do this, but by linking data sets in the right way, going back historically — we are able to ask the very simple question: What fraction of kids do go on to earn more than their parents? We have to be precise about it, measure both kids’ and parents’ incomes, in their mid-30s, adjusted for inflation.
There is no spreadsheet going back generations, offering this data. And so you need to do some clever things statistically to figure out how, with the data you have, you can still do those kinds of calculations.
That’s the methodological innovation in this paper.
I will spare you those details. But the punchline is that back in the middle of the last century, for kids born in 1940 or 1950, something like 90% were earning more than their parents. So everyone was achieving the American dream, so to speak, in America. But then if you look at the recent generation, among people who are turning 30 around now, that number has become 50%. It’s a 50-50 shot as to whether you are going to do better than your parents.
If you plot this over time, you see a steady decline. And so in that sense, the American dream has faded. The US, in some sense, used to be a land of opportunity. It is less so now.
One caveat to that, I would stress, is that those calculations are for US natives. For immigrants, you continue to see that the US is a place where many people rise up. So it’s not that there is no opportunity in the US, but certainly your opportunities to rise up as a low- income American seem weaker than they did in the past. And so that then motivates our research, basically trying to understand why the American Dream is faded and how to restore it.
In my view, this is not just something of economic interest. I mean obviously it’s a big change in the economy that we want to understand, but I think it’s also central to the US’s role in the world, why it is changing, why there are so many social and political concerns here, why there is political unrest in the US. I think it’s all fundamentally related to these trends. And I think you see similar trends in other countries.
What we see is that our team does this kind of work in the US and other people are able to build on it and do similar things in other countries. In places like the UK and in other developed countries, you also see similar trends. While our work focuses on the US, a lot of these patterns are global phenomena, of global interest.
* So why has the American dream faded?
There’s a very nice book by my colleagues Claudia Goldin and Larry Katz, which I think captures the key idea that I am going to share very well, just in its title: The Race Between Education and Technology.
Technology is constantly improving. In a country like the US, there is more and more trade over time, with globalisation, which you can think of as basically a change in technology. As this progress is occurring, you can think of it as kind of a competition between humans and machines; or people in the US and foreign workers, to take the case of globalisation.
Now, if you look at the historical record up to about 1980, education measured in various ways was kind of keeping pace, in its progress, with technology. But since the 1980s, there has basically been a stalling of the level of education and the quality of education in the US. And when I say education, I don’t just mean the number of years that somebody went to school. I mean what they are learning, what types of skills they are acquiring, not just technical skills, but social capital, who they are connected to, who they are influenced by. That basically does not progress as much after 1980 relative to before. Whereas, of course, as we all know, the pace of technology has grown. And so, in a sense, human beings, particularly lower-skilled folks, are losing the race relative to technology.
What is the consequence of that? The amount you earn as a person who just finished high school or didn’t finish high school in the US has basically not changed today, relative to 1980. Why? Because you now face an enormous amount of competition, essentially in the global market, from machines. Nobody wants to pay you a high wage. It’s basically supply and demand playing out.
How does that then tie back in with the American Dream? If you now have poor opportunities to earn a high wage in the labour market, naturally you are going to have less of a chance of outdoing, say, your father, who had a job at General Motors or Ford, a high-paying job making cars, 30 or 40 years ago. And so it’s that macro trend that I think is driving this.
* Your research points to four predictive factors that can shape opportunities: access to education, whether a child grows up in a single-parent home, where you live (with neighbourhood occupying a vital place in your research), and interaction with higher-income households. On this basis, you’ve drawn up an Opportunity Atlas. Take us through it.
So, to segue to this from what we were talking about earlier, which was more at the national level and historical trends, there’s a limit from what you can learn just from that kind of data. And it connects to our conversation about big data methodology. You asked me a question about what has changed over the past 50 years? If all I have are the 50 data points of the past 50 years, you can come up with one explanation, I can come up with another explanation. Tons of things have changed over the past 50 years. It’s going to be impossible to tell which of these things is actually right.
So what we do to try to then make progress is take the data from the relatively modern era for kids born in the 1980s. We are using information from tax records for everyone in the US. It’s a giant data set, 8 billion rolls long, for everyone in America, anonymised, including you, including me, and including everyone else where we could see everyone’s incomes, where they live, who their kids were, where their kids go to college, where they grew up, just a very detailed longitudinal register.
You can then use that first to construct what we call the Opportunity Atlas, a freely accessible tool on the web where you can see the kind of statistics we have been talking about. What are the chances of achieving the American Dream, not just in the nation as a whole, but separately city by city, county by county, zip code by zip code down to the neighbourhood level, or census tracts?
There are 72,000 census tracts in America. Each of them has about 4,000 people. And within each of these places, we take the set of kids who grew up in that neighbourhood, follow them over time, over 30 years, and ask: If you grew up in a low-income family in that place, what are your chances of rising up?
The first very simple finding, before we get to the predictive factors, is that even in the current day, it turns out there are some places in America where you still have great chances of rising up. And there are other places in America where your chances of rising up really look very poor.
For instance, much of the Midwest, places like rural Iowa, relatively rural areas often have very high rates of upward mobility. You might not have expected that. Many people, if you ask them particularly internationally, where in the US do you think is the best chance of achieving the American Dream, if you are growing up in a low-income family? People would think of places like New York or San Francisco, and that turns out not to be the right answer. It’s the rural Midwest where kids have the best chance of rising up. And then there are places, particularly in the South again, big cities such as Carlyle in North Carolina or Atlanta in Georgia, which are some of the most rapidly growing cities in the US, where kids have the poorest chances of rising. And then we can go further and see that even within Atlanta, there are some neighbourhoods where the odds of achieving the American Dream look very good. There are other neighbourhoods where they do not, and so on.
So even before you correlate with other factors, you learn that the origins of these differences are not just about national policies and what’s happening globally. It’s something very, very local, which is very important to understand if we think about policy solutions going forward, because we have to think at that hyperlocal level.
We are focusing here in the US, and some of our former students at Harvard have now started to do analogous work in India. They have created analogous maps in India and have constructed detailed maps, for example, of opportunity in Delhi. And you can see the same exact thing: Enormous differences in children’s outcomes across nearby neighbourhoods.
And this is where, with very large data, you can make progress on these issues. In the US, we look at millions of families that move between neighbourhoods with kids of different ages, and we are able to show that, for every extra year that you spend growing up in a place that appears to have high upward mobility, your own chances of having this increase.
One of the experiments I like the best is looking at a family with two siblings. Say you and your brother moved to a place that has higher levels of upward mobility for the kids who grew up there from birth and, say, you are younger than your brother. We see that you do better than your elder brother exactly in proportion to the gap in your ages and in proportion to the number of extra years that you were exposed to the better environment.
This relates to old debates beyond economics about nature versus nurture and shows you that nurture really seems to matter a lot. Genetics matters, family matters, but clearly childhood environment also matters.
And then we take the next step: Now that we know some of these places are producing causally much better outcomes for some kids relative to others, what are they doing differently?
More mixed-income areas tend to have higher levels of upward mobility. Places with more two-parent families, more stable family structures, have higher levels of upward mobility. Places with better schools and access to higher education, have higher upward mobility.
And then finally, and I think in some ways most importantly, places with greater social capital, in particular connections between low- and high-income people, have higher levels of upward mobility. And the key mechanism there, and this goes all the way back to my own personal experience, is that it’s fundamentally about exposure. I think early exposure fundamentally changes people’s trajectories. So I think building that kind of cross-class interaction is extremely important to expand opportunities in addition to all of the other factors we have identified.
* How do race and gender factor in?
The way I would summarise it is that race absolutely matters in the US today, in terms of determining economic opportunity, but importantly in a way that interacts with gender. So we find big differences in rates of upward mobility and downward mobility for black versus white boys. But when we look at black versus white girls, we find essentially no differences, conditional on starting in families at the same level of income.
So there is a way in which race is interacting with gender. It may be related to discrimination and may be related to mass incarceration. An enormous number of black men in the US are incarcerated or unfortunately even have died by the time they are in their 30s. And so there’s a particularly adverse set of factors affecting black men.
* What would you suggest India can pick up from the kind of research you are doing?
I am not an expert on India and developing country issues, which I think are distinct from US issues in some ways. But of course they are related themes, right?
I think one thing that we have paid less attention to in economics in general, and I would say in the Indian context, is coming back to this point about social connection and integration and giving people opportunities by connecting across the faultlines that exist in society — geographically across religious groups, across caste. Unfortunately, as we are seeing in India and US, if anything, those fault lines are sharpening, not dissolving. And I don’t think that’s an accident. I think that’s partly a consequence of policy choices that are being made.
I think an important thing to think about is how do you bring people together, not just because it’s a good thing to do or because we feel like that leads to a healthier society, which may well be true, but also just from a simple economic perspective of giving people better opportunities.
So the kind of thing I think we need to be thinking about going forward is more of sort of a sociologically-inspired capitalism.
* It seems to me a little paradoxical that these faultlines have sharpened when there are three other trends that should have reduced the faultlines. Such as urbanisation; expanded access to English-language education; most probably most importantly, digitisation…
Ironically, I think those are three great examples of things that create opportunities for more interaction, but unless one is deliberate in actually making that interaction happen, they don’t necessarily lead to it.
Let’s take the case of urbanisation. You would think that in cities, you might be able to meet people from much more diverse backgrounds, and that would lead to more exposure and cross-class connection. On the contrary, what we actually see in the Facebook data is that you have more cross-class interaction in rural, less dense areas than in dense cities.
Why is that? Let me give you an example from my own experience. My wife grew up in a small town in Illinois with 7,000 people. Her dad was the town doctor. She went to the same public school as all the other kids. Her friends were from very diverse backgrounds. There was essentially no way to separate.
Now take a doctor in New York City, with a high income. Think about how New York City ends up being structured in a tremendously stratified way by class, where people are living in very different neighbourhoods, going to completely different schools. They have basically no contact with kids from lower-income families.
So ironically, even though you have more high-income people physically proximate to you, you actually may end up with less interaction with those people. And so the power of big data, the Facebook data in this case, is that we are able to identify directly the amount of cross-class interaction as opposed to just proximity. And what we are able to show is that there are many places, particularly dense urban areas that exhibit a lot of what we call friending bias. The idea that even though you are around many high-income people, you don’t end up befriending them because of the nature of how social interactions play out.
You could say the same thing about digitisation. I suspect that our interactions online are as stratified as they are offline. It’s not like you make friends at random online.
And so what that shows to me is these new developments can create opportunities. We have people who are no longer isolated in some remote village and have absolutely no access to higher income folks or opportunities. But we need to do something deliberate, create those mixed-income schools, those mixed-income neighbourhoods. One interesting pattern we find is if you look at the friendships people form, say in recreational groups in the context of sport or in the context of religion, they tend to cut a lot more across class lines. I think the logic of what’s going on is if you have one thing in common that you all share — the same sport team that you are rooting for, or a shared faith — then things like class fall into the background. And so I think those kinds of institutions, if designed in the right way, can leverage the trends that you are describing to actually lead to more connection and opportunity. But it’s not automatic.
* What’s next for you?
We have always got a big pipeline of projects. There is, as always, a lot more to learn.
A lot of what we are focused on these days is how can you go deeper and actually identify concrete policy solutions to change the kinds of things that we are describing.
For example, in the US, it could involve trying to change affordable-housing programmes. We spend $45 billion a year on affordable housing. Our sense is that those programmes are not working effectively in actually helping families move to higher-opportunity areas. So we have run some experiments and pilot studies providing additional social support to help families use these vouchers to move to higher opportunity areas. Those programmes seem to work very well. So we are now working with housing and urban development agency to try to scale those kinds of interventions.
We are doing similar work in the higher-education space. We had a recent paper on who gets into top private colleges in America and how kids from higher income families are more likely to get in and how that may perpetuate inequality across generations. So working with folks in colleges across America on how you might change admissions policies to distribute opportunity more widely.
We are basically trying to go one layer deeper. We are getting more of an understanding of what’s going on and then trying to use that to really influence policy decisions made at the federal level, at the local level, at non-profits, colleges, etc, so that there’s a real change on the ground going forward. And our hope is while we focus on the US mainly because you can only do so many things and that’s where the data are best available and so on, it’s been nice to see that in India and in many other countries, people are picking up on this.
* If you could give one piece of advice to Indian policymakers, what would it be?
At the biggest level, I would say focusing on economic opportunity is of central interest, not just from the perspective of justice and fairness, which is what many people think about and is of course important, but also if you just care about economic growth.
Forget about fairness, and suppose you just want to maximise India’s GDP growth rate. I think figuring out how you harness more of the incredible talent that is out there that’s not coming through the pipeline and getting these opportunities is the surest way of increasing the growth rate in the long run.
In the US, we have coined this term, Lost Einsteins. There are lots of people who, if you look at how they were doing early in life, say through their performance on math tests and things like that, appeared to have the talent to go on to have a big scientific contribution, patent something very important. But in practice, if you are black, if you are a woman, if you are growing up in a low-income family, your odds of becoming an inventor are drastically lower than if you are a high-income white boy. And so we estimate that you have four times fewer inventors than you otherwise would because of this Lost Einstein problem in the US.
My guess is that, in India, that number is even higher. And we find similar patterns in terms of thinking about who is starting highly influential businesses that end up hiring a lot of workers and so on.
So I guess the single piece of advice I would give is that lots of people are thinking about how India’s growth miracle can be sustained. How do you compete with foreign countries? And I think the single biggest asset a country like India has is its human capital. It’s not fully utilised. And I think there are ways that we can figure out how to fully utilise it. That relates to some of the things we have talked about, like integration, social capital, changes in schools in certain ways, access to higher education.
