The controversy around government surveys in India

A panel under the chairmanship of India’s former chief statistician Pranab Sen will review the conduct of all statistical surveys conducted by the government

The union government has made a committee under the chairmanship of India’s former chief statistician Pranab Sen to review the conduct of all statistical surveys conducted by the government. The decision follows a public debate initiated by at least three economists associated with the Prime Minister’s Economic Advisory Council (PMEAC) on the robustness of the existing survey-based data. To be sure, questions and doubts around government surveys are neither unprecedented nor unwarranted. However, it is important to look at them in a holistic manner to understand the current controversy. Here are five points which explain this in detail.

Questions around govt surveys are neither unprecedented nor unwarranted and we dissect the same in a holistic manner here. (HT file photo)

1. India’s statistical system is in an unprecedented crisis of data availability

The root cause of this is the non-availability of census data due to the delay in conduct of the 2021 decadal census in the country. The decadal census has never been delayed since 1881 and the government’s official excuse of the pandemic being responsible for the delay in conducting the census cannot be believed. While there are some exceptions, census data offers the guiding frame for conduct of almost all sampling-based surveys in the country. Using obsolete census data means the surveys have become obsolete as well.

2. Part of the data availability crisis predates the delay in 2021 census

It started with the government junking the 2017-18 Consumption Expenditure Survey (CES) after its findings were leaked in the media in 2019. Non-availability of the CES, apart from its direct fallout such as lack of information on consumption inequality and poverty has also led to complications such as an obsolete Consumer Price Index (CPI) and GDP series which have not been updated after 2011-12. One can argue with a reasonable degree of confidence that had the CPI been updated, we would have seen a reduction in the weight of food (and associated volatility) in the benchmark inflation numbers. While the government cited data quality issues for junking the 2017-18 CES, a lot of independent economists and former members of statistical bodies criticized the move as ill-advised.

3. Controversy around CES data is not new in India

In fact, this is the crux of what is known as the Great Indian Poverty Debate. There are two aspects of this debate.

The first was a controversy around what is referred to as the recall period in CES. In simple terms it means the period over which a respondent is asked to report consumption spending. To give an example, asking whether someone brought a piece of clothing in the last one week or one year can change reporting of consumption. This was the main controversy when the 55th round (1999-00) CES data was published and it was resolved with the 61st round (2004-05) survey resorting to the 50th round (1993-94) survey. The 66th round CES has different/more recall periods from the 61st round CES. While there is no official word on it, the new CES which is supposed to fill in for the junked 2017-18 round is reportedly once again experimenting with recall periods.

The other part of the CES controversy is the growing gap between estimated consumption in CES conducted by the National Sample Survey Organization (NSSO) and that done by National Account Statistics (NAS), with the latter always being higher. Some economists, such as Surjit Bhalla, have argued that CES underestimates consumption (and hence overestimates poverty). The issue of divergence between CES and NAS estimates has been adequately debated and almost all reasoned conclusions have decided against replacing one with the other. Without prejudice to valid concerns about data quality and associated problems ( more on this later) this is a justified approach, because NAS and CES are meant to measure different things. As was discussed in these pages in an earlier piece, even the government’s post pandemic food security scheme would not have passed the NAS consumption data test.

4. The latest criticism of government surveys by PMEAC economists is, in a way, inspired by the NAS critique

Major factual points raised by PMEAC economists against NSS and other surveys include an overestimation of rural population share, higher rate of non-response among richer respondents etc. While their critique also makes the right noises about non-politicization of the debate around data quality in India and cites earlier academic works criticizing such surveys, the focus on things such as higher rural share and low response rate by the rich, does seem to suggest that the criticism is inspired by the pro-NAS sentiment of NSS and other surveys underreporting the fruits of economic growth in India. While some of their points, such as the NSS undercounting the rich, have always been well-known – this newspaper almost always uses this caveat with its analyses using NSS data – some of their criticisms are a direct fallout of the government delaying the 2021 census. It is interesting to note that a section of economists -- Jean Dreze is the most prominent among them -- have been making an almost similar criticism from an opposite side about private surveys such as the Consumer Pyramid Household Surveys (CPHS) conducted by the Centre for Monitoring Indian Economy (CMIE). Because of non-availability of CES data after 2011-12, surveys such as CPHS have gained widespread acceptability.

5. What will the new committee achieve?

One must wait for its findings to be published before jumping to any conclusions. However, the root cause of the crisis in India’s statistical system, which was once not just the best in the third world, but comparable with first world standards, is not very difficult to pinpoint. It is a mix of political pliability, resource deficit and lack of institutional zeal. As is obvious, these are hardly problems which are plaguing only the statistical machinery in the country.

These problems notwithstanding, any argument that hopes to fix all problems with Indian statistical surveys by methodological tweaks suggested by a committee alone is symptomatic of a problem captured by a Russian poet Vladimir Mayakovsky in these lines written in 1932, “Oh, for just one more decisive conference, concerning, the abolishment of all conferences!”.

A better way to solve the puzzles of reconciling the developments in the Indian economy in the post-reform period with data in various government surveys will be to publish a lot of other data which is sitting with the government but not available to independent researchers. Unit level statistics on income tax returns and the Ministry of Corporate Affairs database on companies are two of the obvious examples which come to mind.