Madness in the method
Methodology of opinion and exit polls and their lack of transparency are worrying.
One of the most engaging debates in the run-up to Verdict 2004 has been on whether opinion and exit polls should be allowed to have a free run or not. Ironically, the issue resurfaced after it seemed to have been settled in favour of continuance of the polls in 1999 when the Election Commission’s ban, the hearing in the Supreme Court, the subsequent withdrawal of the ban and the CEC’s call for a national debate on this question brought the business of pollsters into the public arena. Clearly, the issue wasn’t really settled. But with the Supreme Court reiterating its earlier verdict, it is important to frame the issues that will be debated when the matter comes up before the apex court after the poll results are out.
It has been argued that opinion and exit polls need to be stopped because they are often motivated and are thus unreliable. Then there’s the second point of them influencing voter behaviour and thereby tampering with the electoral process. It is not surprising that the most vociferous demand for outlawing opinion polls has not been made by its end users — readers and viewers — but by political parties and their leaders. Peculiarly, the EC, despite its 1999 misadventure, seconded the demand of these political parties. Fortunately, better sense prevailed and the move to enact a legislation banning exit polls was abandoned, thereby avoiding an unnecessary controversy.
But the hysteria within political parties has been curious. After all, much before the media commission polls, several political parties engage polling agencies. The polls are used in framing strategies, selecting candidates, zeroing in on allies and, in the case of the ruling party (after computing with inputs of intelligence agencies), deciding on the timing of the polls. So are not political parties being hypocritical when they demand a ban on opinion and exit polls? If they can seek external assistance in making an assessment of the political reality and take corrective measures, why should the people who finally vote for them be denied access to information on how a cross-section of people are thinking? Moreover, if parties really doubt the veracity of the polls published in magazines and aired on news channels, then why do they keep commissioning pollsters?
Though the controversy laid the ground for a reasoned debate, the opportunity has been frittered away by either raising it to a hysteric pitch or by limiting its focus to whether such a ban would lead to a violation of a citizen’s fundamental rights. As a result, public attention has not focused on the most crucial issue — the methodology of forecasting used by the agencies and the lack of transparency and the failure of the agencies to sufficiently warn the public that their findings are subject to considerable error margins. The issue of polls influencing voters is different and should be delved into only after resolving these basic questions.
There is a clear case in favour of both the continuance of opinion polls and a call for agencies to take a fresh look at their existing statistical models, their method to determine the size and character of the sample size. After being around for several decades, polling is a legitimate science that provides indicators to what people perceive. By tracking a select group of people, it is also possible to track changes in perceptions. These inputs have been used by companies to fine-tune production and marketing strategies as well as by political parties to shift the focus from one issue to another. In any democracy, any call for limiting the use of any science to a select few groups goes against the basic tenets of a democratic system.
Despite the fact that political forecasting in India has been around for almost three decades in various forms — and in differing intensities — there is a basic flaw in the approach to polling. This is because most agencies engaged in the business have cut their teeth in marketing surveys. As a result, political parties are often presented as brand choices: BJP=Colgate; Congress=Pepsodent. While the market-survey choice works for consumer products, there is a problem when it comes to gauging political behaviour in a country of India’s diversity. In a particular village, the choice of detergent won’t be determined on the basis of caste but on the basis of purchasing power.
But when it comes to political perception, the response of Thakurs will, in most cases, be different from the Dalits. This makes it imperative for polling agencies to draw a sample that reflects the caste pyramid of the country. With no authentic caste-based data available, agencies often refer to past data and interpolate them with several other factors — all of which require the tools of social science and, barring an odd agency, there is none adept in this field.
Campuses in the capital abound with horror stories on how the surveys are conducted — mostly by students on the threshold of embarking into a profession. Their ability for being discerning when it comes to ‘filling the forms’ is suspect because dealing with unknown people requires a skill that comes only with exposure. There are other agencies either outsourcing the survey or using freelance surveyors who make a living by periodically picking work from various agencies. None of the categories of surveyors are actually trained to read between the lines — during the opinion polls — and use personal discretion in selecting the sample during exit polls.
It is possible that not all of this might be true for most agencies. But to come to that conclusion, they need to be more transparent. After all, there must be an explanation as to why agencies have gone off the mark in election after election. Right from the 1999 Lok Sabha election, when two leading agencies ended up forecasting more than 20 seats for the NDA, to the last round when none of the agencies had an inkling of the extent of the BJP’s sweep in Madhya Pradesh and Rajasthan, pollsters have more often than not got it terribly wrong. But because of the various layers of forecasting, they have had an escape route — if the national aggregate does not tally with the final result, then some of the state-wise disaggregate matches the final result!
It is not that pollsters are perfect in countries where there is a longer tradition of polling and are not as politically and culturally diverse as India. What happened in the US in 2001 is still fresh in our memories. But questions regarding pollsters do not abound in these countries like they are doing in India today.
Given the fact that the debate over opinion and exit polls shall continue, the final results of the elections would have a direct bearing on it. This is because it would be possible to determine the ability of pollsters to assess the mood of the nation in an exercise that has been unprecedented — both in terms of time-span and in terms of the number of agencies at work.