20 Nov 13 Of Opinion Polls, Market Research and Indian Elections

Elections are back, and so are opinion polls.
As usual, this has caused much acrimonious debate – not only on the results, but on the efficacy of opinion polls themselves. One group dismisses all polls as unscientific, another views them as biased or manipulated, while yet others want them banned.

Unfortunately, amidst all this cacophony, we rarely hear rational explanations about the limitations of polls, or their different methodologies or impact of sampling. Ideally, pollsters should be helping simplify and explain the results – yet, most of the time, we get absolute results without any caveats on confidence levels and likely error ranges. At the most, we get some data on country-wide sample sizes. But this is not good enough – especially in India.

In the US, for example, a majority vote share in Florida enables us to predict actual delegates won (in Florida) fairly accurately. However, in India, an individual’s vote only counts towards 1 seat. So, vote share in Maharashtra as a whole may have little impact on who wins in Pune. And this is where the problem becomes more complex – translating vote share into seats.

First of all, the sample must reflect the heterogeneity (geographic, social, economic, cultural, religious, demographic, etc.) of the voting public in that constituency. The latter is very difficult, as is truly random sampling. Let us assume for now that the researcher is able to pick a fairly representative sample. How large should the sample be?

Lok Sabha constituencies (2009 data) are of varying size: from around forty-six thousand (Lakshwadeep) to close to 1.9 million voters (Unnao, UP). Let us take a constituency with 1 million voters. According to the theories of statistics and market research, a sample of 1000 people will provide us with a 95% confidence level and a confidence level of +/- 3.1. This means that if the winner has 30% share, we are 95% certain that the vote share is between 26.9% and 33.1%. This is a huge variance, and an effective swing of over 6%. Even if we have a sample of 2000, the confidence level will be +/- 2.2.

This will not matter if the difference between the winner and runner-up is large. But given the proliferation of parties and candidates, and closeness of results in India – even if we had a national survey with 543,000 respondents, we could end up with a high degree of uncertainty.

All this is further compounded by the nature of questions asked, choices given, and mode of questioning. Also, in most places (especially urban), only 50% of the people vote.

This is not to say that opinion polls or market research have no value. In fact, changes over time reflect important trends. However, any tool has its limitations, and in order to use the tool or interpret the results effectively, one must understand these constraints.

The answer is not to ban opinion polls, as that will only lead to even more dubious informal polling. The best solution is to make public all the data, and especially:

• who has conducted the survey
• who has paid for it
• sample size and methodology of choosing the sample
• the social/demographic profile of the actual sample
• actual questions asked
• raw vote shares

This will help analysts and others make sense of the data, and draw their own conclusions. It’s not really the business of the government to regulate this, but industry bodies (in market research and media) could well lay down certain standards. Over time, those who do not offer these disclosures will find their opinions dis-regarded.

And finally, interpretation should not be left to the whims of political parties or TV anchors.