Democracy

Economy

Education

Development

Global Issues

Research

06/25/2009

Reasonably, the probability of SBY-Boediono to win this Wednesday election is higher compared to Mega-Prabowo and JK-Wiranto. The question of course is, by how much?  Noone knows for sure until the election is over.

Enter polls, which are designed to gauge that probability. And there lies the challenge. Opinion polls can suffer from statistical errors, deliberate bias, dishonest respondents, and any combination of the above. Even experienced pollsters cannot avoid them completely.

Perhaps the most revealing example is the polls during the last three US presidential elections. With the exception of the Investor Business Daily in 2008 and Rasmussen in 2004, almost all pollsters went astray. John Kerry's somewhat imminent victory in the early exit polls in 2004 turned out to be a blunder. Obama's double digit advantage in 2008, according to many polls, was far above the election results. And Gore's deficit by 5% to Bush in 2000 in the CNN/USA Today/Gallup polls was completely irrelevant as the election ended in a very tight margin.

How could the pollsters be so wrong? Even Gallup and Rasmussen were perplexed.

Presumably, the surging of polling in Indonesia during the election season is inspired by the same practice in the US. However, unlike the US, Indonesian pollsters just started the practice five years ago. It was understandable when they made a terrible error at that time.

In the run up to the second round of the presidential election in 2004, the polls by the Indonesian Survey Institute (LSI) and the International Foundation for Electoral Systems (IFES) consistently showed 60% support for SBY, 20% for Megawati, and roughly 18% were undecided.  With a margin of errors of 4%, as reported by the two pollsters, if the polls were done repeatedly, 95% of the time the true percentage of voters who would vote for Megawati would be between 16% and 24%.

Statistically, it is difficult to understand how Megawati managed to obtain about 40% of the vote in that election. Maybe the surveys were flawed, intentionally biased, contained many dishonest respondents, or have a sizeable portion of respondents who ended up changing their minds when they are in the voting booth.

That is why polling on the presidential race is a complicated business. And if the more experienced US pollsters can suffer from a terrible blunder, what about Indonesian pollsters?

Much of the anomaly in the US presidential race polls is due to the uncertainty about the party affiliation of the voters.  Unfortunately, this is also true in Indonesia. In fact, the biggest challenge for Indonesian pollsters is to figure out the real political affiliations of the voters. Put simply, if you walk on the street, what is the chance that you will meet a base supporter of SBY-Boediono, Mega-Prabowo, or JK-Wiranto?

This supporter identification is used to weight the responses of the respondents in order to obtain the poll’s aggregate result. Putting too much weight on certain candidate will skew the result.

Ideally, the results of the last April legislative election can be used as a benchmark. However, since the results were relatively diverse, it may not be accurate. In addition, those who voted in April might have different attitude in this July election. Another reason is the expected higher turnout in this presidential election can make the benchmark less relevant.

Some may say that the polls during the legislative election were fairly accurate. But this is rather misleading. First, the number of the contestants involved in the election was huge. In that case, any polls can either be accurate or completely wrong. Had the number of the parties been three or four, the election results and polls would have been useful as a good benchmark.

Secondly, the nature of the two elections is different. Consequently, the polls on the legislative elections render irrelevant to the incoming election.

To make matter worse, to assume that those who voted for candidates from the Democratic Party, the National Mandate Party, the Justice and Prosperity Party will automatically support SBY may be not accurate. Even Mega-Prabowo, for instance, are not necessary fully supported by the grass-root and key figures from their own parties.

So far none of the pollsters makes it clear to the public about the way they determine the weight.

Another big issue is who is sampled. Are the respondents among likely voters, which consists of those respondents who have previously voted? Or, are they among an ‘expanded’ likely voter, which consists of those who are likely to vote but have not necessarily voted in a prior election? Perhaps, it is better for the pollsters to consider these two scenarios.

The latest poll conducted by LSI showed support for SBY-Boediono at 67%, while Mega-Prabowo 16%, and Kalla-Wiranto 9%. Meanwhile, early in June, the Information Research Institute (LRI) reported that SBY would win the election by around 33% of the vote, followed by Kalla and Megawati with 29% and 20% of the vote, respectively.

Unfortunately, there is little incentive to be not skeptic of their results as LSI is known to have close ties to SBY-Boediono camp and LRI is leaning toward JK-Wiranto.

Forget the non-independent pollsters.  Even for the independent ones, unless they clearly describe describe their sampling method and weighting technique, the results of their polls should be taken with a grain of salt.

As Benjamin Disraeli once said, “There are three kinds of lies. Lies, damned lies, and statistics”. Polls that suffer from statistical errors are not lies, but they are just equally misleading.

Link: Megawati: a victim of biased polls?

  Your comment

Back to top