May 28, 2011

Everybody Loves Bombs! According to a Poll...

I've just lifted this from a forum as it's very relevant to a post here last week.  

The justifications for trying to bomb Libya to democracy have had a boost in the last few days, with numerous news sites carrying a poll carried out by a an American research organisation called Ipsos.  The poll was produced on behalf of the respected news giant Reuters.  Although the headline from the researchers themselves was "mixed support" in NATO countries, many news media highlighted what appeared to be some very high levels of support in certain other countries of the world, for the war against the Libyan Government.  Here's a summary of the findings, quoted from the Ipsos site (my underlining):

"The study, which was conducted after a UN resolution approved a “no-fly zone” and NATO nations began bombing missions against Colonel Gaddafi, the current ruler of Libya, indicates that citizens from South Africa (83%) are most likely to support the action, followed then by those from Belgium (78%), Australia (77%), India (75%), France (72%), Sweden (71%), Canada (70%), Brazil (68%), Japan (68%), Spain (64%), South Korea (63%), Poland (62%), the United States (61%), Germany (59%), Great Britain (58%), Hungary (54%), Mexico (53%), and Italy (52%).


Those countries least likely to support the mission are: Russia (23%), Turkey (35%), Argentina (38%), Indonesia (46%) and a completely split Saudi Arabia (50%)."

Interesting stuff.  But I have a problem with it, for two reasons:

My first concern is the idea that what "most people think" is that same as what is right.  Yes, you could argue that it's the cornerstone of democracy, but that is basically true only in principle: every day governments that are democratically elected, make decisions they know people do not want, because they have a mandate to decide.  Democracies like the US or UK are not based on the people deciding.  They are based on the people's deciding who should decide.  And the people then complain about them until they can change them.  


Even that's a risky business - after all, George W Bush won the popular vote. Twice. So did Hitler for that matter.  As another instance, up to 2% of the electorate in some western countries protested about the war in Iraq in 2003 - about the same percentage as took to the streets in Cairo this year. Nobody listened.  Then in the UK for example, 80% of people voted shortly afterwards for one of the parties that had supported it. And  yet polls now show most people think the war was a bad idea. 

So "the people" can't make up their minds can they?  Either way the point is that the media articles around this poll (not the poll itself, to be fair), try to make a political case that the war on Libya is good, because most people say so.  I am not sure that's intellectually sound as any kind of argument.


But let's assume it is - that the majority opinion is paramount on any given subject, and the outcome of this poll is therefore very important.  My other worry is then: How accurate is this poll?  Does it really represent the percentages it claims?  Not every person in the world can be asked the question, but who is asked and how, are important.  So I had a read of the whole press release from Ipsos, and was a bit shocked to say the least at the differences in the impression gained from looking at all the facts, and from just looking at the reported summary above.  To begin with, look at the question asked in the poll:

"As you may know, a number of countries who are part of the North Atlantic Treaty Organization (“NATO”), including the United States, France, Great Britain, Canada and Germany, undertook, with the support of the Arab League, to enforce a United Nations Resolution and intervene, militarily, in Libya. Based on what you have seen, read or heard, do you support very much, support somewhat, oppose somewhat or oppose very much, this intervention?"


The question asked in this poll is factually incorrect.  It was asked during two weeks in April. That was after Germany had pulled out (ten days into the bombing). They pulled out because the "no fly zone" had become a war - on the side of some Libyans against other Libyans. So Germany is not supporting military action, let alone participating, and neither are the four Arab countries who initially pledged to enforce a "no fly zone" - presumably for the same reason.  Canada aren't involved in the intervention either. And the Arab League wasn't "supporting intervention" at the time of this poll: it was calling for a ceasefire. Well, at least they got "Libya" right in the question - not a lot else.

But more importantly, the sample size for most of these countries is pathetic: they cite important countries like South Africa (an important African ally), Saudi Arabia and Turkey, based on only "a minimum of five hundred...maximum of one thousand" clicks on their site from that country. That's hardly a "survey". Around 18,500 people covering 23 countries.  One thousand in India - that's one person for every million.

Considering too that only 6% of adults in India have internet access at all, even Saudi Arabia only 29% and that this question was available for just fifteen days on a single news site, the methodology is ridiculous.  There was no way of ensuring a balanced, representative demographic because the survey was passive; who ever wanted to be in it was counted.  Or whoever could.  It was therefore limited to a tiny sample from an economic elite.  Another thing - it took three weeks to publish the results, i.e. some of these alleged opinions were collected over a month ago.  Hardly "news", then.


Finally, perhaps the most spectacular distortion of this "survey": not only was it limited to people who chose to visit Ipsos's "Reuters poll" page, but it was also only available in English.  What the hell kind of international survey limits its results to the educated, internet-savvy, western-news-reading, and bilingual?

So 83% of mostly wealthy, inevitably white South Africans out of the five hundred who were reading about Libya online in English at Reuters, who decided that they wanted to click the link and vote on a poll...didn't like Qadhafi a month ago.  Is that really worth talking about?  Or that out of a similar number of English speaking Saudis who read their news on Reuters and thought they'd vote in a poll, they were evenly split? And a few hundred English speaking Turks who read their news in English on Reuters etc...they mostly don't like it?  And this is assuming too that the country locations recorded by the site are always correct - and that the respondents are nationals of those countries rather than expatriates or visitors; impossible to know.

All in all this is a tiny, culturally and economically specific sample, made to look like the voice of whole countries. It is the headline in a thousand news stories around the world, based on a few mouse clicks in a coffee break.  Whatever you or I think of the war in Libya, there is a serious issue here.  As more and more people rely on the internet for information, and more companies (even respected ones) use the internet as an easy way to carry out research, the risks are increasing.  Reading the first few lines of this post you might well have had the impression that millions of South Africans are cheering for the bombs to rain on Tripoli.  And yet if you dig through the facts - that the newspapers don't repeat - you find a meaningless bit of web-clicking by a few hundred people at most, on the condition that they preferred to get their news online, in English, at Reuters, and felt like expressing an opinion in this instance.  Regardless of the sample size, that simply could not have been a representative and diverse group of South Africans, Saudis, Turks or Indians.  Or anyone.

Mark Twain wrote a quote that he attributed to Benjamin Disraeli: "There are three kinds of lies: lies, damned lies, and statistics."  Perhaps there is a fourth kind now: internet-based polls.  Or worse, internet polls repeated selectively by newspapers.

4 comments:

Anonymous said...

Basic statistics say that 500 is more than ample to give valid figures. However, you may have a sample bias.

Also I was blocked when trying to follow the blog!

davidalockwood said...

Unfortunately I take all polls with a ‘pinch of salt’ these days; especially since news has now become entertainment!
Hard factual news seems very difficult to find, it all seems biased by the media presenting it. Therefore by default all polls conducted by media will be biased towards their particular point of view and politicians (excluding the likes of Norman Tebbit , Anthony Wedgwood Benn et-al: excuse the England based example) seem incapable of making decisions without the blessing of various media institutions.
The internet has now taken over from the TV (I saw it on TV so it must be true) No one wishes to spend time verifying the truth of articles they see, read or hear; very disturbing when it results in people being killed.

Sorry for the rant!.....

The Linoleum Surfer said...

Anon: Not sure why you'd be blocked - try again, do! Thanks a lot for your comment anyway. And I agree, there's nothing inherently wrong with a small-ish statistical sample in itself.

But the smaller the sample, the more care you have to take to ensure that it is representative. In this case, the self-selecting sample was funneled through such restrictive criteria that the distortion is guaranteed: Taking Saudi Arabia as an example - which Saudis read their news online, read it in English, from Reuters, and feel happy to contribute voluntarily to surveys from their PC?

I would doubt that many of the sample were Saudis at all - this sounds far more consistent with a demographic of bored western expatriates! Maybe even western military personnel among them! Some representation of Saudis, eh?

DL: Don't worry, your rants pale compared to those already posted by me! I agree about the "Chingford Skinhead" and Tony Benn - united in their intransigence from opposite ends of the spectrum, and indifferent to the whims of media-driven public opinion. I admire them both for it, although I rarely share the views of either!

The alarming thing about this poll was that it was carried out by one of the leading organisations, for probably the most serious and professional news organisation on the planet. If they make such glaring errors, whose figures can be trusted?


Thanks again to both of you - your reading, sharing, commenting and (if possible!) following, make it all worthwhile. Big love.


TLS

Al Jamali Crusher said...

Ever considered going back to Reuters and asking them? Just a thought.....