Can the World be Wrong?

Chapter Two: Can the World be Wrong?

 

 “Public sentiment is everything. With public sentiment, nothing can fail. Without it, nothing can succeed."

- Abraham Lincoln

 

“There are lies, damned lies and statistics."

- Mark Twain

 


SUMMARY

  • The central thesis of this book is that global public opinion polling, tracked over time across key countries, can help predict the future.
  • Multi-country public opinion polls give a good approximation of what can be called a global ‘body politic’ with real influence.
  • However, using polling to help predict the future can be wrong in five possible ways:
    • Some polls are deliberately misleading, with biased questions designed to support (often un-named) clients
    • Polls can be technically wrong due to weak sampling methods or faulty translations of questions or responses
    • Some questions are conceptually flawed due to overly simplifying possible responses or muddling concepts (like deficit and debt)
    • Polls can accurately reveal perceptions that are factually incorrect and hence unlikely to make it through the policy development process
    • Even the highest-quality polling can result in findings that do not get reflected in public policy or societal outcomes because of strong opposition from powerful elites, at least for a while.
  • Polling is both an art and a science. We social researchers definitely get it wrong at times just like anyone else, but generally our industry standards are good and our methods and results are sound.
  • This book, based as it is on decade-long trends from scientific surveys across scores of countries, provides reliable metrics for both understanding the world today and developing likely scenarios for the future.
  • To help the reader come to his or her own judgment, the exact question wording, methodological details and country-by-country results are given for all questions in the Appendices.
  • So can the world be wrong? Yes it can, but this book argues that polling trends are going to be less wrong than almost every other way of anticipating the future.

 

Welcome to my world, the world of public opinion polling.

It’s a world of questions and answers, of samples and surprises. The better the questions and the samples, the better the insight into the world. The more often over time the same good questions are asked of fresh scientific samples, the better the insight into the trends shaping the future.

The central thesis of this book is that global public opinion polling, tracked over time across key countries, can help predict the future – or at least yield clear insights into the likely scenarios within which the future will unfold.

Pollsters and other social scientists are fascinated not only by the views majorities of citizens hold on a wide range of topics but on the processes and influences that change these views over time.

I’m very grateful that I found my way into public opinion research because it satisfies two of the big themes in my life. First, numbers have always spoken to me. I get as much from reading a set of number tables as I do from reading a good book. And secondly, equality has always been important to me – something that I didn’t really understand until I visited my family’s early 1800s share-cropper homestead in Scotland. We Scots do seem to rail on against inequality and arbitrary authority!

So stumbling into polling (after taking maths/physics in university and beginning an issues communication consultancy) has been like coming home for me because statistical research involves numerical measurement of opinion and one of its basic principles is that everyone’s opinion is equally important.

I remember when GlobeScan fielded its first global poll in 1997 – to measure environmental concerns across 30 countries to present at a United Nations environment conference. For a number of weeks I would wake up every morning imagining the thousands of interviews that were taking place with all those people around the world on perhaps the first global issue that was touching all of our lives. I felt honoured to be part of giving voice to their collective views. I still feel the same today.

International polling has also introduced me to a world of interesting and committed colleagues in countries around the globe. In addition to GlobeScan’s hand-picked research partners in each of 55 countries, I’m grateful to all my fellow members of the World Association for Public Opinion Research (WAPOR) for the opportunity to work with and learn from them over the years, both at our annual professional conferences and when I have been able to visit with them in their countries.

One of my earliest memories of my many ‘country visits’ over the years was drinking Turkish coffee till dawn in Istanbul with one of that country’s finest pollsters who regaled me with wonderfully insightful tales of opinion and trends in his fascinating country where Europe meets Asia. You can’t learn about a country more deeply than this.

But what about at the global level; can this level of accuracy and insight be achieved? And what about the bigger question that this book tackles: can the world be wrong? In other words, can comparative polling across multiple countries result in wrong conclusions about where the world is heading? My answer is “Yes it can be wrong, but rarely from my experience.”

Using polling to predict the future can be wrong in at least five possible ways. Polling can variously be deliberately misleading, technically wrong, conceptually flawed or factually incorrect. Polling can also be accurate yet not predictive of actual events. Let’s take these each in turn.

In today’s distrustful world, everyone is treated with skepticism and pollsters are obviously not immune (nor should we be). Some people assume that pollsters practice the dark arts and regularly ask questions in ways designed to get the results desired by their clients or their own political leanings. Unfortunately, this is sometimes the case, and it drags down our profession as surely as a self-interested politician drags down political life.

Some less-than-honourable polling agencies have publicly released survey results designed to be supportive of their (often un-named) clients. I’ve seen examples of such polls on nuclear power, climate change and a raft of political platforms. Extreme examples come from the use of “push polling” techniques where faulty facts and biased question wording are used to skew the results. But this is not the truth about the vast majority of polls or pollsters.

Polling can also be technically flawed and therefore misleading. It may use a weak sample that is not representative of the total population, an inappropriate methodology, or make other technical errors like faulty translation of the questions into local languages and dialects. On-line polls in countries with very low Internet penetration that purport to represent broad public opinion are another example, which can be as misleading as polls sponsored by self-interested organizations. The established research industry has long argued that media organizations should have strict policies of not reporting “junk polls” such as these. Reporting of any poll results that do not include enough information to enable the reader to judge its accuracy and neutrality, deserve to be treated with skepticism.

Thirdly, in spite of our best intentions we pollsters can sometimes get it wrong even with a good sample. A flawed question wording on a complicated topic can produce findings that misrepresent the public view. This can particularly be a problem when we ask too few questions on a topic or form conclusions based on superficial questions that don’t get to the substance of the issue.

A simple question that forces respondents to choose between spending cuts or tax increases to reduce overall government deficits is a good example. Detailed research shows that opinion changes by spending area and most want a combination of the two. A pollster can also confuse respondents with question wording that muddles concepts that are already unclear in the public mind such as deficit and debt, or climate change and ozone depletion. In other cases, local, cultural or religious factors can make global questions irrelevant in certain countries. For example, a question about belief in God is not a salient question in the (humanistic) Buddhist and (multi-deity) Hindu worlds. Polling is like other fields of activity: garbage in, garbage out.

Fourthly, polling can accurately measure perceptions of the public that are factually incorrect. Polls showing that most Europeans see immigration as bad for the economy accurately reveal perceptions that don’t have the added advantage of being true. Economic studies show migration to be strongly positive for economies.

Finally, polling may result in a totally accurate picture of public sentiment that will never be reflected in public policy or eventual societal outcomes because of strong opposition from powerful forces with vested interests. From my 25 years of experience, while power elites can choose to ignore public opinion for a while, strong opinion trends revealed through our tracking research have almost always prevailed in the end, be it in China or in Western democracies.

There are inevitable lags in any governance system, but generally we have found that strong polling trends eventually get manifested in changed government policies and citizen behaviour. This is the reason we are emphasizing tracking research in this book – where the same questions are asked in the same way over a number of years.

Figure 1 shows the power of long-term trend research in telling a story. It shows the evolution of concern about climate change between citizens of the G7 industrialized countries and citizens of the so-called BRIC countries (Brazil, Russia, India, China). Back in 1992, G7 citizens were on average 10% more likely than citizens of BRIC countries to rate climate change as a “very serious” problem. By 2014 the reverse was true, mainly due to a decline in perceived seriousness among G7 citizens.



Polling is both an art and a science. We social researchers definitely get it wrong at times just like anyone else, but generally our industry standards are good and our methods and results are sound.

Most social researchers really want to understand how society is evolving. We follow rigorous sampling and survey methods to ensure the laws of statistics apply. Most reputable polling firms are members and follow the standards laid down by our global professional bodies, the World Association of Public Opinion Research (WAPOR) and the European Society for Opinion and Marketing Research (ESOMAR). Most of us also regularly present our methods and results at professional conferences, opening ourselves to peer review.

Much of the polling that underpins this book was conducted by GlobeScan Incorporated, a company founded (as Environics International) in 1987. We have been fortunate to have built a network of respected local research partners across 55 countries, many of them willing to field global polls as part of their societal contribution. A list of the fine research institutes contributing data to this book is included in Appendix B.

The research findings presented here usually involve over 20,000 interviews across 20 or more countries from six continents, and are statistically representative of a universe of 2.5 billion of the world’s people. Most of the 20 largest economies in the world (the so-called G20 countries) are included in each poll. In each country, the surveys use the same methodologies as those used to predict national elections, being accurate within 3 or 4 percentage points nationally, 19 times out of 20.

The following map shows the countries usually included in GlobeScan’s polls.



GlobeScan and its national research partners use in-home or telephone interviewing. This ensures the key scientific underpinning and therefore legitimacy of polling is respected – that the sample is constructed in a manner that gives everyone in the population being surveyed an equal chance of being selected for the survey. Random digit dialing for telephone research is a good example of this. Fresh samples are drawn each time.

All this methodological rigour, plus the fact that tracking polls ask the same questions in the same way over time, mean that the resulting trends in key opinions are reliable metrics for both understanding the world today and developing likely scenarios for the future. Hence, they form a good basis for making well-informed decisions at both the personal and organizational level.

As mentioned earlier, the crafting of questions is key to the art and science of polling. This is especially true when doing multi-country research in 20-30 languages across many cultures. Simple language, free of cultural bias is key. The neutrality of questions is another key factor – neutrality in both tone and content. In my experience, most social researchers are passionately neutral and really want to understand where society is heading.

Given the importance of the research questions, exact question wording for all the findings presented in this book is provided in Appendix A. So you can judge for yourself. Many of the tracking charts in the book use multi-country averages to keep complexity to reasonable levels; but in the interest of transparency detailed country-by-country results for these questions are given in Appendix C.

At the end of the day, my 25 years in the polling business suggests that many people’s definition of good polling is research that agrees with their prior views on a given subject. You might watch for this yourself so that you’re open to some of the surprising findings in this book.

There is also real opposition among power elites almost everywhere against increasing the influence of the will of the people on the affairs of state. A senior official in one country’s foreign service reacted to one of my research presentations with the admonition, “Surely you’re not suggesting that we shape our foreign policy on public opinion!” I told him they could do far worse. And they have.

But strong trend-lines such as those presented in this book will very likely become manifest in the world eventually, through market and policy changes. There are delays in every system, and major events like 9-11 and the Great Recession can definitely change the course of certain things. But most of the trends presented in this book have been affected very little by even these mega events (as can be seen from the tracking charts).

Over the years, GlobeScan has generated a number of research results that some people couldn’t quite believe at the time but which ended up being very predictive of subsequent events.

  • The huge expectations of companies to act on broader societal issues revealed in our 30-country Millennium Poll in 1999, led to a decade in which companies have had to out-compete each other in demonstrating their corporate social responsibility.
  • Our 1998 poll for a global consumer product company showing American and Chinese citizens similarly wired on environmental topics has been followed by a decade in which the environmental concerns of the Chinese population have actually overtaken those in the West and transformed Chinese government policy into a leading position on renewable energy and sustainability.
  • Our 1999 research and analysis for the UN Environment Programme identifying a surprisingly large Ethical Consumer segment of the global population has been followed by the recession-defying growth in ethical consumerism generally and the Fairtrade brand specifically.

So can the world be wrong? Yes it can, but I would argue polling trends are going to be less wrong than almost every other way of predicting the future. I would strongly advise organizations and individuals that are significantly vulnerable to the trends outlined in this book to at very least develop a planning scenario for the kind of future suggested here.


The Future of Global Polling

The rise of global polling is one of the things that has significantly shaped the first decade-and-a-half of the 21st century; and it promises to grow in importance and influence.

Pioneering efforts in the 1980’s and 90’s by the University of Chicago and University of Michigan have been greatly broadened since the late 1990’s by the Pew Research Center, Gallup International, WorldPublicOpinion.org and by GlobeScan.

At its best, global polling taps the collective wisdom of average citizens the world over. It can be seen as an early form of global democracy, giving voice to a large and increasing percentage of the Earth’s people, using scientific methods.

At its worst, polling can live down to the serious skepticism that some hold for this ‘devil science.’ Mark Twain’s quote at the beginning of this chapter was far from the last to cast dispersion on statistical research. However, the pervasiveness of statistics these days from many sources, and the hunger for a factual basis for understanding our world and for making decisions, has brought survey research into much wider favour. Whether it’s the consumer confidence index, voting intentions, product useage, or views on specific policy options expressed through referenda or polls, scientific survey research based on rigorous samples has become an important source of understanding and an authoritative input to decision-making.

But to live up to this role, global polling must evolve quickly. And it is. 

I predicted five years ago that truly globally-representative polls will be a reality within a decade. Sure enough, the Gallup Organization in the US has recently launched its World Poll claiming annual polling in 140 countries around the world, using telephone or in-person interviews.

I believe we can’t rest until we have established a number of annual surveys of truly representative global samples that accurately reflect the weight of opinion in proportion to their percent of the global population. We also need to develop better ways of presenting global opinion. Currently almost all pollsters use the UN model for weighting global opinion of ‘one nation one vote,’ even though this weights the importance of Chinese and Indian opinion equally with smaller countries like The Netherlands and Colombia. The polling profession needs to develop and agree a more suitable weighting system and then all major polling organizations should follow it.

While more scientific than street protests, polling is equally reliant on the Fourth Estate, media, for its power and impact. Both the polling profession and media organizations have much to do to live up to our responsibilities as the influence of these polls grows and takes on almost a global governance role.

Researchers must do a better job of addressing skepticism towards polling by better educating people on the scientific validity of representative random samples, and finding better ways of portraying and interpreting multi-country results. For its part, the media must develop its internal expertise and policies that can differentiate good polling methodology from bad polls; then do a better job reporting the former and never reporting the latter.

The role of survey research is not limited to public opinion but can also involve stakeholders who are paying close attention to or have knowledge of particular topics, organizations or sets of issues. For example, surveys of climate experts and decision makers, chief executives of companies, or Nobel Prize-winning scientists can tap the views of people that are particularly knowledgeable or engaged in specific issues and are therefore able to go beyond their personal opinion to give a considered judgment on topics based on their professional or life experience. Similarly, the views of citizens living near industrial facilities, or those most affected by a particular government policy have sometimes more relevant and important perspectives than the general public.

GlobeScan has found that our best contributions to important policy debates is by combining the contextural insight from public opinion polls with more detailed findings from surveys of experts and stakeholders. In this book, I draw findings from GlobeScan’s surveys of an authoritative panel of sustainable development experts4 across 60+ countries to illustrate this added value.

I suggest that transparent, quantitative and iterative surveys of particular stakeholders will become increasingly established as part of global governance processes in the Internet age. Transparency International’s “Bribe Payers Index” is a good example, where business executives across the world rate how likely companies from 28 different countries are likely to pay bribes in order to win business.

Another increasingly-surveyed sub-audience is the “Netizen”, an on-line citizen reached using on-line samples and survey techniques. In many countries these surveys cannot be credibly presented as representative of public opinion in the country due to low penetration of the Internet in households. However, on-line polls are relatively inexpensive to do and if done rigorously, can be representative of the on-line community in a given country, or ‘Netizens.’ This can be seen as a surrogate for surveying the more wealthy, educated and informed citizens who often tend to be influential in shaping social norms and political trends. However, in most countries this is not the same as true public opinion.

To demonstrate how far from being representative of the general population and therefore public opinion in most countries we need only look at the percentage penetration of household Internet use in countries across the world. Even amongst the so-called Group of Twenty (G20) largest economies, there are countries where the percentage of the national population that have Internet in their home is too low for on-line surveys to be representative of public opinion.



We must always remember that the key scientific underpinning and therefore legitimacy of polling is to have balanced questions and a sample constructed in a manner that gives everyone in the population being surveyed an equal chance of being selected for the survey.

Having said this, conducting public opinion surveys in countries with high Internet penetration (like the US, Canada, UK, Netherlands, Germany, France, Japan, South Korea, etc.) yield reliable results, especially using Internet panels that have been built in a rigorous manner – for example, by giving computers and Internet connectivity to citizens unable to afford it, so their opinions can be included in proper measure.

With all its blemishes, it can be successfully argued that using strong trends in public opinion across even the G20 countries will still be better than other methods for predicting the future because the sampled populations constitute the most influential portion of what could be called the global “body politic.” 

For a more definitive proof of this, please re-read this book in 20 years time! What seems challenging and unrealistic at a certain point in time can appear prescient a decade or so in the future.



References:

4 This hand-picked panel of 5,000 experts and stakeholders across the world includes SD leaders from business, government, academe, NGOs, think-tanks, consultancies, and the media. About 1,000 of them answer each of our quarterly surveys, now co-managed by GlobeScan and SustainAbility.

5 Source: The World Poll 2011, by Gallup Organization. Results are based on telephone or face-to-face interviews with at least 1,000 adults in each country, aged 15 and older.

* Most Internet connection in India is through mobile telephones.