The Wisdom of Crowds

This afternoon a friend emailed me a copy of a speech given back in 2004 by Roger Kerr on The Wisdom of Crowds (pdf). In the speech Kerr gives two nice examples of how crowds can be right. One is about the space shuttle Challenger,

In 1986, the American space shuttle Challenger exploded soon after it was launched, with the loss of all its crew. Within minutes, the stock market reacted: the shares of the four companies that had most to do with the construction of the shuttle had all lost value. But one of them, Thiokol, the manufacturer of the solid-fuel booster rocket, lost more ground than the others. Moreover, the next day, when the shares of the other three companies began to recover, Thiokol’s continued to fall. Six months later, the official enquiry confirmed what the stock market (but no one else) apparently knew from the start: that the seals on the booster rocket were defective and had caused the explosion. An academic study of the unique, almost immediate and – as it happened – well-founded collapse of Thiokol’s stock turned up no evidence to support likely explanations, such as media speculation or Wall Street hype. In the end, the researchers concluded that there must have been some insider dealing, although they couldn’t prove it.

Another explanation is the wisdom of crowds.

Kerr's other example is from Who Wants to be a Millionaire?

In the television quiz show Who Wants to be a Millionaire? contestants stumped by a question can enlist the aid of friends or relatives chosen on the basis that they think they are smarter or better informed than themselves. They can also opt to ask the studio audience. The ‘experts’ turned out to be right 65 percent of the time. But the members of the audience – supplying answers anonymously and independently of one another – were collectively right 91 percent of the time. If you ever appear on that show, you may do better to rely on the crowd than on your smart friends!

Later in the speech Kerr considers the then radical idea of policy analysis markets.

This idea seems to have grown out of the Iowa Electronic Markets (IEM) project that was set up in 1988 as a competitive alternative to opinion polls. The IEM is simply a market for betting on the outcome of elections: any member of the public can “buy and sell futures ‘contracts’ based on how they think a given candidate will do in an upcoming election”(p 17). The interesting thing is that between 1988 and 2000 the IEM generally outperformed opinion polls, even though only a few hundred people participated in them. Note the difference between a betting market and an opinion poll. In the betting market, people are making bets on how other people will vote and backing their judgment with their own money. In a poll, people say how they themselves intend to vote, but there are no sanctions if they don’t give a truthful answer or if they change their minds later. The incentive to get a forecast right is obviously much greater in a
betting market.

Some private corporations have tried to tap the wisdom of their employees by setting up ‘decision markets’, based on the same principle as the IEM, on the likely success of their new products. Those cited by Surowiecki – Hewlett-Packard’s on new printer sales, and Innocentive’s on which new drugs were likely to win approval from the US Food and Drug Administration – have been remarkably accurate. How much easier for humble employees to contribute their knowledge, and to gain from doing so, through an anonymous market rather than through some consultation committee, with its built-in biases, managers unwilling to register inconvenient information, and employees afraid of the consequences of speaking up.

A leading advocate of decision markets is Robin Hanson, of George Mason University in the United States. In an article written in 1999, he says:

… we suffer from a serious failure to share information. The so-called Information Revolution has greatly improved our ability to find out what others have said. However, it has done much less to improve our ability to find out what other people know … Speculative markets are a neglected way to help us find out what people know. Such markets … have many advantages over standard institutions for information aggregation, such as news media, peer review, trials, and opinion polls. Speculative markets are decentralised and relatively egalitarian, and can offer direct, concise, timely, and precise estimates in answer to questions we pose.

Just as self-interest motivates us to influence the market price of a good by buying it or not buying it, so the gains from betting can motivate us to contribute accurate and relevant information to a collective decision, and more reliably so than through mechanisms where no such incentives exist.

Kerr then asks, How then would a policy analysis market work? My answer, go to iPredict and find out. Kerr continues

The term comes from one such market that was set up by the Pentagon in 2003 to provide intelligence on the Middle East. The general public would be invited to make bets on possible events in the region, with a view to allowing policy-makers to bypass the vested interests of intelligence agencies in their pet theories and analyses and get some new, informal but reliable information into the policy-making arena. But it was not to be. Congressional pressure forced the Pentagon to scrap the idea on the grounds that it was morally wrong to encourage people to profit from bets on terrorist outrages, even though it was not on that sort of question that the policy market was expected to focus. Yet would we not approve if a government agent profited in career terms as a result of accurately forecasting the attacks of 9/11?

Later Kerr says

It’s interesting to speculate on how policy analysis markets could be developed to cover the entire field of policy. A private organisation – a think tank, a newspaper, a television or radio company, even a public opinion polling company – could set up a betting market on, say, whether our current Labour-led government will, on present policies, achieve its ‘top priority’ objective of returning New Zealand to the top half of the Organisation for Economic Cooperation and Development (OECD) income rankings by some specified date. The qualification ‘on present policies’ would be necessary because the government could react to a betting market forecast of failure by changing its policies to ones more likely to succeed. And that is exactly how such a market could help improve public policy. It could be expected to be more reliable, and hence more heeded, than opinion polls, which are vulnerable to politicians’ growing skill in manipulating voter apathy, ignorance, and cynicism.

I can only hope that Roger Kerr is one of the many now putting their money where their mouths (and wisdom?) are by trading in the markets of iPredict. And helping economic research in the process.