You Know More than You Think: How to tap the wisdom of the crowd in your head

[one...two...tree 2,762]

By Jack Soll and Richard Larrick

There is an old saying that two heads are better than one. This saying received empirical support in social psychology in the 1920s, when a series of studies showed that groups were more accurate than their individual members. In an early demonstration of the phenomenon, for example, Columbia University’s Hazel Knight asked students to estimate the temperature in a classroom. When the estimates were averaged together, the resulting group answer was more accurate than the estimate of a typical member.

Early authors found this surprising and attributed it to some mysterious group property. Eventually, however, it was recognized as a product of statistics: Using a large sample of imperfect estimates tends to cancel out extreme errors and converge on the truth. Subsequent research in forecasting demonstrated the power of averaging compared to more sophisticated statistical methods of combination. The power and simplicity of averaging was summed up in the title of James Surowiecki’s 2004 best-selling book, “The Wisdom of Crowds.”

In a fascinating new article in Psychological Science, Stefan Herzog and Ralph Hertwig turned the old aphorism on its head: One head can be nearly as good as two. Herzog and Hertwig had participants make estimates about quantitative values they did not know with certainty—specifically, dates in history. They then had participants make second estimates. Could this “crowd in the mind” help improve judgments? The answer is yes, and the literature on the wisdom of crowds helps us understand why.

Crowds, of course, are not always wise. They are more likely to be wise when two principles are followed. The first principle is that groups should be composed of people with knowledge relevant to a topic. The second principle is that the group needs to hold diverse perspectives and bring different knowledge to bear on a topic. Valuing diversity has become a truism, but it is interesting to consider exactly how diversity improves decision making. People inevitably make errors. The question is whether people make similar errors, in which case individuals are interchangeable and there is little benefit gained from a crowd; or whether people make different errors, in which case their errors will often cancel out. Differences in perspective are created both through who is in the group—when people have different experiences, training, and judgment models—and through process—when ideas are formed and expressed independently from the ideas of others. Interestingly, the benefits of diversity are so strong that one can choose group members that differ pretty widely in their ability and still gain—as long as there is added diversity.

Herzog and Hertwig used the insights of the “wisdom of crowd” perspective to make one head nearly as good as two. After participants made their first guesses at the dates of historical events, they then made a second estimate using one of two methods. In one condition, participants simply gave a second estimate. This condition did little to increase either knowledge or diversity.

In the second condition, participants were given detailed directions for making their follow-up guess: “First, assume that your first estimate is off the mark. Second, think about a few reasons why that could be. Which assumptions and considerations could have been wrong? Third, what do these new considerations imply?... Fourth, based on this new perspective, make a second, alternative estimate.” When the participants used the more involved method, the average was significantly more accurate than the first estimate. The “crowd within” achieved about half the accuracy gains that would have been achieved by averaging with a second person.

Herzog and Hertwig called their more involved process “dialectical bootstrapping.” You can pull yourself up by your own proverbial bootstraps by assuming that you are wrong, providing a second estimate based on a search for new evidence, and then averaging the two estimates. (Interestingly, in Herzog and Hertwig's studies, bootstrapping did not lead to second estimates that were more accurate than the first. The benefit of dialectical bootstrapping was only realized when the first and second estimates were averaged together. Compared to simply providing a second judgment, dialectical bootstrapping creates diversity —it leads to estimates that are more likely to have offsetting errors.)

Although we each have the ability to generate multiple perspectives, we typically become enamored of only one way of approaching a problem—with its accompanying embrace of the truth and error. We tend to stop on the first pass. Dialectical bootstrapping allows us to tap more perspectives that are already inside our own heads. A key insight from Herzog and Hertwig’s study is that we each carry around our own crowd, our own diverse board of advisors, but we gain their wisdom only if we run a good board meeting. A good board meeting makes sure that there is diversity at the table and that different perspectives are not just voiced, but incorporated into a final decision. Herzog and Hertwig have designed a process for a good board meeting—in the privacy of your own mind.

http://www.scientificamerican.com/article....hink&page=2

[w¡n9Nµ7 §£@¥€r 123]

Crowds, of course, are not always wise. They are more likely to be wise when two principles are followed. The first principle is that groups should be composed of people with knowledge relevant to a topic. The second principle is that the group needs to hold diverse perspectives and bring different knowledge to bear on a topic.

What you're calling a 'crowd' is simply a group of SME's.

[one...two...tree 2,762]

Crowds, of course, are not always wise. They are more likely to be wise when two principles are followed. The first principle is that groups should be composed of people with knowledge relevant to a topic. The second principle is that the group needs to hold diverse perspectives and bring different knowledge to bear on a topic.

What you're calling a 'crowd' is simply a group of SME's.

SME's?

[w¡n9Nµ7 §£@¥€r 123]

Subject Matter Experts

[one...two...tree 2,762]

Subject Matter Experts

Ohhhhhhhhh... sounds familiar.

[justashooter 430]

so if the diversity you add is ####### you still get #######, which is usually worse than what you started with.

same conclusion i've had from the beginning, but i didn't get paid in tax dollars to do a study on it. just figured it out in my spare time during the last 45 years.

[Ban Hammer 13,364]

Subject Matter Experts

and then there's subject matter experts generating major answers.

[mawilson 2,133]

Prediction markets are a good example of "collective wisdom".