Ask OCE — February 24, 2006 — Vol. 1, Issue 5

At a county fair in 1906, a prized bull was offered to the person who could come closest to guessing the animal’s weight. Various “experts” such as cattlemen and butchers made their guesses, as did ordinary folks. An individual did not make the closest guess. The best guess (only a pound off the bull’s true weight) came by averaging all of the guesses in the contest. Thus, the group’s guess was better than any one expert’s. In case after case, from game show audiences to a rescue crew searching for a lost nuclear attach sub, diverse groups are generally more successful at finding solutions to problems than any single expert.

This is the central thesis behind James Surowiecki’s bestseller The Wisdom of Crowds. The author’s contention is that a broad, autonomous crowd, which Surowiecki defines as “any group that acts collectively to make decisions and solve problems,” will invariably make better decisions than expert individuals. A crowd can consist of anything from an informal group to a government agency. Surowiecki cautions against over-reliance on single experts within an organization, explaining that even knowledgeable experts have limited amounts of information on a given subject and biases. Reliance upon experts can lead to a silo mentality that narrows the range of opinions.

Surowiecki identifies four characteristics of wise crowds:

• The crowd must be diverse, which allows the individual members to bring different pieces of relevant information to the table.
• It needs to be decentralized so that no individual member is dictating what the answer should be.
• It needs a mechanism for distilling the group’s opinion into a collective verdict.
• The individual crowd members have to be independent, so they remain true to their own information and perspective while not worrying about what the other members of the group think of them.

Surowiecki explains: “The paradox of the wisdom of crowds is that the best group decisions come from lots of independent individual decisions.”

This independence of opinion is a key element in the building of wise crowds. If a particular group is biased towards or invested in a certain outcome, the result can be disaster. Also when diverse opinions are marginalized, the result can be what the author calls “a dumb crowd.” Dumb crowds make dumb decisions.

Wise crowds tend to be great at solving “cognition problems” — problems that have a right, wrong, or best answer. Surowiecki concedes that crowds are not good at addressing problems of skill. We would never want a group to perform brain surgery collectively.

The wisdom of crowds is clearly part of engineering culture. In the process of developing its technical products, NASA employs wise crowds all the time. The success of the review process throughout a project’s lifecycle — both technical reviews (i.e. PDR, CDR) and Non-Advocacy Reviews (NAR) — depends upon the assembly of wise crowds of outsiders to help evaluate the work of project teams. If one expert alone could do the job of these wise crowds, it’s safe to bet that NASA and others in the aerospace industry would have figured this out long ago.

The key to building wise crowds is avoiding the trap of consensus-seeking in situations where a diversity of opinions will yield better a better result. Think back to the county fair example. A persuasive expert might have swayed members of the crowd to recalibrate their guesses, thereby destroying the independence of thought that was critical to the correct outcome. Seeking the wisdom of a crowd is clearly not appropriate or practical in every situation, but it can lead to better decision-making under the right circumstances.