The Limits of Crowd Wisdom

Jaron Lanier in You Are Not A Gadget commenting on the limits of crowd wisdom:

There are certain types of answers that ought not be provided by an individual. When a government bureaucrat sets a price, for instance, the result is often inferior to the answer that would come from a reasonably informed collective that is reasonably free of manipulation or runaway internal resonances. But when a collective designs a product, you get design by committee, which is a derogatory expression for a reason.

Collectives can be just as stupid as any individual, and in important cases, stupider. The interesting question is whether it’s possible to map out where the one is smarter than the many.

(back to the book: )

Scientific communities likewise achieve quality through a cooperative process that includes checks and balances, and ultimately rests on a foundation of goodwill and “blind” elitism — blind in the sense that ideally anyone can gain entry, but only on the basis of a meritocracy. The tenure system and many other aspects of the academy are designed to support the idea that individual scholars matter, not just the process or the collective.

Yes, there have been plenty of scandals in government, the academy and in the press. No mechanism is perfect, but still here we are, having benefited from all of these institutions. There certainly have been plenty of bad reporters, self-deluded academic scientists, incompetent bureaucrats, and so on. Can the hive mind help keep them in check? The answer provided by experiments in the pre-Internet world is “yes,” but only provided some signal processing is placed in the loop.

…

The “wisdom of crowds” effect should be thought of as a tool. The value of a tool is its usefulness in accomplishing a task. The point should never be the glorification of the tool. Unfortunately, simplistic free market ideologues and noospherians tend to reinforce one another’s unjustified sentimentalities about their chosen tools.

Since the internet makes crowds more accessible, it would be beneficial to have a wide-ranging, clear set of rules explaining when the wisdom of crowds is likely to produce meaningful results. Suroweicki proposes four principles in his book, (The Wisdom of Crowds), framed from the perspective of the interior dynamics of the crowd. He suggests there should be limits on the ability of members of the crowd to see how others are about to decide on a question, in order to preserve independence and avoid mob behavior. Among other safeguards, I would add that a crowd should never be allowed to frame its own questions, and its answers should never be more complicated than a single number or multiple choice answer.

More recently, Nassim Taleb has argued that applications of statistics, such as crown wisdom schemes, should be divided into four quadrants. He defines the dangerous “Fourth Quadrant” as comprising problems that have both complex outcomes and unknown distributions of outcomes. He suggests making that quadrant taboo for crowds.