Three groups of researchers claim to have untangled the process by which many Wikipedia entries achieve their impressive accuracy (1, 2, 3). They say that the best articles are those that are highly edited by many different contributors.
[…] The main lesson for tapping effectively into the ‘wisdom of crowds’, then, is that the crowd should be diverse. In fact, in 2004 Lu Hong and Scott Page of the University of Michigan in Ann Arbor showed that a problem-solving team selected at random from a diverse collection of individuals will usually perform better than a team made up of those who individually perform best — because the latter tend to be too similar, and so draw on too narrow a range of options (5). For crowds, wisdom depends on variety.
The story covers at least five different studies of Wikipedia accuracy and community structure. I’m not about to choose which paragraphs to excerpt and which to leave out, so go read the whole thing. Some studies raised my eyebrows, for example one group’s apparently circular decision to measure the quality of an entry based on the Wiki community’s internal ratings. But overall the article is well worth the time.
To me the most interesting point is the way that Wikipedia turns the old rule that people are smart, crowds are stupid entirely on its head. Wiki entries, which you can think of as a form of extended conversation or debate, only get more accurate the more people jump in, and diversity seems as important as total numbers.
Bloggers ought to pay attention to this. I would even extend the point to say that a diverse commentariat should be a good measure of accurate writing, although it presents a chicken and egg problem in that increasing the number of readers who will howl if you screw up one way or the other often makes writers more careful to get things right, or stick to verifiable facts. In my view that ought to to count strongly in favor of blogs like Obsidian Wings which take considerable care to maintain a multipartisan community.
What distinguishes internet communication from crowd behavior? For collective action like bulk emailings and phone campaigns, not much. People can subsume their will to a collective just as easily online as in a noisy mob. But in Wikipedia as in blogging silent agreement is death. We’re less like a mob than an enormous coffeehouse full of well-read people, or at least well-Googled, arguing with each other. The main effect of technology is to streamline that argument and facilitate communication, cross-referencing and fact-checking on an unprecedented scale.
Testing the converse principle, intellectual monocultures like LGF and the Office of Special Plans ought to, and do, put out unreliable crap. Surprise, filtering out the noisiest critics also shuts down your most enthusiastic fact-checkers.
While we’re talking about Stephen Colbert, note the very next story at Nature News:
South Africa expected to propose elephant cull
Public consultation could suggest killing to control population.