Marc Fawzi at Evolving Trends attacks the whole notion of the wisdom of crowds. It’s a development of the disappointing experience he had when digg suddenly made him the number one site on WordPress for a short period, apparently on the basis that he had come up with a catchy headline. Marc’s issue with digg’s ability to discriminate the best stories was “not because they gave me a short burst of traffic. I didn’t like them the minute I realized that submitting the same idea with a better title (which is entirely true) made it so much more popular!” More on that story below. He concludes that “while a crowd can be a decent calculator of subjective measurable value, it will always produce a dumb choice when it comes to subjective quality” and calls for a return to the old order whereby experienced editors and qualified professionals decided what’s important.
The wisdom of crowds is a book by James Surowiecki which describes how the masses can come up with better solutions to problems than experts. It also became part of Tim O’Reilly’s seminal paper describing the nature of web 2.0.
If it were merely an amplifier, blogging would be uninteresting. But like Wikipedia, blogging harnesses collective intelligence as a kind of filter. What James Suriowecki calls “the wisdom of crowds” comes into play, and much as PageRank produces better results than analysis of any individual document, the collective attention of the blogosphere selects for value.
To summarise what O’Reilly is saying, he feels that the most popular blog postings, or the ones that achieve the best search engine ranking, are also the best ones. Google works by analysing the number of inbound links to a site, using this as a determination of quality. Since (in the best possible world) all of those links were created by individual human beings acting intelligently, the Google algorithm is using the sum of all of our individual judgements to decide how useful a page is.
There are several ways to spam Google, but they do seem to get found out eventually. It’s also questionable how discriminating the majority is. Linkage is not necessarily a foolproof determiner of quality. An article by David Beckham on the existence of God would probably gain a higher rank than one by A J Ayers. In addition, it’s hard for new sites to ‘break in’ to Google. Nobody has links to your site because it didn’t exist before, and you’ll have to compete against the thousands of links to your competitors’ sites before you’ll be able to get a look in.
Is this better than having a human editor? The question doesn’t really arise since there are billions of pages. There are 47 million blogs indexed on Technorati alone, with 75,000 new blogs started every day. The only practical way to organise the information is by machine. Since machines can’t (yet) decide how clever or funny or moving a page is, then involving some human judgement, the number of inbound links, seems to be the best kludge we have available. Google is the number one search engine because people think it produces the best results. If it didn’t, people would go elsewhere. Nobody, not even Google, is saying that their algorithm is perfect, but it is quite clearly ‘good enough for now’, harnessing the wisdom of crowds to produce better search results than human beings alone could possibly produce, no matter how qualified or dedicated.
I’d also like to briefly point out the example of ebay. Getting a high reputation within the e-bay community is such a motivating factor that it drives really high levels of service. Yes, there have been a number of bad experiences there and people who abuse the system, but on the whole, the success of e-bay could not have happened without a community that cared enough to make it successful. It’s very hard to quantify the overall level of service satisfaction on ebay, but it could not continue to exist if it wasn’t ‘high enough’. Again, this is the product of a crowd of individuals making their own decisions and thinking: “If I deliver a high level of service (a) I will make more sales and (b) more people will trust shopping at e-bay. I win from both of those things.” Their collective intelligence has made e-bay a good place to buy things. Like Google, it is not perfect, but it certainly seems ‘good enough’ to fund an enormously successful internet company.
The wisdom of crowds does exist on the internet. These are two examples. The problem comes when the conditions are not correct for that wisdom to happen. For mobs to be smart, they have to all operate individually using only their own agenda and they have to care. Surowiecki gives the example of a lost submarine. It had been several hours since the vessel had lost contact and the possible search area was impossibly massive. The man in charge of the rescue operation, John Craven, solicited opinions from all sorts of people as to where the sub might be: naval officers, salvage experts and mathematicians. Then he drew a map and pinpointed all the suggestions. Then he calculated the median of all their answers — not where the majority thought it would be, but the average. Lo and behold, that’s where the sub lay. The important things to note are that this was a group of people making independent judgements based on their own experience, agenda and training. That’s how a crowd is able to make a wise communal decision.
When internet sites operate on majority rule and the community that runs it is relatively small and it’s very easy to make your impression felt — such as news voting sites, like digg and reddit, or some of the social bookmarking sites — then the results become less accurate. Wisdom loses ground to lobbyists, unintelligent clicking at buzzwords and sheep-like approval of anything other people have said was important. Visitors to such sites see the stories or sites that have already been found popular by previous visitors and this creates a vicious circle whereby only stories reflecting the hard-core of users — the sort of people who spend hours of their day on the sites — ever get promoted near to the top of the list. A new model needs to be found for these sites, in my opinion, that is able to operate by aggregating a host of genuinely individual decisions — the true wisdom of crowds.
Share this post:
Hmmm, thought-provoking! I would agree with the previous comment that there is a hierarchical difference between those that create AND follow links & those that just follow links. However, I would argue that a ‘crowd’ in its most fundamental form also has a hierarchy, albeit in a relatively diffused and chaotic form. Over time, a crowd will develop into a fairly strict hierarchy of personalities.
Throw a bunch of people together and watch what happens over time (e.g. a jury debating a case, school children constructing the rules to a game, football fans discussing their team’s performance in the pub). Some people try to take control by producing, some people take part by consuming and others do very little of either.
So it could be argued that even crowds are subject to certain hierarchical dynamics much as the Google PageRank system is and furthermore that the relative wisdom of that crowd changes as this hierarchical effect matures.
Dave,
Great observation.
Let’s play intellectual ping pong if you’d care to.
I updated the rebuttal on Evolving Trends, with slight clarification, and I happen to agree with you but I have yet to digest its implication on my argument.
This is fun.
Marc
I think we are in more agreement than it might initially seem. The trouble is that ‘crowd’ means different things, and is a pretty emotive word. In the submarine example I cited, the rescue commander Craven didn’t solicit the opinions of just anyone, only people with the training or experience to hold a valid opinion.
However, when it comes to ‘guessing the number of jelly beans in the jar’, anyone is entitled to an opinion. The book isn’t advocating mob-rule and neither am I. Just as juries can’t contain criminals, the insane or people with an axe to grind, wise crowds don’t contain everyone — only people with some sort of valid opinion.
The issue is whether we are making/can make that happen on the internet.
I think the deeper issue you hinted at in your article is the fact that we do need automation of judgment (see my article on the co-evolution of man and machine to get an idea where all that would lead, in the sci-fi predicted/predicated future of ours.)
But this automation of judgment should simply take into consideration the tastemakers’ opinions (i.e. it should take into consideration a non-arbirary hierarchy, i.e. one that represents real difference in the quality of judgment)
Digg does not do that.
Google does a much better job at that.
Marc
I guess the reason Google works (and digg to a lesser extent — I agree — simply because it takes a much smaller sample to exert an influence) is that the majority of people like what the majority of people like. Popular culture is about fashion and being in with the crowd.
Where Google and digg (and all the rest) fall down is when you are the minority. And everyone is in some sort of minority.
I’ll come back tomorrow with an update to all this.
In case of digg it’s more than about the sample size. It’s about the lack of a non-arbitrary hierarchy.
People are not equal in their IQ and ability to judge art, music or literature, so why should they be treated all equally by digg? Sounds like a socialist model to me.
Marc
P.S. Google is off the hook on this one.
This is actually very close to a very old economic question of deciding who gets to decide.
If we are to say that ‘taste makers’ are those that get to vote, how do we then decide who gets to elect the taste makers? Are we to assume that all taste makers are equal?
–bruce
On the internet, and particularly with the rise of Web 2.0, free blogging, cheap access, etc., it seems the taste makers elect themselves. Presumably, only someone with something to say is going to do that, though, which by-and-large is the sort of taste maker we want, isn’t it? Google, therefore, is a wise crowd.
And that is why Web 2.0 as a phenomenon is proving so viral. Those that have ‘a lot to say’ (wise or otherwise) are heavily rewarded both by those with a similar mentality and by those that like merely to listen.
I don’t believe this effect ‘en masse’ creates wisdom. Purely because my own experience has been that many of the wisest of people understand that due to the nature of the universe we live in, there are no absolutes and therefore opinions, even well-versed ones, are just that. It perhaps shouldn’t therefore be ‘the wisdom of crowds’ but instead the ‘power of crowds’.
Ian: “Presumably, only someone with something to say is going to do that, though, which by-and-large is the sort of taste maker we want, isn’t it?”
Allowing someone to elect themselves as the spokesperson for the rest of us isn’t all that hard.…the problem comes when Microsoft, Yahoo, Google, Disney, etc and every blogger with a blog, all want to elect themselves.
So fine, they are all taste makers, now when they disagree who gets the final vote? The crowds of course. Call them wise, call them stupid, call them powerful call them weak, call them whatever you want, but just remember that at the end of the day, the crowds are the ones that decide if you are the next google or not.
That said, on the other side of the coin, and yes, I believe that all economists should come with both hands prepared to make statements, on the other hand, I think it is well within the right of the ‘producers’ to set the rules for the game.
Jury selection, for example, is no small task, and the rules of that ‘game’ have been fine tuned and tweak for centuries. Are the flawless, hardly, do they do pretty good job, yep. I think we can say the same about Google, are the rules that Google uses flawless, hardly, do they do a pretty good job, I think a large number of people vote yes with their browsers everyday.
Digg, may or may not be the right set of rules. Show me a social system and I would be hard presses not to find a way to ‘hack’ it. From airline upgrades, to real estate brokerage discounts, there are few, if any, social systems that don’t have a backdoor built into them.
Does the existence of a favorable hack, discount the validity of a system? Hardly. It’s easy enough for me to pull the fire alarm and get out of a days work, but at some point my employer is gonna figure that out. In a similar way, getting a few clicks from Digg users may or may not be proof that the whole system is de-bunk, or maybe not. One thing for sure, there are more than a few companies taking more notice of Digg and the unique browsers that view that URL seems to be going up.
http://www.alexa.com/data/details/traffic_details?&range=3m&size=medium&compare_sites=&y=t&url=www.digg.com#top
Also, I do think that Digg has a hierarchy; You’l also notice that ‘Diggers’ are not that important in the overall system.
See:
http://blogs.zdnet.com/web2explorer/index.php?p=88
From what I know, it’s not a straight populus vote, and yes, I would expect a number of click throughs to any link listed, as they have a ton of people checking stuff out.
–bruce
I agree with David regarding the fact that there are no absolutes. It follows from Godel’s proof of incompleteness (meta-mathematics) about the very basis for formal theory and proof: mathematics.
Richard Dawkins registered a similar opinion, that reality, logic, science and everything we know is the result of the software that runs our mind (I hope it’s not Web 2.0 software.)
That’s why I generally start the most important statements in my posts with “I believe” since there is nothing else at the end of the day but my personal belief vs that of others.
Given the non-absoluteness of the universe, I believe that opinions that don’t bifurcate can only be right 50% of the time (it’s funny and true.)
The latest thought in String Non-Theory is that the 3rd dimension is an illusion. I in my belief, everything ultimately boils down to an illusion, and it all comes down to ‘poetry.’
Yes, poetry.
No more, no less.
:)
Marc
I have posted some further thoughts on this here http://internet-biz.blogspot.com/2006/07/problem-with-crowds.html
M-theory, now that is getting pretty theoretical right there. The whole SEO and traffic generating phenomenon isn’t recent, but proves that a lot of people are trying hard to market themselves. I do agree that blogs are a form of product and that some producers would like to monetize them at some point.
I find that the whole culture of the web is decentralizing itself from the main stream. Meaning that people are forming cliques that aren’t always thought of. For example, think of a blogger who reads a blog he likes. He adds it to his blogroll, he checks out the blogroll of the blog he reads and so on and so forth.
Marc: “Given the non-absoluteness of the universe, I believe that opinions that don’t bifurcate can only be right 50% of the time (it’s funny and true.)”
Ok, ok, so…basically?
‘All arguments are wrong, therefore any argument is wrong, and since this is an argument, it must be wrong as well.….that is unless you then add sub-argument that states that your argument is wrong, in which case either your original or your sub-argument is more less spot on…’
Hmm, I guess we could bet on Red, Black and Green and be assured that one of those 3 is a winner.…
Not sure where this leads to, but it doesn’t seem very productive in figuring out why both Digg and Google are so popular.…
–bruce
[…] (this post was last updated at 11:30am on July 5, ‘06) Please do see Ian Delaney’s well-written set of counter arguments at TwoPointTouch. I think they compliment this article. My reply to Ian’s argument re: Google’s PageRank being an implementation of the ‘wisdom of crowds’ model is that Google does not let the crowd judge the worthiness of a given link. It let’s the writers, bloggers like Ian, myself, e-zines, news publishers, organizations, etc, i.e. the tastemakers in society (or the producers), judge what it good and what is not. This is distinctly different from letting those who simply consume make the judgment. In the food chain, the producer or tastemaker is above the consumer. That represents a hierarchy which is lacking in a crowd. Thus, the Google model does not rely on the wisdom of the ‘crowd.’ […]
[…] This post is a recycling of my response to a comment made by David Cruickshank on Ian Delaney’s TwoPointTouch. […]
[…] Wisdom 2.0 […]
[…] Well, the wisdom of crowds debate rages on. As Marc quite rightly points out in the comments to my last post, Google rankings depending on in-bound links means that the crowd in question has already qualified itself as a content producer rather than a consumer: it isn’t “the masses”. In the meantime, David points to the rather unwise hierarchy that exists in many crowds such as your class at school. […]
[…] Evolving trends from someone that isn’t Marc Fawzi […]
[…] Does this cross the line into traditional business rather than Web 2.0 business? Maybe. But if I owned one of these companies, that really wouldn’t keep me up at night. Being a Web 2.0 business doesn’t give you a license to run at a loss, I’m afraid to say, so get over it. In any case, as Anthony Mayfield has recently pointed out, many of these ‘wisdom of crowds’ services obey a 1% rule. That is to say, one percent of us make a video, vote for the news, create a blog, while the remainder either comment on it, or digest it as they always did. As Marc Fawzi recently argued against a naive post I made, even Google operates a hierarchy, since only the producers and taste-makers actually produce any links to anything — again, it’s the 1% that are creating PageRanks, not the 99%. When clever web applications harness the intelligence of their users, they’ll only be effective when the intelligence they’re harnessing is up to the job. Everyone has a right to musical taste, so last.fm will work by including everyone. On the other hand, digg voters are, by-and-large, technology enthusiasts, so they’ll produce a front page appealing to tech fans. Fewer people, but the right interests and enthusiasms to work for large numbers of bystanders. […]