The Failure of the "Experts."

If the market is always right, then why are many "experts" saying that it is wrong?

One does not have to read very many blogs and newspapers or listen to very many television pundits to know that many of the so-called experts have been calling for a bear market. To my memory, it seems to me (admittedly anecdotally) that many of these writers and pundits have been calling for this bear market phase since the end of 2006, some for even longer. Despite their dire predictions of a recession, a sub-prime blowup, a housing market meltdown, and high inflation, the market has chugged right along. To read a bit of humor which gets at how frantic some of these experts have become, Bill Rempel wrote the following on his blog:

The Next Phase of Bearish Punditry

"Expect to see this soon, at a bear blog near you:

The government is lying to us, and we are actually in a deflationary recession now. They managed to hide this from us through a clever fakery of job numbers and GDP calculations, depressed gold prices through central bank selling, and hedonic adjustments to the CPI. The entire reason the yield curve de-inverted is because the government is manipulating the rates through sales of long bonds. There will come a point in time when the banks run out of gold and bonds to sell, and the damage to the economy will be too obvious to hide. Then, oh, then, the truth will come out! As soon as those stupid, moronic bulls realize this, the stock market will collapse. Just you wait."

Why the market is smarter than the experts.

If one accepts that the experts are wrong, and the market is right, then it becomes important to understand WHY the market is smarter than the experts. In James Surowiecki's The Wisdom of Crowds, the author elaborates on the conditions that often exist within crowds which allow them to make more accurate predictions and provide better solutions than the so-called experts. Some the examples given in the book are of the jelly bean count (widely replicated--the median guess is usually very close to the actual number of jelly beans) and the estimation of the weight of an ox by a crowd of spectators.

The author asserts that in order for the crowd (and by extension, the markets) to be smarter than experts, the following criteria must be met:

Diversity of opinion: Each person should have private information even if it's just an eccentric interpretation of the known facts.

Independence: People's opinions aren't determined by the opinions of those around them.

Decentralization: People are able to specialize and draw on local knowledge.

Aggregation: Some mechanism exists for turning private judgments into a collective decision

Traders need to ascertain if the criteria are currently being fulfilled in the market. If they are not, there may be a failure of crowd intelligence, and the market may be going through a bubble period, or may be experiencing a period of irrational pessimism. For example, during the 1999-2000 bull run in the Nasdaq, the average barber and bartender became stock market “experts.” All independence was lost as nearly everyone jumped aboard the technology train, heading to instant riches. As independence was lost, so went diversity of opinion, until POP! the bubble burst.

The market in June of 2007 seems to meet all the conditions necessary to be smarter than the “experts.” The current high levels of short interest function as a proxy for Diversity of Opinion. Independence seems to be present as there does not seem to be a glut of average Joes entering the market (Think about the Shanghai market as an extreme lack of Independence) The explosion of Web 2.0 and proliferation of bloggers point to Decentralization of knowledge, and Aggregation (which the markets do all the time) is evident with the availability of a range of brokerage options, from full service to discount. Almost anyone can now participate in the markets.

Why are the experts not as smart as the market?

This issue needs a blog post of its own, but I will lay out some simple ideas about why the experts fail.

Experts typically suffer from a lack of cognitive diversity. Simply put, they operate within a narrow framework of thought. One person simply cannot aggregate all varieties of data, opinion, research, and experience as efficiently and effectively as the market does.

Experts also suffer from a lack of humility. This overconfidence in themselves creates a plethora of biases which diminish their capacity to receive and aggregate information which is counter to their beliefs.

Finally, information cascade can result as the experts seek the opinion of other experts (usually they seek out experts who have the same beliefs as they do.) Typically, if they are presented with information that is congruent with their own beliefs, it confirms what they thought (their beliefs are correct), and if the information from other experts is not congruent with their beliefs, it is dismissed as being incorrect.

If they are often wrong, why are they still considered "experts?"

I believe that what we are currently witnessing in regards to many experts calling for a bear market and literally being wrong month after month is simply an example of survivorship. We do not see all the writers, pundits, and analysts who were consistently wrong, as they are now out of work, or have moved to a new career, or have changed their opinions. What we are left with are those who have not yet been fired, or humiliated enough.

Finally, what the wisdom of crowds shows is that even though many of the experts are consistently wrong, as traders, we still need to consider their points of view. We must aggregate all available data, consider other points of view, remain humble, and above all, never consider ourselves to be experts.