Sunday, June 17, 2007

The Failure of the "Experts."

If the market is always right, then why are many "experts" saying that it is wrong?

One does not have to read very many blogs and newspapers or listen to very many television pundits to know that many of the so-called experts have been calling for a bear market. To my memory, it seems to me (admittedly anecdotally) that many of these writers and pundits have been calling for this bear market phase since the end of 2006, some for even longer. Despite their dire predictions of a recession, a sub-prime blowup, a housing market meltdown, and high inflation, the market has chugged right along. To read a bit of humor which gets at how frantic some of these experts have become, Bill Rempel wrote the following on his blog:

The Next Phase of Bearish Punditry

"Expect to see this soon, at a bear blog near you:

The government is lying to us, and we are actually in a deflationary recession now. They managed to hide this from us through a clever fakery of job numbers and GDP calculations, depressed gold prices through central bank selling, and hedonic adjustments to the CPI. The entire reason the yield curve de-inverted is because the government is manipulating the rates through sales of long bonds. There will come a point in time when the banks run out of gold and bonds to sell, and the damage to the economy will be too obvious to hide. Then, oh, then, the truth will come out! As soon as those stupid, moronic bulls realize this, the stock market will collapse. Just you wait."

Why the market is smarter than the experts.

If one accepts that the experts are wrong, and the market is right, then it becomes important to understand WHY the market is smarter than the experts. In James Surowiecki's The Wisdom of Crowds, the author elaborates on the conditions that often exist within crowds which allow them to make more accurate predictions and provide better solutions than the so-called experts. Some the examples given in the book are of the jelly bean count (widely replicated--the median guess is usually very close to the actual number of jelly beans) and the estimation of the weight of an ox by a crowd of spectators.

The author asserts that in order for the crowd (and by extension, the markets) to be smarter than experts, the following criteria must be met:

Diversity of opinion: Each person should have private information even if it's just an eccentric interpretation of the known facts.

Independence: People's opinions aren't determined by the opinions of those around them.

Decentralization: People are able to specialize and draw on local knowledge.

Aggregation: Some mechanism exists for turning private judgments into a collective decision

Traders need to ascertain if the criteria are currently being fulfilled in the market. If they are not, there may be a failure of crowd intelligence, and the market may be going through a bubble period, or may be experiencing a period of irrational pessimism. For example, during the 1999-2000 bull run in the Nasdaq, the average barber and bartender became stock market “experts.” All independence was lost as nearly everyone jumped aboard the technology train, heading to instant riches. As independence was lost, so went diversity of opinion, until POP! the bubble burst.

The market in June of 2007 seems to meet all the conditions necessary to be smarter than the “experts.” The current high levels of short interest function as a proxy for Diversity of Opinion. Independence seems to be present as there does not seem to be a glut of average Joes entering the market (Think about the Shanghai market as an extreme lack of Independence) The explosion of Web 2.0 and proliferation of bloggers point to Decentralization of knowledge, and Aggregation (which the markets do all the time) is evident with the availability of a range of brokerage options, from full service to discount. Almost anyone can now participate in the markets.

Why are the experts not as smart as the market?

This issue needs a blog post of its own, but I will lay out some simple ideas about why the experts fail.

Experts typically suffer from a lack of cognitive diversity. Simply put, they operate within a narrow framework of thought. One person simply cannot aggregate all varieties of data, opinion, research, and experience as efficiently and effectively as the market does.

Experts also suffer from a lack of humility. This overconfidence in themselves creates a plethora of biases which diminish their capacity to receive and aggregate information which is counter to their beliefs.

Finally, information cascade can result as the experts seek the opinion of other experts (usually they seek out experts who have the same beliefs as they do.) Typically, if they are presented with information that is congruent with their own beliefs, it confirms what they thought (their beliefs are correct), and if the information from other experts is not congruent with their beliefs, it is dismissed as being incorrect.

If they are often wrong, why are they still considered "experts?"

I believe that what we are currently witnessing in regards to many experts calling for a bear market and literally being wrong month after month is simply an example of survivorship. We do not see all the writers, pundits, and analysts who were consistently wrong, as they are now out of work, or have moved to a new career, or have changed their opinions. What we are left with are those who have not yet been fired, or humiliated enough.

Finally, what the wisdom of crowds shows is that even though many of the experts are consistently wrong, as traders, we still need to consider their points of view. We must aggregate all available data, consider other points of view, remain humble, and above all, never consider ourselves to be experts.


L.J. said...

Great post woodshedder. A lot of the so-called market experts spend too much time thinking and talking about what the market should be doing rather than using their expertise to make money based on what the market actually IS doing.

Stock market punditry has fallen almost as far as political punditry where any clown who was ever a low level staffer and is able to pull things from his arse gets to bloviate on cable TV. Whether or not the person is ever right about anything has no bearing on whether they get to be labeled an expert and continue to get airtime. It's just silly.

MyFriendFate said...

Interesting post.

I've learned that right and wrong is a worthless, ambiguous concept which is totally extraneous to the real goal, which is making money.

Besides, the market is neither right nor wrong, it just is.

Jeff said...

iio, I agree the point is to make money.

And that, as I see it, is the most significant problem with the vast majority of the perma-bear pundits. We have no way to verify any profits, as the vast majority do not post any audited returns. In fact, the ones at ticker-sense will not even own up to their sentiment. They vote, but they won't let the public know which way they voted. They can, month after month, make predictions, yet we never see the true result: can one be a perma-bear during a multi-year bull market AND make money?

I have a feeling the semantics of "wrong" and "right" are causing difficulties for you. Sure, if a trader is trading for himself, thinking in terms of wrong or right may be counter-productive.

However, when one has paying clients, or is paid to espouse a sentiment, or presents himself as an expert, then there absolutely is a wrong and a right. I guarantee you if someone was managing your money, and you never made any gains, or worse, lost some, you would have no problem saying his strategy of being a bear during a multi-year bull market was wrong. How about a newsletter? Would you continue to subscribe if the picks consistently lost money?

As for the market not being right or wrong, I disagree. If you subscribe to the idea that crowds are smarter than the individuals making up the crowds, then it is easy to make the leap to say that sometimes the crowds will get it wrong. Knowing how to identify a market that is getting it wrong or right based on the qualities of the crowd participating in the market is probably an exploitable edge.

MyFriendFate said...

I understand what you are saying. And I agree its natural to think that way. It helps put things in perspective. You can call it semantics, but to me its a little more philosophical. Its training your mind to think in terms other than black and white. You could say that I am a permabear. I think in 7 years, the market will be lower than it is today. But even if I am right on that prediction, its hard to translate it into profits. So, instead I try and keep opinions like that at a safe distance in the back of my brain where they don't do too much damage.

On another note, whats up with Stockalicious? They sure do go down alot. I need to set my blog up like yours, so if the widget doesn't load, I can still read the blog.

Jeff said...

iio- I just re-read my comment to you, and it seems aggressive. Sorry about that. I did not mean it that way in the slightest bit.

You hit the nail on the head- you do not let your own biases get in the way of making money. And the major difference between you and other bears is that you make money during a bull market, because you are still willing to go long, and take on some risk. I visit your blog regularly, and you will do very well when you move from paper to real.

I think that when one arrives at "expert" status, either in their own mind, or the minds of their followers, that they suddenly mistake their opinions for something more valuable. It is that struggle to stay humble, in many facets of life, trading included, that is often very difficult; yet to me, it is one of the easiest ways to stay happy and healthy and profitable.

I have no idea about stockalicious. It is getting annoying. I am checking out covestor. I may use their widget if I like their product.

MyFriendFate said...

That was very well put and I couldn't agree more about the importance of humility (unless you happen to be the fly -- for him it seems like high returns and cockiness go hand in hand!)