Saturday, November 15, 2008

If your stockbroker calls you a rat, consider yourself flattered.

From Jonah Lehrer's blog comes an oh-so-timely reminder of the human mind's uncanny ability to fall prey to apophenia -- seeing patterns or connections in random data. You can read the full post here, or just settle for a few highlights below:

Look, for example, at this elegant little experiment. A rat was put in a T-shaped maze with a few morsels of food placed on either the far right or left side of the enclosure. The placement of the food is randomly determined, but the dice is rigged: over the long run, the food was placed on the left side sixty per cent of the time. How did the rat respond? It quickly realized that the left side was more rewarding. As a result, it always went to the left, which resulted in a sixty percent success rate. The rat didn't strive for perfection. It didn't search for a Unified Theory of the T-shaped maze, or try to decipher the disorder. Instead, it accepted the inherent uncertainty of the reward and learned to settle for the best possible alternative.

The experiment was then repeated with Yale undergraduates. Unlike the rat, their swollen brains stubbornly searched for the elusive pattern that determined the placement of the reward. They made predictions and then tried to learn from their prediction errors. The problem was that there was nothing to predict: the randomness was real. Because the students refused to settle for a 60 percent success rate, they ended up with a 52 percent success rate. Although most of the students were convinced they were making progress towards identifying the underlying algorithm, they were actually being outsmarted by a rat.

So don't listen to those talking heads telling you why the market rose or fell. They're just like those Yalies, convinced they've found a pattern where none exists.

No comments: