Investors are notorious for chasing performance. If a mutual fund or advisor or trading strategy has done well recently, chances are much greater that traders will commit money to that strategy or product, often independently of the long term performance, general suitability, or distinguishing features of the strategy or product.  I’ve seen the same behavior among the audience for our paid newsletters: after a winning month, new subscribers are more likely to rush in, and if we have a flat or down month, interest from new readers drops. This is exactly the kind of backwards thinking that dooms most investors to underperform even basic market benchmarks: most investors would literally be better off allocating every cent to a plain vanilla index fund, rather than jumping around from one strategy to the next like insects in the lighting section of a hardware store. I get frustrated on behalf of smaller and newer traders in particular, because while they tend to have low risk tolerance and tend to face higher transaction costs – i.e., they’re the group who can least afford to chase performance – they’re also the most likely to do exactly that. You don’t see smart, profitable institutions switching from following commodity trends to selling volatility to trading fixed income every time one of those asset classes has a nice run.

Read the full post here

People nevertheless use the stockmarket as a barometer of economic health. So a rise in equity markets can be (and has been) seen by governments and central bankers as evidence that the economy is headed in the right direction. That can lead to policy mistakes, such as a lax monetary stance, and further irrational exuberance.

Housing is more complicated than the stockmarket since people get utility from their homes (shelter, relaxation) while simultaneously treating them as assets. Even so, a rise in house prices that outpaces GDP growth does not make a society richer. Instead, all that is achieved is a transfer of wealth from first-time buyers to retirees exiting the property market.

In theory house prices can rise faster than GDP for a while if citizens decide to devote more of their incomes to housing services (for example, they may prefer a bigger flat to a bigger car). In practice it is hard to disentangle such structural shifts from the speculation that is prominent in all property booms.

It is the link between speculation and asset prices that explains this crisis. The ability to borrow money to buy assets fuelled the rise in asset prices. And the wealth effect of higher prices persuaded those in English-speaking countries to borrow money to sustain consumption.

Not long ago the BBC transmitted a programme about credit-card use. One man said he felt “wealthier” because he was given a credit-card limit of £5,000 ($8,000). Of course, once he used the card he was poorer. Not only did he have to repay the £5,000, but he had to service a double-digit interest rate as well. Similarly those who buy an overvalued asset with borrowed money have not made themselves richer but poorer.

Read the full article here

The widespread failure of economists to forecast the financial crisis that erupted last year has much to do with faulty models. This lack of sound models meant that economic policymakers and central bankers received no warning of what was to come.

As George Akerlof and I argue in our recent book Animal Spirits, the current financial crisis was driven by speculative bubbles in the housing market, the stock market, energy and other commodities markets. Bubbles are caused by feedback loops: rising speculative prices encourage optimism, which encourages more buying and hence further speculative price increases — until the crash comes.

You won’t find the word “bubble,” however, in most economics treatises or textbooks. Likewise, a search of working papers produced by central banks and economics departments in recent years yields few instances of “bubbles” even being mentioned. Indeed, the idea that bubbles exist has become so disreputable in much of the economics and finance profession that bringing them up in an economics seminar is like bringing up astrology to a group of astronomers.

The fundamental problem is that a generation of mainstream macroeconomic theorists has come to accept a theory that has an error at its very core — the axiom that people are fully rational. As the statistician Leonard “Jimmie” Savage showed in 1954, if people follow certain axioms of rationality, they must behave as if they knew all the probabilities and did all the appropriate calculations.

So economists assume that people do indeed use all publicly available information and know, or behave as if they know, the probabilities of all conceivable future events. They are not influenced by anything but the facts, and probabilities are taken as facts. They update these probabilities as soon as new information becomes available and so any change in their behavior must be attributable to their rational response to genuinely new information. If economic actors are always rational, then no bubbles — irrational market responses — are allowed.

Abundant psychological evidence, however, has now shown that people do not satisfy Savage’s axioms of rationality.

Read the full article here

Anchoring
Anchoring is the act of latching on to a given piece of information and using that as a point of reference for making decisions. Unfortunately, many investors anchor on things that are irrelevant to a business’s value, such as their own personal cost basis in a given stock or the 52-week trading high. Rather, we should focus on the thing that matters the most, the estimated future cash flow of a company.

Availability Bias
This mental shortcut concerns the relative importance of information. The importance our minds attach to information is correlated to how often we see the information. If we see and think about something often, our brains attach greater importance to it.

Endowment Effect
People place a higher value on things that they already own than things they do not own. Meaning, we would sell our possessions at a much higher price than at which we would buy the very same possessions if we did not already own them.

Sunk Cost Aversion
Sunk costs are costs that cannot be recovered once incurred. Once something is paid for in either time or money, our instinct is that we must soldier on and get some benefit for the expense, lest we feel like we are wasting resources. This is a variation of loss aversion, which is a concept that says people feel the pain of a loss at double the magnitude they feel the pleasure of a gain of the same amount. Two tips here. First, if a stock is clearly worth far less than what we originally paid for it, we should be willing to sell if today’s price is above our estimate of current value; that we are realizing a loss should be irrelevant to the decision. Second, if we spend several hours to research a given opportunity, we should still be willing to walk away. Our instinct will be to like the opportunity since we just spent time on it, but our goal should be to have rationality outweigh instinct.

Herd Behavior
Our deepest instincts tell us that there is safety in numbers. Beyond having a desire to do what is perceived to be socially acceptable, we often believe others have useful information from which we can take cues. After all, we all like to be liked, and the bigger group may know something we don’t. Simply, if “everybody’s doing it,” we feel the pressure to take that same action, whatever it may be. Plus, with investing being an activity where having incomplete information is the norm, this instinct to take cues from others can be amplified.

Recency Bias
We live in the here and now, and the ability to contemplate things far in the past and/or future is a uniquely human ability that requires higher cognitive functions. Yet our instincts can still get the better of us on occasion. Recency is the tendency to weigh recent events much more heavily into our decision-making than more distant events. It is a sort of mental short-sightedness where we think much more about our current situation than the much broader historical perspective. This can cause us to assume that the current state of the world–good or bad–persists into the future, rather than reverting to a long-run mean.

Confirmation Bias
Our brains inherently do not like conflict; they prefer to have a consistent, harmonious view of the world. They are wired to avoid cognitive dissonance–having two different ideas that are incompatible with each other. Our instinct is to search out information that confirms our existing views, accepting data that plug neatly into our preconceived biases, while rejecting data that do not support what we already think. Information that is consistent is processed more easily and does not increase stress.

Overconfidence
It’s an unfortunate fact that people tend to believe that their skill level is much higher than what it is in reality. For instance, the vast majority of drivers believe their driving ability is above average, even though this is statistically impossible. Unfortunately, Lake Wobegon is but fiction, and there is not a place where “all the women are strong, all the men are good-looking, and all the children are above average.”

This positive illusion we carry about ourselves allows us to be, as the famous book is titled, “Fooled by Randomness,” and attribute positive outcomes to our personal skills rather than luck or a trend over which we really had no control. Overconfidence can help us get through the stresses of our lives, but it can be deadly in the world of finance by causing one to overplay his or her hand.

Read the full article to learn more about each emotion

Related: Why Smart People Make Big Money Mistakes And How To Correct Them: Lessons From The New Science Of Behavioral Economics

Is The Market Rational? No, say the experts. But neither are you–so don’t go thinking you can outsmart it.
By Justin Fox
December 9, 2002

Buy and hold. Diversify. Put your money in index funds. Pay attention to the one thing you can control–costs–and keep them as low as possible. Today that is pretty standard, if often unheeded, investment advice. Forty years ago it was revolutionary. The revolution started on college campuses, in particular at the University of Chicago, and it went by the unrevolutionary-sounding name “efficient markets.”

“In an efficient market,” wrote Chicago professor Eugene Fama in a landmark paper he delivered at the 1969 annual meeting of the American Finance Association, “prices ‘fully reflect’ available information.” That is, in an efficient market you can’t beat the market unless you have inside information. So why bother trying?

That logic led, among other things, to the creation of index funds that aim to mimic, not beat, the likes of the S&P 500 and the Wilshire 5000. Today such funds account for about 10% of total U.S. stock market capitalization, as well as 60% of what little money has flowed into equity mutual funds so far this year. But millions of small investors have continued to ignore the advice derived from efficient-markets theory, preferring instead to trade stocks and pile in and out of mutual funds in search of elusive market-beating returns (blowing much of their money on fees and commissions in the process).

Meanwhile, back on campus, a new generation of finance professors has been ripping Fama’s teachings to shreds. The organizing principle for this new breed of scholars is not efficient markets but something called behavioral finance. Behavioral finance teaches that stock market investors are irrational, that future stock price movements are at least partly predictable from past behavior, and that careful analysis of past trends and financial reports can pay off. Which happens to be the way most investors see the market already.

Over the next few pages we’re going to take you on a journey through the academic battles that have brought us to this point. This is FORTUNE’s annual Investor Guide, not The Chronicle of Higher Education, so we wouldn’t tell this story if we didn’t think it had relevance for investors battered by the experience of the past few years. The message that the behavioral finance guys have for investors is that yes, you can beat the market, but–for reasons that are essential to the whole behavioralist case–you almost certainly won’t. As a result, they end up offering much of the same investment advice that the efficient markets folks do. Only this time we might actually listen.

Gene Fama, it must be said, doesn’t buy any of this. It’s a cold, gray mid-autumn day in Chicago, but Fama is wearing a loud shirt with too-short sleeves that prove him to be probably the buffest 63-year-old finance professor on the planet. With the hint of a Boston accent that remains after two-thirds of a lifetime in Chicago, he’s talking the academic version of trash. “I don’t know that it’s progressed much beyond the level of curiosity items,” Fama says of the research done by the behavioralists. “They’ve got lots of interesting curiosity items.”

For years Fama was more than tolerant of this search for “curiosity items.” Unlike his longtime Chicago colleague Merton Miller, who saw the behavioralists’ work as an ideological assault on free markets (Miller died in 2000), Fama always encouraged empirical research, even research that revealed seemingly inefficient market behavior. In a 1991 sequel to his famous “Efficient Capital Markets” paper of 1969, he acknowledged that reality had in fact turned out a lot messier than he and other efficient-markets theorists had envisioned–although not so messy that the theories couldn’t accommodate it.

In 1997, though, Fama wrote still another paper, one that argued that a lot of the purported market anomalies discovered by the behavioralists were due to bad statistical work, and that the behavioralists’ attempts at building a theoretical alternative to the efficient-markets hypothesis had so far been “embarrassing.” The paper was something of a sensation–until interest in the D.C.-area sniper sent a piece called “Multiple-Victim Public Shootings” to the top of the charts in October, it was the most downloaded work in the almost six-year history of the Social Science Research Network, a leading academic website. Fama had no problem with behavioral finance as a critique of efficient-markets theory. But he thinks it’s a disaster as a replacement. “I don’t know what asset pricing would look like in a world that really took behavioral finance seriously,” he says. “If you really think prices are incorrect, what are you going to tell me about the cost of capital?”

We’ll get back to that question. But first let us document the behavioralists’ triumph. Half of this year’s economics Nobel went to their patron saint, Princeton psychologist Daniel Kahneman (the other half went to Vernon Smith of George Mason, whose economic experiments have also shot holes in efficient-markets dogma). Then there’s the John Bates Clark Medal, awarded by the American Economic Association every two years to the most important U.S. economist under 40: The 1999 and 2001 editions both went to behavioralists. On the pop-culture front, Yale efficient-markets skeptic Robert Shiller’s 2000 bestseller Irrational Exuberance was the most talked-about book by an economist in years.

The most dramatic development of all, though, may be that the office directly below Fama’s at Chicago’s Graduate School of Business now belongs to behavioralist pioneer Richard Thaler, 57. A magazine profile last year characterized Thaler, to the undying amusement of his students, as “thick-set,” but that’s not quite fair. He is not the jock that his upstairs neighbor is–Fama beats him at tennis. But Thaler, who arrived in Chicago in 1995 after years in the relative academic wilderness of Cornell University, appears to have eclipsed Fama as the most influential faculty member at the business school that has had more influence on the study of finance than any other.

For decades the University of Chicago was the temple for those who believed that markets always got things right. Thaler’s intellectual journey, in fact, began in the early 1970s on a campus that followed Chicago’s example to almost absurd lengths: the University of Rochester. “I can remember my professors yelling, ‘The price is right! The price is right!’” recalls Dartmouth finance professor Kenneth French, who arrived at Rochester just after Thaler left. “It was like a bad game show.”

Rochester was an extreme version of the direction the entire economics profession–not just its finance offshoot–had taken. The idea that economic activity can be explained as rational individuals trying to maximize their wealth goes back at least to 18th-century Scotsman Adam Smith, and the thought that all this was best expressed mathematically occurred to economists as long ago as the 1870s. But the approach didn’t really take off until the 1947 publication of MIT professor Paul Samuelson’s Foundations of Economic Analyses, which recast the principles of economics in the language of Newtonian calculus. The mathematization of economics that followed swept all before it. While academic economics had previously allowed room for multiple streams of thought, it took just a couple of decades before mathematical models built on the assumption of rationality were the only game on campus.

One of the most compelling of those models, and the one that seemed most closely to fit real-world data, was the efficient-markets hypothesis. It had its roots in empirical research that appeared to show stocks moving in a random walk–albeit with an upward trajectory determined by rising corporate earnings. The theoretical explanation, first proffered by Samuelson in 1965 and soon elaborated by Fama (who got his Ph.D. at Chicago in 1964) and others, was that stock prices fluctuate randomly because all knowable information about the value of a stock is already discounted in the price. That is, prices change only in reaction to news, which is by definition unpredictable.

For this view of price movements to work, the market has to behave rationally. That doesn’t mean every last investor has to be rational; it’s enough to assume that the hordes of irrational investors are irrational in different ways, thus canceling each other out. Or failing that–and here the theory begins to wobble–rational investors would be able to take advantage of the market’s temporary insanity to make a killing and push prices to where they belong.

That brings us back to Thaler, who was working on a Ph.D. in economics at Rochester in the early 1970s. His dissertation was an attempt to put a value on human life by looking at how much more people were paid to work in risky fields like mining and logging. He was working on the assumption, of course, that people rationally weighed the risk of death in their decision to accept a job.

Along the way Thaler decided to ask a few friends how much they’d be willing to pay to eliminate a one-in-1,000 chance of immediate death and how much they would have to be paid to willingly accept an extra one-in-1,000 chance of immediate death. What he found was that they wouldn’t pay much for the extra margin of safety but demanded huge sums to accept added risk–which isn’t, strictly speaking, rational. “I came to two conclusions about these answers,” Thaler wrote years later. “(1) I had better get back to running regressions if I want to graduate, and (2) the disparity between buying and selling prices was very interesting.”

Thaler did discuss his subversive thoughts with a few trusted colleagues and people from other disciplines. One of those people happened to be a newly minted psychology Ph.D., who sent Thaler a copy of a 1974 article by Israeli psychology professors Amos Tversky and Daniel Kahneman. (Tversky died in 1996; if he were still around, he surely would have shared in this year’s Nobel.) The article argued that in making decisions involving probability and risk, people rely on mental shortcuts that “are highly economical and usually effective but … lead to systematic and predictable errors.”

It was that last part that was so significant. That people make judgment errors wasn’t news, but if those errors were “systematic and predictable,” well, that was something an equation-wielding economist could get up and run with. (And making decisions involving probability and risk is what investing is all about, although Thaler wasn’t really thinking about that at the time.) Thaler wangled a short-term research gig at Stanford University when Tversky and Kahneman were visiting professors there in 1978 and ended up staying for 15 months. Then he played a key role in unleashing Tversky and Kahneman’s ideas on the economics profession.

The profession did not immediately respond with great enthusiasm. But over the years Thaler began to collect a few allies. Some were economists who shared his fascination with psychology. Others were number-crunching finance professors who had stumbled across seemingly irrational market phenomena–from short-lived ones like the “January effect” of rising stock prices in the first weeks of the year, which disappeared not long after people started writing about it, to the persistent tendency of “value” stocks with low price-to-book ratios to outperform the market. Throughout the 1980s the ranks of the doubters grew, but they remained a fringe element.

The efficient-markets guys, meanwhile, not only had come to occupy the academic mainstream but also had moved in on Wall Street. Not surprisingly, their initial relations with the Street had been hostile. What the professors were saying, after all, was that highly paid fund managers and analysts were not worth a dime. Some of the professors clearly reveled in that: In one famous mid-1960s exchange, a money manager asked MIT’s Paul Cootner, “If you’re so smart, why aren’t you rich?” To which Cootner replied, “If you’re so rich, why aren’t you smart?”

The answer to that second question was that people on Wall Street didn’t have to be smart to get rich, since they could make money off fees and brokerage commissions even when their market calls stank. But the devastating bear market of the 1970s caused some investors to question whether the people with whom they’d entrusted their money really were worth the expense. One logical result of such thinking was the index fund, which instead of trying to outsmart the market simply tried to imitate it while charging much lower fees than actively managed funds do. The first index fund for institutional investors was started in 1971 by Wells Fargo Investment Advisors (now Barclays Global Investors) in San Francisco. The first such fund for retail investors–the Vanguard Index Trust–was launched five years later.

Meanwhile a few finance scholars of a more diplomatic bent than Cootner began spreading their ideas of risk and return in the real world. Princeton economist Burton Malkiel’s A Random Walk Down Wall Street, published in 1973, probably played the biggest role in bringing efficient-markets thinking to the retail investing masses. But on Wall Street itself, the most important messenger was William Sharpe.

Sharpe, now 68, grew up in Southern California and learned his economics at UCLA. He was of the efficient-markets school, but his work (for which he won the economics Nobel in 1990) appealed even to those who still hoped to beat the market. In an efficient market the only way to outperform the market is to take on more risk. Sharpe devised a simple measure of risk based on past volatility, called “beta,” that could be used to build balanced portfolios–and to measure whether active money managers were actually beating the market or just taking on extra risk.

Sharpe wasn’t content to make his point merely in academic journals. He wrote textbooks on investments and finance and did so much consulting for Wall Street firms and pension funds that he gave up full-time teaching at Stanford in the mid-1980s. In 1996 he even launched a dot-com, Financial Engines, to make his advice available to small investors. So while Sharpe believes in efficient markets, he has also spent much of his career helping investors make choices. That, it turns out, makes him a big fan of behavioral finance. “As a practical matter, I still think it’s prudent to assume that the market is pretty close to efficient in terms of pricing and risk and return and all that,” Sharpe says. “On the other hand, we’ve certainly learned from cognitive psychology that ordinary human beings need to have alternatives framed in ways that can help them make right decisions rather than wrong decisions.”

Most of the wrong decisions investors make, behavioral research has shown, stem from overconfidence. That is, we think we know more than we do. We trade too much, we don’t diversify enough, and we extrapolate from the recent past to make assumptions about what will happen next.

As a result, much of what the behavioralists have to offer in terms of advice has to do with protecting retail investors from themselves. That’s why Thaler spends a lot of his time thinking about how best to design 401(k) plans. It’s why Sharpe incorporates behavioralist research into the advice Financial Engines doles out. And it’s almost certainly why Daniel Kahneman, when asked by a CNBC anchorman the day after his Nobel was announced in October what investment tips he had for viewers, responded, “Buy and hold.”

When I recount Kahneman’s words a few weeks later to Fama, he reacts with glee. “That means I won!” he shouts. It is, on one level, an absurd claim. The behavioralists are now clearly the dominant stream in academic finance, having made the leap from outsider status during the 1990s as a new generation of professors rose to positions of prominence. But the real-world phenomenon that cemented the behavioralists’ victory also illustrates why, when it comes to actual investing advice, they sound so much like Fama and Sharpe.

That real-world phenomenon was the stock market bubble of the late 1990s. According to strict efficient-markets thinking, there must be a rational explanation for what happened. Fama describes those sky-high Internet stock valuations as a risky but not crazy bet that one or two of those money-losing Net companies would end up as big as Microsoft. But he’s almost all alone on this one. “We have just lived through the biggest bubble of all time,” says Malkiel, who now calls himself a “random walker with a crutch.” Fama’s favorite collaborator, Dartmouth’s French, is on the verge of using the b-word as well when he stops himself. “I work very closely with Gene,” he says. “He would be very upset if I used that word in print.”

Yale economist Robert Shiller has no such compunctions about ticking off Gene Fama. In 1984 he declared that the logical leap from observing that stock price movements were unpredictable to concluding that the prices are in fact right “represents one of the most remarkable errors in the history of economic thought.” That was Shiller’s first brush with fame. He got more popular attention after the 1987 stock market crash, which the efficient-markets professors had trouble explaining. (“It’s weird,” Sharpe told a reporter at the time. Later his mother called to berate him: “Fifteen years of education, three advanced degrees, and all you can say is, ‘It’s weird’?”)

Shiller is 56 and did his economics training under Samuelson at MIT. He and Thaler have long been allies, but Shiller seems less interested than many of the other behavioralists in assembling the cognitive-psychology building blocks of a market bubble (which would involve that persistent flaw of extrapolating from the recent past to make assumptions about what will happen next). Instead he’s perfectly willing to accept at face value the conventional wisdom that markets are sometimes taken over by fads and mass hysterias. By the mid-1990s Shiller had become convinced that we were entering into one of those mass hysterias. His evidence was straightforward: Price/earn-ings ratios were really high. He began sounding the alarm wherever he could, including the offices of the Federal Reserve Board. Then he wrote Irrational Exuberance, which hit bookstores in March 2000, just as the market peaked.

The book’s perfect timing was dumb luck, Shiller himself says. And while he took most of his own money out of the stock market in the 1990s, his advice to investors now is to “diversify completely” and not try to beat the market. This happens to be what Sharpe would tell you. Or Fama. Or Thaler. The dirty little secret of the behavioralists is that, for all their work on investor irrationality and market anomalies, they still believe that markets work pretty well and that trying to outguess the collective wisdom of millions of investors is usually futile. In answer to Fama’s question of how they plan to calculate the cost of capital in a world where prices are incorrect, the behavioralists say that for the purposes of such calculations, they’ll just assume that prices are right.

But efficient-markets theory has a dirty little secret, too, which is that for the market to remain efficient, there have to be lots of rational investors who believe enough in the market’s inefficiency to spend their careers trying to beat it. Behavioralist theory, of course, has no problem accommodating the belief that some investors can beat the market. In fact, several behavioralist professors, Thaler included, have money-management firms that try to take advantage of the anomalies they discover in their research.

But there’s a limit to the riches that can be dredged from market anomalies. That’s because “markets can remain irrational longer than you can remain solvent.” This aphorism is usually attributed to economist and speculator John Maynard Keynes, and there are those who contend that the whole of the behavioralist case is contained in chapter 12 of Keynes’s 1936 General Theory, with its wonderful depiction of investing as a game of musical chairs. But the argument of modern behavioralists includes a crucial observation that wasn’t in Keynes–that professional investors are now under so much pressure from their customers that they cannot make the kind of long-term bets that might beat the market. If they do, as was the case with a lot of value-oriented mutual funds in the late 1990s, they can soon find themselves without any customers’ money to invest.

That gets us to a world in which an investor with enough staying power and contrarian gumption can beat the market, but the vast majority of mutual funds and hedge funds don’t. In other words, the behavioralists have reconciled the success of a Warren Buffett (which efficient-markets purists have absurdly termed dumb luck) with the overwhelmingly empirical evidence that most professional money managers fail to beat the market.

This is, we posit, a major intellectual accomplishment. What does it mean for you? That’s easy: Buy and hold. Diversify. Put your money in index funds. Pay attention to the one thing you can control–costs–and keep them as low as possible.

- market maven Arthur Zeikel