Great chefs know that it takes more than the right ingredients to make an outstanding stew. If you put everything together in just the right way, ordinary ingredients can turn into magic. In this article, Jeff Merriman-Cohen shows how the same thing is true for investing.

The ideal portfolio may be different for every investor, but that doesn’t mean there are 150 million perfect variations.

Nevertheless, based on the Suggested Portfolios on our Web site and the strategies we manage for clients, there are probably thousands of combinations that could qualify, depending on any one person’s needs.

How can an investor choose the right one?

In this article, I’ll walk through some of the steps I used when I was still meeting with clients (something that’s ruled out by my current job) for the first time. I hope this will give you some good ideas on how to put together a combination that’s just right for you.

The most important initial conversation with any new client is about risk. It’s the most basic part of investing, the topic that most of the industry (and most investors) would be happy to avoid altogether.

Let me be blunt about this: Investors who don’t understand risk cannot understand the most important decisions and choices they must make.

Read the full article here

If you have the emotional makeup for it, timing can reduce your risks and enhance your returns. If you’re satisfied that you have what it takes to be a market timer, here are the best tips I know for doing it successfully. They aren’t necessarily listed in order of priority. In fact, I suggest that you regard each one of them as the top priority.

1. Use mechanical strategies.
Timing financial markets is already plenty hard without worrying about making predictions or (even worse) thinking you have to decide who is right when smart economists and savvy analysts make conflicting forecasts and draw different conclusions. If you leave the final decisions to subjective factors, you will never be sure what you are supposed to do at any given moment. That will cause you anxiety and delay. And you’ll have a system you can’t count on. Rely primarily on trend-following systems that are based mainly on trends that are impacted by actual prices in the market. There’s nothing speculative about prices. They reflect what buyers and sellers are doing, and that’s about as reliable an indicator of the direction of the market as you can find.

2. Do not — repeat DO NOT — pay much attention to the effect of every trade.
The majority of individual trades will be irrelevant to your long-term results. If you feel you must focus on each trade and agonize over what it means, that’s a sure sign you are not cut out to be a successful timer. Dwelling on each trade is a sure way to drive yourself nuts, and it won’t improve your results at all.

3. Use timing systems that are right for you and your temperament.
The perfect strategy for you will match your time horizon, will respect your emotional needs and will operate within your tolerance for risk and change. There are short-term systems that trade frequently, long-term systems that trade infrequently and intermediate-term systems that typically trade two to six times a year. Over long periods of time, no group has an inherent return advantage over the others. But the practical and emotional differences are important. If you have a strong desire to perform closely in synch with the market, use short-term systems, which are good at quickly reacting to today’s highly volatile market swings. However, short-term systems demand that you make many trades, and each trade has potential tax consequences unless you are investing in a tax-sheltered account. The volume of trades demands a lot of attention, produces a lot of paperwork and tests the patience of many mutual funds, which sometimes won’t accept accounts from very active timers. If on the other hand you have a strong aversion to whipsaws, you can use long-term systems. But doing so will sometimes make you wait for a move of 20 percent or more before you buy or sell. For the best compromise, do as we do: Use intermediate-term systems. This level of activity is likely to be accepted by most mutual funds and is not too demanding emotionally.

4. Use multiple timing systems, stick with them and let them act independently in your portfolio.
Even the most productive system from the past may be a mediocre performer in the future, and the reverse could also be the case. We use four U.S. equity timing models, each of which governs 25 percent of our portfolio. We have faith in the systems as a group. But we don’t have enough faith in any one system to let it govern the whole portfolio. You shouldn’t either. Just as you should not chase recent performance in your choice of mutual funds or asset classes, do not chase recent high-flying timing systems. One of the smartest timers in this business, a fellow newsletter publisher whose work I respect, has averaged about 9 percent over the past decade. He chooses good timing systems, which have produced average returns of more than 18 percent before he adopts them. But he doesn’t stick with those systems. Whenever the system he has been using disappoints him, he finds another one that would not have done so, based on real or hypothetical past performance, and then he switches to that system. In theory, this may seem like a valid way to search for the very finest system on the planet. But in truth, such “superstar” timing models simply do not exist. They are a myth. Good performance one year doesn’t mean anything about performance the next year – not anything. This is one of the hardest facts for investors to accept, but it’s true. Therefore, we believe your best bet is to find several robust timing models and stick with them.

5. Remember that whether you use a buy-and-hold approach or market timing, asset allocation is the most important investment decision you will make as an investor.
Use many assets or asset classes that move up and down at different times and at different speeds. Include international diversification, whether you invest in equities, bonds or both. Just as you never know which timing model will be the star performer in a given quarter or year, you never know which asset class will be the overachiever and which will be the laggard.

6. Follow your systems and your strategy.
Put them into action without fail and without exception. Remember this Chinese proverb: “He who knows but does not act, still does not know.” If you do only one thing right and everything else wrong, make sure this is the one thing you do right. This is the most essential key of all. If buying diet books and exercise equipment took off pounds, obesity would not be a major health concern. You can devise the greatest portfolio in the history of investing, but it will do you no good unless you commit your money to it. The greatest timing models do you no good unless you apply them. Therefore, do whatever is necessary to get it done.

7. Before you start timing, take off the rose-colored glasses, if you are wearing them.
Focus in advance on the difficulties you can expect as well as the ultimate rewards you hope to achieve. Accepting the rewards of success will be easy. But you’ll never get to the finish line unless you can deal with the hurdles along the track. Know the level of interim losses you are likely to encounter with your strategy, and make sure you are willing to accept them. In the early 1970s, buy-and-hold investors in the Standard & Poor’s 500 Index suffered a 39 percent loss in one year. Even timing can be ugly. In our Worldwide Equity strategy, all our back-testing has failed to produce a decline as high as 15 percent in any 12-month period. Yet we believe that any strategy with the potential to produce returns of 13 to 15 percent a year also has the potential to lose 15 percent in a year, so that’s the figure we use when we project the expected worst-case scenario. Bottom line: Do not expect magic from any timing system.

8. Give timing enough time to work. In the short term, anything can happen.

In the long term, if you have chosen a strategy carefully and you follow the discipline, you should be rewarded accordingly. But how long is long enough? There are two places to look for the answer. The first is in your own psychology. Do you normally undertake long-term projects or strategies, comfortable knowing that you’ll have to wait for any payoff? If so, you may be a good candidate for market timing. But if on the other hand you are usually quick to judge the success or failure of something you start, and if you need instant gratification, you’ll probably have trouble being a successful market timer. The second place to look for the answer is in statistics and history. Arm yourself (or have your manager do this for you) with the past statistical performance, either real or hypothetical, of your proposed investment. Over the longest period for which you have data, determine the depth of the largest drawdown. Find out how long it took to return to break-even. One of our most aggressive timing programs, which we call by the shorthand of +2 to -1, meaning it attempts to double the performance of large U.S. stocks when the market is rising and to do the opposite of the market during declines, once took nearly two years to recover from a 20 percent drawdown. Are you prepared to endure that in order to make returns of more than 20 percent? Here’s an even tougher example of the extraordinary patience required of investors: In 1973 and 1974, the S&P 500 declined by 44.9 percent. The index eventually regained its pre-decline level. But it took 66 months for some investors just to break even. Unless you are sure you’d stick with a strategy through the longest historical drawdown for which you have data, don’t embark on that strategy.

9. Unless you are absolutely committed to being a market timer, use both timing and buy-and-hold.
This gives you two non-correlated approaches that will have differing results in any given period. Over long periods, carefully chosen investments in similar assets may generate similar returns with buy-and-hold and market timing. But in the meantime, the average of the two may give you lower losses, less risk and (perhaps most important) less anxiety than either market timing or buy-and-hold alone. This combination may be more appealing to many people than the peaks and valleys of each approach separately.

10. Make sure you understand in advance the realities of market timing.
And make sure you are prepared for them. I cannot emphasize this point too much, so I hope you’ll read the following overview. If you invest money that’s governed by timing and you’re surprised by everything that happens to your investment, you will always feel off balance. You’ll come to dislike and distrust timing. And even if you follow your system, timing will produce anxiety for you. That is just the opposite of what it’s intended to do.

Do managers of bond mutual funds generate value for fund holders by successfully timing the market? In the September 2009 update of their paper entitled “Measuring the Timing Ability and Performance of Bond Mutual Funds”, Yong Chen, Wayne Ferson and Helen Peters evaluate the ability of U.S. bond fund managers to time nine common factors related to bond returns. The nine factors reflect the term structure of interest rates, credit and liquidity spreads, currency exchange rates, mortgage spread and equity market returns. The authors also define seven benchmarks matching different bond fund styles. Using monthly returns for more than 1,400 U.S. bond mutual funds and contemporaneous bond market factor and benchmark data during January 1962 through March 2007, they conclude that:

• Across all bond mutual funds over the entire sample period, the mean monthly return is 0.62% and the standard deviation of monthly returns is 1.51%.
• After controlling for return series non-linearities unrelated to timing, the evidence for market timing ability is on average neutral to weak.
• With these controls, 75% of bond funds significantly outperform style-matched benchmarks before fund costs (average expense ratio of each fund plus an assumed round trip trading cost associated with the fund style), but there is no evidence of net outperformance on average after costs.

In summary, evidence provides weak support for a belief that managers of U.S. bond mutual funds can on average time the bond market, but fund costs/fees offset any associated net outperformance of reasonable benchmarks.

“Do you have the mental fortitude to accept huge gains?”

“This comment usually gets a hearty laugh, which merely goes to show how little most people have determined it actually to be a problem. But consider how many times has the following sequence of events occurred? For a full year, you trade futures contracts, making $1000 here, losing $1500 there, making $3000 here and losing $2000 there. Once again, you enter a trade because your (trading) method told you to do so. Within a week, you’re up $4000. Your friend/partner/acquaintance/broker/advisor calls you and, looking out only for your welfare, tell you to take your profit. You have guts, though, and you wait. The following week, your position is up $8000, the best gain you have ever experienced. ‘Get out!’ says your friend. You sweat, still hoping for further gains. The next Monday, your contract opens limit (down) against you. Your friend calls and says, ‘I told you so. You got greedy. But hey, you’re still way up on the trade. Get out tomorrow.’ The next day, on the opening, you exit the trade, taking a $5000 profit. It’s your biggest profit of the year, and you click your heels, smiling gratefully, proud of yourself. Then, day after day for the next six months, you watch the market continue

to go in the direction of your original trade. You try to find another entry point and continue to miss. At the end of six months, your method finally, quietly, calmly says, ‘Get out.’ You check the figures and realize that your initial entry, if held, would have netted $450,000.”

“So what was your problem? Simply that you had allowed yourself, unconsciously, to define your ‘normal’ range of profit and loss. When the big trade finally came along, you lacked the self esteem to take all it promised . . .who were you to shoot for such huge gains? Why should you deserve more than your best trade of the year? Then you abandoned both (trading) method and discipline. To win the game, make sure that you understand why you’re in it. The big moves in markets only come once or twice a year. Those are the ones which will pay you for all the work, fear, sweat and aggravation of the previous years. Don’t miss them for reasons other than those required by your objectively defined method. The IRS categorizes capital gains as ‘unearned income’ that’s baloney. It’s hard to make money in the market. Every dime you make, you richly deserve. Don’t ever forget that.”

. . . Robert Prechter – the Elliott Wave Theorist (1992)

One of the tenets of modern portfolio management most damaged due to the financial crisis has been asset allocation.  We have discussed how during a bear market correlations tend to one, the myth of the all-weather portfolio and how investors may need a more dynamic approach to asset allocation.  It seems we are not alone in our opinion(s).

Noted finance professor Andrew Lo of MIT has a piece in the Financial Times discussing how the practice of portfolio management has been upturned in part due to the financial crisis – asset allocation included.  While we recommend you read the entire piece, the bottom line is that the investment world is now much more complicated post-crisis.  Lo writes:

Diversification is still a good idea, but it has become much harder to achieve. Thanks to the increasing competition for additional yield, every type of investment vehicle and strategy has experienced substantial growth in assets under management.

The asset classes (and dynamic) strategies that have been touted as portfolio diversifiers have seen an influx of capital and managers.  Lo cites the case of the “carry trade” that has become popular enough to have spawned an ETF that follows the strategy.

Read the full post here

Interesting paper from authors Nils Tuchschmid, Erik Wallerstein and Sassan Zaker of Julius Baer Asset Management. They suggest that hedge fund clones are broadly succeeding in replicating the investment returns of real hedge funds. At the same time, however, some clones have exhibited too much correlation with equity markets and also have raised fears among some investors about the increasing complexity of replication models, the researchers warned.

The report, which looked at 21 clones over the period April 2008 to May 2009, concluded: “Hedge fund replication products seem to deliver competitive performance relative to hedge funds. More importantly they are able to deliver this at a far lower fee level than hedge funds.”

The authors’ research found that the vast majority of clones exhibited a correlation of at least 70 per cent to industry benchmarks operated by Hedge Fund Research and Credit Suisse/Tremont. Most lost less than the typical 10-15 per cent declines recorded by the industry at large, although Wallerstein cautioned,  the relative performance of clones in a bull market remained unproven. “Shortable” clones, which allow investors to benefit from losses in the underlying industry, appear to succeed in mirroring long approaches, he added.

Read the paper here

For quite some time now financial advisers of all stripes have been in search of the elusive “all-weather portfolio.”  That is, an asset allocation that serves to protect investors in bad times (bear markets) and performs well in good times (bull markets).  Does an all-weather portfolio really exist?

Prior to the economic crisis many would have answered in the affirmative and would have pointed to the large university endowment funds as examples of investors who had achieved this goal.  However the aftermath of the credit crisis and ensuing bear market indicate these funds have failed to achieve this goal.

Maybe it isn’t that case that asset allocation models are broken.  It may simply be the case that we are asking too much of asset allocation as a discipline.  In what other investing endeavor do we expect to have the best of all possible worlds?

Read the full article here

From the Abstract:

This paper investigates the level and the determinants of the optimal number of hedge fund managers in a Fund of Hedge Funds (FOFs). The paper also analyzes the impact that this level has on the performance and the volatility of returns of the typical FOF. Several important findings emerge. First, we find that the number of underlying hedge funds (HFs) included into a FOF has a negative and significant impact on the volatility of returns but less of an impact on the actual returns. However, if we properly classify the FOFs into several larger categories of interest, we find evidence that the FOFs having between 6 and 10 hedge fund managers perform the best. On average this group of FOFs has assets under management of around $200 million. Second, further evidence shows that there is a positive relationship between the size of the FOF portfolio and the lifetime of the fund. Third, several factors that influence the number of HF managers into a FOF include, but are not limited to the amount of leverage, the redemption frequency, the size of the fund, the total number of assets managed by the FOF manager, whether the fund issues a K-1 schedule for tax purposes, the currency in which the fund trades, the geographical focus, and the strategy pursued.

From the Introduction:

Many institutional investors, having no experience in hedge fund manager selection are willing to pay the additional layer of fees of owning a pre-packaged and diversified FOFs rather than setting up an in-house FOF. The number of underlying hedge fund managers in a FOF can play a primordial role in its performance and its survival. We believe this is the first paper to our knowledge that examines the optimal number of underlying hedge fund managers in FOFs. Numerous papers have stated what the optimal number of hedge fund managers in FOFs should be, but none have used an actual dataset to examine this.

Read the paper here

The advice rolls off the tongues of financial planners and appears frequently in the pages of financial magazines such as Money: To have any shot at retiring well, you need to invest a good portion of your money in stocks.

But mention this to Boston University School of Management professor Zvi Bodie, author of “Worry-Free Investing,” and you’ll get a stern reminder of how equities often betray investors. And you’ll get an earful about how millions of us are taking too much risk with our nest eggs.

……….

But don’t you need the growth that stocks provide to combat the risk of inflation?

Inflation is exactly what Treasury Inflation-Protected Securities (TIPS) and I bonds were created to protect against. Even if equities did perform well in periods of inflation, you’re exposing yourself to an even greater risk of a stock market decline. And as it turns out, anytime there’s been significant inflation, equities have been a terrible investment. Just look at the 1970s.

So you’d tell an investor to have 100% of his retirement money in TIPS?

Yes. In fact, I have 100% of my own retirement money in TIPS. I do have a small account of nonretirement funds in which I invest in bonds, options, and stocks.

Currently, long-term TIPS earn just 2% after inflation. How is anyone going to be able to retire on so little growth?

If you look at most online retirement calculators, they make two assumptions: one, that you want to retire at age 65, and two, that people will be able to save only a certain amount — say 10%. As a result, they spit out risky portfolios to get a higher return. Well, who says we all want to retire at 65 and can save only 10%? What if I retire at 70 or 75? What if I save 30%? Suddenly, you don’t need to take so much risk in your portfolio. Now, if you put 100% in TIPS, you will have to save upwards of 20% of your annual pay, even if you’re young, to retire at age 65. But I think it would be more reasonable to expect to retire at a later date.

Read the full interview here

Is The Market Rational? No, say the experts. But neither are you–so don’t go thinking you can outsmart it.
By Justin Fox
December 9, 2002

Buy and hold. Diversify. Put your money in index funds. Pay attention to the one thing you can control–costs–and keep them as low as possible. Today that is pretty standard, if often unheeded, investment advice. Forty years ago it was revolutionary. The revolution started on college campuses, in particular at the University of Chicago, and it went by the unrevolutionary-sounding name “efficient markets.”

“In an efficient market,” wrote Chicago professor Eugene Fama in a landmark paper he delivered at the 1969 annual meeting of the American Finance Association, “prices ‘fully reflect’ available information.” That is, in an efficient market you can’t beat the market unless you have inside information. So why bother trying?

That logic led, among other things, to the creation of index funds that aim to mimic, not beat, the likes of the S&P 500 and the Wilshire 5000. Today such funds account for about 10% of total U.S. stock market capitalization, as well as 60% of what little money has flowed into equity mutual funds so far this year. But millions of small investors have continued to ignore the advice derived from efficient-markets theory, preferring instead to trade stocks and pile in and out of mutual funds in search of elusive market-beating returns (blowing much of their money on fees and commissions in the process).

Meanwhile, back on campus, a new generation of finance professors has been ripping Fama’s teachings to shreds. The organizing principle for this new breed of scholars is not efficient markets but something called behavioral finance. Behavioral finance teaches that stock market investors are irrational, that future stock price movements are at least partly predictable from past behavior, and that careful analysis of past trends and financial reports can pay off. Which happens to be the way most investors see the market already.

Over the next few pages we’re going to take you on a journey through the academic battles that have brought us to this point. This is FORTUNE’s annual Investor Guide, not The Chronicle of Higher Education, so we wouldn’t tell this story if we didn’t think it had relevance for investors battered by the experience of the past few years. The message that the behavioral finance guys have for investors is that yes, you can beat the market, but–for reasons that are essential to the whole behavioralist case–you almost certainly won’t. As a result, they end up offering much of the same investment advice that the efficient markets folks do. Only this time we might actually listen.

Gene Fama, it must be said, doesn’t buy any of this. It’s a cold, gray mid-autumn day in Chicago, but Fama is wearing a loud shirt with too-short sleeves that prove him to be probably the buffest 63-year-old finance professor on the planet. With the hint of a Boston accent that remains after two-thirds of a lifetime in Chicago, he’s talking the academic version of trash. “I don’t know that it’s progressed much beyond the level of curiosity items,” Fama says of the research done by the behavioralists. “They’ve got lots of interesting curiosity items.”

For years Fama was more than tolerant of this search for “curiosity items.” Unlike his longtime Chicago colleague Merton Miller, who saw the behavioralists’ work as an ideological assault on free markets (Miller died in 2000), Fama always encouraged empirical research, even research that revealed seemingly inefficient market behavior. In a 1991 sequel to his famous “Efficient Capital Markets” paper of 1969, he acknowledged that reality had in fact turned out a lot messier than he and other efficient-markets theorists had envisioned–although not so messy that the theories couldn’t accommodate it.

In 1997, though, Fama wrote still another paper, one that argued that a lot of the purported market anomalies discovered by the behavioralists were due to bad statistical work, and that the behavioralists’ attempts at building a theoretical alternative to the efficient-markets hypothesis had so far been “embarrassing.” The paper was something of a sensation–until interest in the D.C.-area sniper sent a piece called “Multiple-Victim Public Shootings” to the top of the charts in October, it was the most downloaded work in the almost six-year history of the Social Science Research Network, a leading academic website. Fama had no problem with behavioral finance as a critique of efficient-markets theory. But he thinks it’s a disaster as a replacement. “I don’t know what asset pricing would look like in a world that really took behavioral finance seriously,” he says. “If you really think prices are incorrect, what are you going to tell me about the cost of capital?”

We’ll get back to that question. But first let us document the behavioralists’ triumph. Half of this year’s economics Nobel went to their patron saint, Princeton psychologist Daniel Kahneman (the other half went to Vernon Smith of George Mason, whose economic experiments have also shot holes in efficient-markets dogma). Then there’s the John Bates Clark Medal, awarded by the American Economic Association every two years to the most important U.S. economist under 40: The 1999 and 2001 editions both went to behavioralists. On the pop-culture front, Yale efficient-markets skeptic Robert Shiller’s 2000 bestseller Irrational Exuberance was the most talked-about book by an economist in years.

The most dramatic development of all, though, may be that the office directly below Fama’s at Chicago’s Graduate School of Business now belongs to behavioralist pioneer Richard Thaler, 57. A magazine profile last year characterized Thaler, to the undying amusement of his students, as “thick-set,” but that’s not quite fair. He is not the jock that his upstairs neighbor is–Fama beats him at tennis. But Thaler, who arrived in Chicago in 1995 after years in the relative academic wilderness of Cornell University, appears to have eclipsed Fama as the most influential faculty member at the business school that has had more influence on the study of finance than any other.

For decades the University of Chicago was the temple for those who believed that markets always got things right. Thaler’s intellectual journey, in fact, began in the early 1970s on a campus that followed Chicago’s example to almost absurd lengths: the University of Rochester. “I can remember my professors yelling, ‘The price is right! The price is right!’” recalls Dartmouth finance professor Kenneth French, who arrived at Rochester just after Thaler left. “It was like a bad game show.”

Rochester was an extreme version of the direction the entire economics profession–not just its finance offshoot–had taken. The idea that economic activity can be explained as rational individuals trying to maximize their wealth goes back at least to 18th-century Scotsman Adam Smith, and the thought that all this was best expressed mathematically occurred to economists as long ago as the 1870s. But the approach didn’t really take off until the 1947 publication of MIT professor Paul Samuelson’s Foundations of Economic Analyses, which recast the principles of economics in the language of Newtonian calculus. The mathematization of economics that followed swept all before it. While academic economics had previously allowed room for multiple streams of thought, it took just a couple of decades before mathematical models built on the assumption of rationality were the only game on campus.

One of the most compelling of those models, and the one that seemed most closely to fit real-world data, was the efficient-markets hypothesis. It had its roots in empirical research that appeared to show stocks moving in a random walk–albeit with an upward trajectory determined by rising corporate earnings. The theoretical explanation, first proffered by Samuelson in 1965 and soon elaborated by Fama (who got his Ph.D. at Chicago in 1964) and others, was that stock prices fluctuate randomly because all knowable information about the value of a stock is already discounted in the price. That is, prices change only in reaction to news, which is by definition unpredictable.

For this view of price movements to work, the market has to behave rationally. That doesn’t mean every last investor has to be rational; it’s enough to assume that the hordes of irrational investors are irrational in different ways, thus canceling each other out. Or failing that–and here the theory begins to wobble–rational investors would be able to take advantage of the market’s temporary insanity to make a killing and push prices to where they belong.

That brings us back to Thaler, who was working on a Ph.D. in economics at Rochester in the early 1970s. His dissertation was an attempt to put a value on human life by looking at how much more people were paid to work in risky fields like mining and logging. He was working on the assumption, of course, that people rationally weighed the risk of death in their decision to accept a job.

Along the way Thaler decided to ask a few friends how much they’d be willing to pay to eliminate a one-in-1,000 chance of immediate death and how much they would have to be paid to willingly accept an extra one-in-1,000 chance of immediate death. What he found was that they wouldn’t pay much for the extra margin of safety but demanded huge sums to accept added risk–which isn’t, strictly speaking, rational. “I came to two conclusions about these answers,” Thaler wrote years later. “(1) I had better get back to running regressions if I want to graduate, and (2) the disparity between buying and selling prices was very interesting.”

Thaler did discuss his subversive thoughts with a few trusted colleagues and people from other disciplines. One of those people happened to be a newly minted psychology Ph.D., who sent Thaler a copy of a 1974 article by Israeli psychology professors Amos Tversky and Daniel Kahneman. (Tversky died in 1996; if he were still around, he surely would have shared in this year’s Nobel.) The article argued that in making decisions involving probability and risk, people rely on mental shortcuts that “are highly economical and usually effective but … lead to systematic and predictable errors.”

It was that last part that was so significant. That people make judgment errors wasn’t news, but if those errors were “systematic and predictable,” well, that was something an equation-wielding economist could get up and run with. (And making decisions involving probability and risk is what investing is all about, although Thaler wasn’t really thinking about that at the time.) Thaler wangled a short-term research gig at Stanford University when Tversky and Kahneman were visiting professors there in 1978 and ended up staying for 15 months. Then he played a key role in unleashing Tversky and Kahneman’s ideas on the economics profession.

The profession did not immediately respond with great enthusiasm. But over the years Thaler began to collect a few allies. Some were economists who shared his fascination with psychology. Others were number-crunching finance professors who had stumbled across seemingly irrational market phenomena–from short-lived ones like the “January effect” of rising stock prices in the first weeks of the year, which disappeared not long after people started writing about it, to the persistent tendency of “value” stocks with low price-to-book ratios to outperform the market. Throughout the 1980s the ranks of the doubters grew, but they remained a fringe element.

The efficient-markets guys, meanwhile, not only had come to occupy the academic mainstream but also had moved in on Wall Street. Not surprisingly, their initial relations with the Street had been hostile. What the professors were saying, after all, was that highly paid fund managers and analysts were not worth a dime. Some of the professors clearly reveled in that: In one famous mid-1960s exchange, a money manager asked MIT’s Paul Cootner, “If you’re so smart, why aren’t you rich?” To which Cootner replied, “If you’re so rich, why aren’t you smart?”

The answer to that second question was that people on Wall Street didn’t have to be smart to get rich, since they could make money off fees and brokerage commissions even when their market calls stank. But the devastating bear market of the 1970s caused some investors to question whether the people with whom they’d entrusted their money really were worth the expense. One logical result of such thinking was the index fund, which instead of trying to outsmart the market simply tried to imitate it while charging much lower fees than actively managed funds do. The first index fund for institutional investors was started in 1971 by Wells Fargo Investment Advisors (now Barclays Global Investors) in San Francisco. The first such fund for retail investors–the Vanguard Index Trust–was launched five years later.

Meanwhile a few finance scholars of a more diplomatic bent than Cootner began spreading their ideas of risk and return in the real world. Princeton economist Burton Malkiel’s A Random Walk Down Wall Street, published in 1973, probably played the biggest role in bringing efficient-markets thinking to the retail investing masses. But on Wall Street itself, the most important messenger was William Sharpe.

Sharpe, now 68, grew up in Southern California and learned his economics at UCLA. He was of the efficient-markets school, but his work (for which he won the economics Nobel in 1990) appealed even to those who still hoped to beat the market. In an efficient market the only way to outperform the market is to take on more risk. Sharpe devised a simple measure of risk based on past volatility, called “beta,” that could be used to build balanced portfolios–and to measure whether active money managers were actually beating the market or just taking on extra risk.

Sharpe wasn’t content to make his point merely in academic journals. He wrote textbooks on investments and finance and did so much consulting for Wall Street firms and pension funds that he gave up full-time teaching at Stanford in the mid-1980s. In 1996 he even launched a dot-com, Financial Engines, to make his advice available to small investors. So while Sharpe believes in efficient markets, he has also spent much of his career helping investors make choices. That, it turns out, makes him a big fan of behavioral finance. “As a practical matter, I still think it’s prudent to assume that the market is pretty close to efficient in terms of pricing and risk and return and all that,” Sharpe says. “On the other hand, we’ve certainly learned from cognitive psychology that ordinary human beings need to have alternatives framed in ways that can help them make right decisions rather than wrong decisions.”

Most of the wrong decisions investors make, behavioral research has shown, stem from overconfidence. That is, we think we know more than we do. We trade too much, we don’t diversify enough, and we extrapolate from the recent past to make assumptions about what will happen next.

As a result, much of what the behavioralists have to offer in terms of advice has to do with protecting retail investors from themselves. That’s why Thaler spends a lot of his time thinking about how best to design 401(k) plans. It’s why Sharpe incorporates behavioralist research into the advice Financial Engines doles out. And it’s almost certainly why Daniel Kahneman, when asked by a CNBC anchorman the day after his Nobel was announced in October what investment tips he had for viewers, responded, “Buy and hold.”

When I recount Kahneman’s words a few weeks later to Fama, he reacts with glee. “That means I won!” he shouts. It is, on one level, an absurd claim. The behavioralists are now clearly the dominant stream in academic finance, having made the leap from outsider status during the 1990s as a new generation of professors rose to positions of prominence. But the real-world phenomenon that cemented the behavioralists’ victory also illustrates why, when it comes to actual investing advice, they sound so much like Fama and Sharpe.

That real-world phenomenon was the stock market bubble of the late 1990s. According to strict efficient-markets thinking, there must be a rational explanation for what happened. Fama describes those sky-high Internet stock valuations as a risky but not crazy bet that one or two of those money-losing Net companies would end up as big as Microsoft. But he’s almost all alone on this one. “We have just lived through the biggest bubble of all time,” says Malkiel, who now calls himself a “random walker with a crutch.” Fama’s favorite collaborator, Dartmouth’s French, is on the verge of using the b-word as well when he stops himself. “I work very closely with Gene,” he says. “He would be very upset if I used that word in print.”

Yale economist Robert Shiller has no such compunctions about ticking off Gene Fama. In 1984 he declared that the logical leap from observing that stock price movements were unpredictable to concluding that the prices are in fact right “represents one of the most remarkable errors in the history of economic thought.” That was Shiller’s first brush with fame. He got more popular attention after the 1987 stock market crash, which the efficient-markets professors had trouble explaining. (“It’s weird,” Sharpe told a reporter at the time. Later his mother called to berate him: “Fifteen years of education, three advanced degrees, and all you can say is, ‘It’s weird’?”)

Shiller is 56 and did his economics training under Samuelson at MIT. He and Thaler have long been allies, but Shiller seems less interested than many of the other behavioralists in assembling the cognitive-psychology building blocks of a market bubble (which would involve that persistent flaw of extrapolating from the recent past to make assumptions about what will happen next). Instead he’s perfectly willing to accept at face value the conventional wisdom that markets are sometimes taken over by fads and mass hysterias. By the mid-1990s Shiller had become convinced that we were entering into one of those mass hysterias. His evidence was straightforward: Price/earn-ings ratios were really high. He began sounding the alarm wherever he could, including the offices of the Federal Reserve Board. Then he wrote Irrational Exuberance, which hit bookstores in March 2000, just as the market peaked.

The book’s perfect timing was dumb luck, Shiller himself says. And while he took most of his own money out of the stock market in the 1990s, his advice to investors now is to “diversify completely” and not try to beat the market. This happens to be what Sharpe would tell you. Or Fama. Or Thaler. The dirty little secret of the behavioralists is that, for all their work on investor irrationality and market anomalies, they still believe that markets work pretty well and that trying to outguess the collective wisdom of millions of investors is usually futile. In answer to Fama’s question of how they plan to calculate the cost of capital in a world where prices are incorrect, the behavioralists say that for the purposes of such calculations, they’ll just assume that prices are right.

But efficient-markets theory has a dirty little secret, too, which is that for the market to remain efficient, there have to be lots of rational investors who believe enough in the market’s inefficiency to spend their careers trying to beat it. Behavioralist theory, of course, has no problem accommodating the belief that some investors can beat the market. In fact, several behavioralist professors, Thaler included, have money-management firms that try to take advantage of the anomalies they discover in their research.

But there’s a limit to the riches that can be dredged from market anomalies. That’s because “markets can remain irrational longer than you can remain solvent.” This aphorism is usually attributed to economist and speculator John Maynard Keynes, and there are those who contend that the whole of the behavioralist case is contained in chapter 12 of Keynes’s 1936 General Theory, with its wonderful depiction of investing as a game of musical chairs. But the argument of modern behavioralists includes a crucial observation that wasn’t in Keynes–that professional investors are now under so much pressure from their customers that they cannot make the kind of long-term bets that might beat the market. If they do, as was the case with a lot of value-oriented mutual funds in the late 1990s, they can soon find themselves without any customers’ money to invest.

That gets us to a world in which an investor with enough staying power and contrarian gumption can beat the market, but the vast majority of mutual funds and hedge funds don’t. In other words, the behavioralists have reconciled the success of a Warren Buffett (which efficient-markets purists have absurdly termed dumb luck) with the overwhelmingly empirical evidence that most professional money managers fail to beat the market.

This is, we posit, a major intellectual accomplishment. What does it mean for you? That’s easy: Buy and hold. Diversify. Put your money in index funds. Pay attention to the one thing you can control–costs–and keep them as low as possible.