Monte Carlo simulations vs historical backtests

Nobody knows what markets will do over your retirement. So planners test a plan against many possible futures and count how often it works. There are two main ways to make those futures.

A historical backtest replays the returns that actually happened, starting from each past year. A Monte Carlo simulation draws random returns from a statistical model, thousands of times. A third approach, bootstrapping, sits between them.

Each method answers a slightly different question. Knowing how they differ helps you read a chance of success sensibly.

Historical backtests

A backtest asks: if I'd retired in each past year, with this plan, would the money have lasted?

Say you have a 30-year plan and a century of yearly returns. The test retires you in the first year of the data and runs 30 years. Then it retires you a year later, and so on. Each starting year gives one result. The share that never ran out is the success rate.

Strengths:

• The returns really happened. Crashes, inflation spikes and slow recoveries come in the order they occurred.
• Stocks, bonds and inflation move together the way they actually did.
• It's easy to explain. "This plan would have survived every period except two" is concrete.

Limits:

• Few independent periods. A century contains only about three 30-year periods that don't overlap. Retiring in two neighboring years gives nearly the same result, so the tests aren't separate trials.
• Recent years drop out. A retirement that started 15 years ago doesn't have 30 years of results yet, so it can't be counted.
• Only what happened. The future may hold worse sequences than the past. A backtest can't show them.
• One country's history. Results reflect one market's experience, which was strong by world standards.

Monte Carlo simulations

A Monte Carlo simulation builds futures from assumptions. You set an average return and a typical yearly swing for each type of investment, and how they move together. The simulation draws a random year, then another, until it has a whole retirement. It repeats this thousands of times.

Strengths:

• Many futures. Thousands of paths give a smoother, more stable success rate.
• Futures worse than history. Random draws can string together bad years in ways that haven't happened yet.
• Your own assumptions. You can test lower future returns than the past, or higher inflation.

Limits:

• Garbage in, garbage out. The answer depends on the average and swing you assume. Small changes in expected return move the result a lot.
• Tails can be too thin. A simple bell-curve model makes very bad years rarer than they've been in real markets. Models with fatter tails correct for that.
• No memory. Basic models treat each year as independent. Real markets have stretches of high inflation or slow growth that last years.

Bootstrapping: reshuffled history

A bootstrap builds futures by drawing chunks of real history at random. A five-year block keeps the real order inside each chunk, including crashes and recoveries, and shuffles the chunks.

This gives many more futures than a straight backtest, while keeping real-world patterns inside each block. It still can't produce a single year worse than history's worst.

A worked example

Nadia and Raj test the same plan three ways. They plan to spend $70,000 a year from a $1,600,000 portfolio for 35 years. The results below are invented for the example.

Why the spread? The historical test has few periods, and none of them include the worst sequences the random models can draw. The fatter-tailed model adds rare, severe years. None of these numbers is the right one. Together they say the plan works in most futures, with a meaningful risk in poor ones.

Nadia and Raj decide what they'd do if markets went badly. They plan a 10% spending cut if their balance falls below a set level. Retesting with that cut lifts all three results above 95%.

How to read the results

1. Look at the range, not one number. If methods disagree widely, your plan is sensitive to how markets behave.
2. Study the failures. When does the money run short, and how much is missing? A shortfall at 94 is a different problem from one at 72.
3. Test lower returns on purpose. Run your Monte Carlo with returns a point or two below history.
4. Remember you can adjust. Both methods usually assume you never change your spending. Real people do. See spending guardrails.

How the app handles both methods

In Outlook, you choose how returns are drawn:

• random returns from a bell-curve model, a lognormal model, or a model with fatter tails
• reshuffled history in blocks, with a block length you choose
• every historical starting year in order
• fixed average returns, with no randomness

The app runs up to 1,000 simulated futures on the free plan and up to 10,000 on Plus. A random seed means the same plan gives the same results each time, so changes you make are compared fairly. Inflation varies in the simulated futures, and investments move together according to a correlation setting.

Outlook also has stress tests that replay your plan through difficult historical starting years. You can define success as never running out, never dropping below a spending floor, or leaving a minimum estate. See chance of success, explained.

Frequently asked questions

Which method is more accurate?

Neither predicts the future. Backtests show how a plan would have done in real history. Simulations show a wider range of what could happen under your assumptions. Using both gives a fuller picture.

Why does my chance of success change when I rerun it?

With a fixed seed, it shouldn't. If you change the seed or the number of futures, small changes are normal. Differences of a point or two aren't meaningful.

Should I aim for 100%?

Usually not. Reaching 100% often means spending much less than you could in most futures. Many people aim for a high but not perfect number and plan adjustments for the bad cases.