How to Backtest Strategies Correctly in NinjaTrader

A backtest is only useful if it can tell you the strategy is bad. Most of them can't, because they ignore commissions, use too little data, or get tuned until the equity curve looks perfect. The result is false confidence, and false confidence is expensive in live trading. This guide covers a cleaner way to run a test in NinjaTrader 8's Strategy Analyzer.

Step 1: Get the Right Data

The Strategy Analyzer can run on bars built from minute data, daily data, or tick data. Minute bars process quickly and suit strategies with wide stops, but the order of fills inside each bar is assumed, not recorded. Tick data records each trade and gives a better fill estimate, which matters when a one or two tick difference changes the outcome. It takes longer to process. Your data provider or broker feed supplies the history, so check how far back it goes.

Futures need a decision about contract months. Test on the front-month contract with a merge policy set up in the instrument settings so the history splices across expirations. If your test window comes back much shorter than you expected, suspect a broken rollover merge, not a data limit. Also don't assume roll dates or sessions are the same across products. Check the CME product and holiday schedule.

Use enough history to include different conditions: quiet stretches, trending stretches, and sharp volatility. A window of only one kind of market will flatter a strategy built for that kind. See market regimes for how to label them.

Step 2: Set Costs Before You Run

Open the Strategy Analyzer from the Control Center, then choose the strategy, instrument, date range, and data series. Enter your real commission per contract, since a zero default makes everything look better. Add slippage. One tick is a reasonable starting assumption for a liquid contract in normal hours, and more for market orders or thin overnight trading. These are assumptions to stress, so run it at two or three levels.

Here's why it matters, in dollars. On ES a tick is $12.50. A strategy that earns $10 per trade gross, pays $5 round-turn commission, and gives up one tick of slippage nets $10 minus $5 minus $12.50, which is negative $7.50 per trade. The gross number looked fine and the strategy loses money.

Step 3: Read the Right Numbers

Profit factor is gross profit divided by gross loss. Below 1.0 the strategy lost money. Above 1.0 it made some, but how much is meaningful depends on trade count and costs. Very high values on a short sample are a prompt to look for overfitting, not a reason to celebrate.

Expectancy is win rate times average win minus loss rate times average loss. Hypothetical: a 45% win rate with a $300 average win and $100 average loss gives 0.45 times 300 minus 0.55 times 100, which is 135 minus 55, or $80 per trade before costs. Subtract commission and slippage and see what is left. A high win rate alone says little. A 75% win rate with a small win and a big loss can have negative expectancy.

Max drawdown is the largest peak-to-trough drop in the test. If it is bigger than what your account or a firm's loss limit can absorb, the strategy would have been stopped out before the recovery. Check max consecutive losers as well, because you need to know whether you'd stick with it through that streak. Sharpe ratio compares return to volatility. Treat it as a comparison tool between versions of your own test, not as a pass mark.

Trade count matters more than any single ratio. Thirty trades is not a statistically proven edge, and a few hundred trades across several conditions tells you more. No fixed number guarantees anything.

Curve Fitting

The optimizer can test hundreds of parameter sets in minutes. If you search long enough, some combination will fit the history. Warning signs:

1. The best settings are oddly specific, and nearby values perform much worse.
2. One sub-period carries nearly all the profit.
3. Adding a few weeks of new data changes the 'best' parameters.
4. The trade count is small.
5. You can't explain why the rule should work, only that it did.

A real effect usually holds across a neighborhood of values. Check that the results change gradually when you nudge a setting up or down.

A Testing Procedure

1. Choose the in-sample and out-of-sample split before you start, for example the first 70% of the data for design and the last 30% held back.
2. Optimize or tune on the in-sample part only, then lock the settings.
3. Run the locked settings on the held-back data once, with costs. If results fall apart, you found noise.
4. Use NinjaTrader's walk-forward option for a stricter version, where the strategy is re-optimized on a rolling window and tested on the next one.
5. Test different cost assumptions, such as double the slippage, and see whether it survives.
6. Run it on SIM or replay data and compare to the backtest, remembering a simulated fill is not an exchange queue or real slippage.

What a Backtest Can't Tell You

It can't tell you how you'll behave during a drawdown, whether your orders would have filled, or whether the future resembles the past. A backtest is a filter for ideas, not a promise. YMI bot behavior is documented in the Marty bot specs and the KPL bot specs, so read those instead of assuming test results from a blog post. The edge calculator is a quick way to check expectancy math on your own numbers. The NinjaTrader help guide at the NT8 help guide covers Strategy Analyzer settings in detail.