How to Convert a Discretionary Observation Into a Testable Trading Rule

Experienced traders often notice real patterns before they can say what the pattern is. 'It looks like it wants to go higher' may be a genuine read, but it is not a rule: you can't test it, hand it to someone else or execute it the same way twice under stress. Converting the observation into precise terms is how you find out whether there is anything there.

Step 1: write the observation precisely

Start in plain language and keep adding specifics until every term can be measured. Weak: 'ES rallies after big down moves.' How big, how long, how much of a rally? Better: 'When ES opens at least 15 points below the prior regular-session close, it recovers at least half the gap within 90 minutes of the 9:30 am ET open.'

Notice what is missing: any frequency. Don't write one in until you have counted it. The example here is an illustration of the process, not a validated setup.

Step 2: define entry, stop, target and time

Every component should become a yes or no question during live trading. For the gap example:

• Entry: buy when a 5-minute bar closes above the prior close minus 7.5 points (half of the 15-point minimum gap).
• Stop: below the opening bar's low, with a minimum distance you choose, for instance 4 points.
• Target: the prior close, or at least 2 times the risk, whichever you define.
• Time filter: no entries after 10:30 am ET. If the trigger hasn't happened, skip.

If the trade comes down to 'it looks like it might work', the rule isn't finished.

Step 3: candidate disqualifiers

Most patterns fail in certain contexts. Write down conditions that you think should disqualify the setup, and be clear that these are hypotheses to test, not discoveries. For the gap example, candidates might be a major scheduled release near the open, an unusually large gap, very high volatility, or a gap that comes with a clean break of a major multi-day level. Each filter shrinks the sample, so add one only for a market-mechanics reason you can state in a sentence, and test whether it helps.

Step 4: test on historical data

In NinjaTrader 8 you can run Market Replay on stored data or write the rule as a NinjaScript strategy and backtest it in the Strategy Analyzer. A Playback connection and a SIM account are separate pieces, so confirm both are set up before assuming your replay orders are routing where you expect. If you can't code, a precise written specification is something a developer can build from.

For each instance record entry, stop, target, result, fees and maximum adverse excursion (MAE): the worst move against you before the trade resolved. MAE shows whether stops are in the right place. If many winners come within a few ticks of the stop, the stop is too tight to survive normal movement. If winners almost never go against you at all, the stop may be wider than it needs to be.

Use the same data fairly: realistic commissions, a conservative slippage assumption and no peeking. A simulated fill is not a record of where you would have been in the exchange queue, so be skeptical of any rule that only works at the touch price.

Step 5: evaluate without overfitting

Compute expectancy from your tested trades. The edge calculator does the arithmetic. Then be honest about sample size: 30 to 50 instances is enough to say a rule is worth more testing, not enough to say it works. A positive number from a small sample can come from luck, and noisy samples are the usual way people fall in love with something that isn't there.

Revising a rule is normal. The danger is changing it to make the history look better. A filter added because you can explain the market mechanism is a hypothesis. A filter that excludes Tuesdays because Tuesdays happened to lose in your sample is a fit to noise. Guard against it by deciding in advance how many variations you will try, and by holding back a stretch of data you only look at once the rules are final. If a rule only works after many adjustments and fails on the held-back period, treat that as a negative result.

Step 6: write it down

Document the rule so another person, or you in a bad mood, would make the same decisions. A usable format: name, setup preconditions, entry trigger, position size method, stop, target, time filter, disqualifiers, test results with sample size and date range, and known failure modes. Update it when you change it, and keep the old versions.

A rule with a fixed formula is easier to test than one that depends on discretion. The YMI Key Price Levels are an example of a fixed reference built from defined inputs: four levels anchored to the 9:30 am ET opening print that do not move intraday. The method is outlined on the KPL trading page. Whatever rule you build, the discipline is the same: define it, test it, record the costs and be willing to throw it away.