Known caveats & honest limits

Every number in the Workbench lives under the caveats on this page. None of them make the data useless — but pretending they don't exist would.

All of it is hypothetical

Backtest results are hypothetical performance: computed after the fact, with the benefit of complete data, under stated modeling conventions, without the capital, nerves, or fat-finger risk of live trading. Hypothetical results are systematically flattering. Past performance — simulated or real — does not indicate future results. The Workbench is analytics and education software; nothing in it is investment advice or a recommendation.

Selection bias is built into catalog scale

With 245,250 parameter combinations, the top of any sorted column looks spectacular — some combination always won the past by chance. We publish a demonstration of exactly this: a cohort of top-10 in-sample strategies that earned $96.23/day in-sample went on to earn $3.33/day out-of-sample — for that slice, the in-sample ranking carried no forward information.

The tools that exist because of this:

• Confluence — does the parameter neighbourhood confirm the result, or is it an isolated spike?
• Longer periods — more days, more regimes, less luck.
• Monte Carlo — the distribution, not the single path.
• The app's own framing rule, worth repeating: a sort order reflects the statistic you sorted by — it is not a recommendation.

Short windows are noise

A 1W period is five trading days. Metrics computed on a handful of days — including MAR, which divides two noisy numbers — swing violently and mean almost nothing. The short windows exist for inspecting recent behavior, not for grading strategies. Grade on long windows; glance at short ones.

Conventions that shape the numbers

• Win rate counts spread-legs (a both-sided iron condor is two trades per day) — consistent everywhere in the app, but a different convention than some tools use; don't compare raw win rates across tools.
• Stopped losses are recorded at the stop level (the modeling convention); live stop fills can be worse — measured and quantified here.
• Sharpe and Sortino are computed on daily P&L (risk-free rate 0, annualized by √252). Same-name metrics elsewhere may use returns-based formulas — again, compare within the app, carefully across tools.
• METF variants have shorter history (from February 2023) than MEIC variants — their "Total" numbers cover fewer regimes.

Data freshness has a rhythm

New data lands roughly weekly and the whole grid is re-scored. The data through date in the app is the truth about what you're seeing; between re-scores, recent days simply aren't reflected yet. If a date looks a few days old, that's the cycle working as designed.

The bottom line

Use the Workbench to reject fragile constructions and to understand the historical shape of robust ones. What it cannot do — what nothing can do — is promise the future. Your account, your decisions, your responsibility.