Skip to main content

Backtest Design Principles

Snowtrail backtests are built to provide institutional-grade performance analysis that avoids common pitfalls in signal evaluation.

Signal Architecture​

Each product has its own signal architecture tailored to the market it covers. The number and type of signals vary by product:


  • GBSI-US: uses two signals: system stress (fundamentals-only regime) and price context (has price already moved?). The positioning signal is withheld until it is re-sourced from the NYMEX NG futures contract.
  • GBSI-EU: uses four signals: system stress, composite (multi-factor setup), dispersion (cross-country divergence), and price context.
  • GLMI: uses two signals: marginality (which basin sets the marginal LNG price?) and price context (JKM-TTF spread dynamics).
  • PEMI: uses three signals: grid stress (outage exposure), event escalation (outage worsening), and supply pressure. Note that supply pressure has no weather input, despite its former name; it is a convenience flag equivalent to outage_exposure_level >= 2.
  • WRSI: uses three signals: forecast stress (forecast-instability regime), seasonal context (does weather matter now?), and model agreement (cross-model consensus).
  • WSSI-US: uses storage attribution (weather-attributable share of the weekly storage change) and three signals: demand shock (forecast demand anomaly, the graded series), seasonal context, and model agreement.

A common design principle applies across all products: the primary signal is fundamentals-only. Price is treated as an outcome to be contextualised, never as an input to the core regime signal. Where a price context signal exists, it is evaluated separately so the fundamental signal can be assessed on its own merits.

Robustness Approach​

Backtests are designed to resist overfitting and survivorship bias:

  • Point-in-time inputs: Signals are computed using expanding or rolling windows over data that was knowable at the time. Where a descriptive secondary metric's truth window overlaps the period before a row is knowable, the backtest row says so (for example GBSI-US carries lookahead_flag = True on its descriptive secondaries).
  • Block bootstrap confidence intervals: Metrics include uncertainty bounds derived from block resampling, preserving time-series autocorrelation.
  • Autocorrelation-adjusted p-values: Statistical tests account for serial dependence in regime classifications and outcomes.
  • Two-null significance rule: Since 2026-09-29 a headline is significant only if both the block-permutation p-value and the structure-preserving null p-value (headline_p_value_structure) are below 0.05.
  • False Discovery Rate (FDR): Multiple testing corrections are applied when evaluating metrics across regimes and directions.
  • Minimum sample requirements: Regime-direction combinations with insufficient observations are flagged rather than reported as significant.

Institutional Metrics​

Backtests focus on metrics relevant to institutional decision-making:

  • Vol Multiplier: The ratio of realised volatility during stressed regimes to calm regimes. Published as a descriptive column; no product's headline test supports it as a position-sizing or VaR input.
  • Tail Lift: The relative probability of extreme price moves during stressed vs calm periods. A descriptive column; not a validated VaR or CVaR input for any product.
  • Decision Lens: Per (regime, direction) observed characteristics, enabling nuanced interpretation of what each regime state means in practice.
  • Failure Analysis: False stress rate, missed stress rate, and worst-period analysis provide transparency around where and when the signal underperforms.

What Backtests Do Not Measure​

Backtests are not designed to evaluate price prediction accuracy. They measure whether regime classifications correspond to meaningfully different market conditions, not whether they forecast returns.