Backtests
Backtests provide pre-computed, point-in-time correct performance analysis for each Snowtrail signal. They are designed to answer the question: how well does this signal capture the market condition it was built to detect?
Golden Rule
Backtest against the question the product answers.
Each product's backtest measures whether the signal's regime classifications correspond to meaningfully different market outcomes, not whether they predict price direction.
Available Datasets
Every product provides three backtest datasets:
| Dataset | Description | SDK Method |
|---|---|---|
| Summary | Pre-computed backtest metrics including regime persistence, flip-flop rate, and vol multiplier | backtest_summary() |
| Outcomes | Per-regime outcome metrics: realised volatility, vol multiplier, tail lift | backtest_outcomes() |
| Transitions | Regime transition matrix showing probabilities of moving between states | backtest_transitions() |
GLMI additionally provides backtest_combinations() for two-signal analysis (4 basin regimes x 3 price contexts).
Key Metrics
- Vol Multiplier: How much more volatile the market is during stressed regimes vs calm regimes
- Tail Lift: Probability of extreme moves during stressed vs calm regimes
- Regime Persistence: How stable regime classifications are over time
- Flip-Flop Rate: Frequency of rapid regime changes (lower is better)
Methodology
Backtests use:
- Walk-forward design: No future data used in any calculation
- Block bootstrap confidence intervals: Statistical robustness for metrics
- Block permutation tests: The headline p-value is computed by permuting whole blocks, so serial correlation survives the null. Block permutation is used rather than block bootstrap because a bootstrap holds subgroup size fixed and therefore cannot detect sample-size artefacts on heavy-tailed data.
- Add-one p-value estimator:
(1 + exceedances) / (1 + permutations). A resampling test cannot support a p-value of exactly zero; where no permutation is as extreme as the observation the honest statement isp < 1/n. - False Discovery Rate (FDR): Controls for multiple testing
- Minimum sample enforcement: A backtest below the minimum refuses to emit metrics rather than returning placeholder values that read as a genuine null.
- Native-grain evaluation: Each product is scored at the frequency it actually updates. A monthly signal merged onto daily prices would otherwise multiply one observation into ~21 and inflate significance.
Example
from snowtrail import Snowtrail
client = Snowtrail(api_key="your-api-key")
# Get backtest summary for GBSI-US.
# Read the headline_* columns first -- they carry the ONE statistic this
# product's claim rests on, and they are named identically across all six
# products so you can compare like with like.
summary = client.gbsi_us.backtest_summary()
print(summary[["backtest_date", "headline_metric", "headline_value",
"headline_p_value", "headline_direction", "flip_flop_rate_2w"]])
# Get per-regime outcomes
outcomes = client.gbsi_us.backtest_outcomes()
print(outcomes[["regime_label", "sample_n", "vol_multiplier", "vol_2d_median"]])
# Get transition matrix
transitions = client.gbsi_us.backtest_transitions()
print(transitions)