Forward test: record forecasts before their outcomes exist.
`snapshot` writes each tracked model's forecast quantiles at the seven backtest horizons from the latest price to data/forecasts/<origin>.csv. It refuses stale data (older than two days) and duplicate dates, so snapshots can't be reconstructed after the fact; committing them dates them. `forward` scores every recorded forecast whose target date has passed, reusing the backtest's scoring (now factored out as evaluate.score). Tracked: random_walk, drift_rw, cycle, powerlaw, plus powerlaw_ou, powerlaw_ou_param and cycle_on_powerlaw, which development data couldn't settle. `just weekly` runs update, snapshot and forward. First snapshot: 2026-09-23 (BTC $84.4K). The first outcomes are due 2026-10-23.
This commit is contained in:
@@ -36,3 +36,14 @@ clean:
|
||||
# Run A/B experiments on development data (default: all)
|
||||
ab *names:
|
||||
python -m btcmodel ab {{ names }}
|
||||
|
||||
# Record tracked models' forecasts for the forward test (commit the new file)
|
||||
snapshot:
|
||||
python -m btcmodel snapshot
|
||||
|
||||
# Score recorded forecasts whose target dates have passed
|
||||
forward:
|
||||
python -m btcmodel forward
|
||||
|
||||
# The weekly routine: new prices, a new snapshot, and scores so far
|
||||
weekly: update snapshot forward
|
||||
|
||||
Reference in New Issue
Block a user