Commit Graph
2 Commits
Author SHA1 Message Date
sam 67d016fe25 Forward test: record forecasts before their outcomes exist.
`snapshot` writes each tracked model's forecast quantiles at the seven
backtest horizons from the latest price to data/forecasts/<origin>.csv. It
refuses stale data (older than two days) and duplicate dates, so snapshots
can't be reconstructed after the fact; committing them dates them.
`forward` scores every recorded forecast whose target date has passed,
reusing the backtest's scoring (now factored out as evaluate.score).

Tracked: random_walk, drift_rw, cycle, powerlaw, plus powerlaw_ou,
powerlaw_ou_param and cycle_on_powerlaw, which development data couldn't
settle. `just weekly` runs update, snapshot and forward.

First snapshot: 2026-09-23 (BTC $84.4K). The first outcomes are due
2026-10-23.
2026-09-24 03:08:30 -07:00
sam cfc27a38de Rewrite as a probabilistic model with walk-forward evaluation.
Replace the 2024 model (model.py, ~2000 lines) with the btcmodel package, the
baseline for future work:

- Forecasts are quantiles of log price at each horizon, scored with CRPS in a
  walk-forward backtest (origins every 30 days from 2014, horizons 1 month to
  4 years). Skill is relative to a zero-drift random walk, with circular
  block-bootstrap intervals and a count of independent windows.
- Development data stops at 2024-11-26, the last day the 2024 model saw.
  Later outcomes are a holdout, scored only by `backtest --holdout`.
- Models: random_walk, drift_rw, and cycle (the 2024 model's cycle-position
  drift, now kernel-smoothed and recency-weighted). On development data
  nothing beats the random walk with confidence; cycle loses at every horizon.
- Prices: the Investing.com archive moves to data/ (cut at 2024-11-26; its
  last row was intraday) and is extended with Coinbase daily closes by
  `update`.

Also: Nix flake dev shell (Python 3.13, pandas 3), ruff in place of black,
pytest suite, and a rewritten README. NOTES.md is removed as inaccurate, and
poetry is dropped.
2026-09-24 02:19:02 -07:00