bitcoin-model/NOTES.md

# Bitcoin Price Model Documentation

## Model Overview
A probabilistic price projection model combining log returns analysis, cycle awareness, and Monte Carlo simulation. The model generates projected price ranges with confidence intervals, balancing short-term market dynamics with long-term cyclical patterns.

## Core Design Principles

### 1. Return Analysis
- Uses log returns for better handling of exponential growth
- Combines multiple timeframes for volatility estimation
- Implements adaptive window sizing based on market conditions
- Handles volatility clustering through regime-aware adjustments

### 2. Cycle Integration
- Recognizes Bitcoin's ~4 year (1460 day) halving cycle
- Maps historical returns to cycle positions (0-1 scale)
- Adjusts expectations based on position in cycle
- Handles transitions between cycles with uncertainty scaling

### 3. Market Era Recognition
Three distinct eras with specific characteristics:
- Early (2013-2017): Higher base volatility, conservative trends
- Transition (2017-2020): Futures market introduction period
- Mature (2020+): Institutional participation, reduced base volatility

### 4. Uncertainty Estimation
- Generates both point estimates and confidence intervals
- Adapts uncertainty based on market conditions
- Uses asymmetric volatility response
- Implements dynamic confidence interval calibration

## Architecture

### Key Components
1. **Trend Analysis (`analyze_trends`)**
   - Calculates cycle-position-specific returns
   - Applies position-aware smoothing
   - Handles cycle boundaries

2. **Volatility Estimation (`calculate_volatility`)**
   - Adaptive window sizing
   - Multi-timeframe integration
   - Era-specific scaling
   - Regime detection and response

3. **Price Projection (`project_prices`)**
   - Monte Carlo simulation engine
   - Dynamic uncertainty scaling
   - Confidence interval calculation
   - Trend integration

4. **Projection Adjustment (`get_projection_adjustments`)**
   - Time-varying uncertainty scaling
   - Market condition response
   - Cycle position awareness
   - Minimum uncertainty bounds

# Model Performance & Validation

## Performance Characteristics

### Normal Market Conditions
- MAPE: 30-40% typical
- 95% CI Coverage: ~95%
- 68% CI Coverage: ~73%
- Best performance in mature market periods (2020+)
- Most reliable for 3-6 month horizons

### Stress Periods
- MAPE: 30-60%
- 95% CI Coverage: ~95%
- 68% CI Coverage: ~76%
- Wider but well-calibrated confidence intervals
- Maintains reliability through increased uncertainty

### Key Strengths
1. Consistent confidence interval coverage
2. Rapid adaptation to volatility changes
3. Robust handling of cycle transitions
4. Well-calibrated uncertainty estimates

### Known Limitations
1. Higher error during market structure changes
2. Increased uncertainty in early cycle periods
3. Limited incorporation of external factors
4. May underestimate extreme events

## Validation Framework

### Backtest Configuration
- Minimum training period: 8 years
- Validation period: 2 years
- Rolling window approach
- Separate evaluation of normal/stress periods

### Key Test Periods
1. **Cycle Transitions**
   - Pre/post halving periods
   - Historical halvings (2016, 2020, 2024)
   - Cycle peak/trough transitions

2. **Market Structure Changes**
   - Futures introduction (2017)
   - Institution adoption (2020-2021)
   - Major market events (e.g., COVID crash)

3. **Recent History**
   - 2021 bull market
   - 2022 drawdown
   - 2024 recovery

### Validation Metrics
1. **Accuracy Measures**
   - MAPE (Mean Absolute Percentage Error)
   - RMSE (Root Mean Square Error)
   - Maximum deviation

2. **Calibration Measures**
   - Confidence interval coverage
   - Uncertainty estimation accuracy
   - Regime transition handling

3. **Stability Measures**
   - Parameter sensitivity
   - Training period dependence
   - Regime change response

# Technical Implementation

## Core Functions

### Volatility Calculation
```python
def calculate_volatility(df, short_window=30, medium_window=90, long_window=180):
    """
    Adaptive volatility calculation combining multiple timeframes.

    Features:
    - Dynamic window sizing based on market conditions
    - Era-specific scaling factors
    - Regime-aware adjustments
    - Robust error handling and fallbacks
    """
```

Key parameters:
- `short_window`: Fast response (default 30 days)
- `medium_window`: Primary estimate (default 90 days)
- `long_window`: Stability baseline (default 180 days)

Adaptive features:
- Windows shrink in high volatility periods
- Expand during low volatility
- Minimum size constraints for stability
- Weighted combination based on regime

### Cycle Position
```python
def get_cycle_position(date, halving_dates):
    """
    Calculate position in halving cycle (0 to 1).
    0 = halving event
    1 = just before next halving
    """
```

Position calculation:
- Linear interpolation between halvings
- Special handling for pre-first-halving
- Extension mechanism for future cycles
- Built-in boundary condition handling

### Price Projection
```python
def project_prices(df, days_forward=365, simulations=1000,
                  confidence_levels=[0.95, 0.68]):
    """
    Generate price projections with confidence intervals.

    Core simulation parameters:
    - Number of paths: 1000
    - Confidence levels: 95% and 68%
    - Dynamic uncertainty scaling
    """
```

## Data Requirements

### Input Data
Minimum fields:
- Date
- Close price
- Trading volume (optional)
- High/Low (optional)

Format requirements:
- Daily data preferred
- Sorted chronologically
- No missing dates
- Prices > 0

### Training Data
Minimum requirements:
- 2 years for basic operation
- 8 years recommended
- Must include at least one cycle transition
- Should span multiple market regimes

## Error Handling

### Data Validation
- Missing value detection and interpolation
- Outlier identification
- Zero/negative price handling
- Volume anomaly detection

### Runtime Guards
- Minimum data length checks
- Window size validation
- Numerical stability checks
- Regime transition handling

### Fallback Mechanisms
1. Simple volatility calculation
2. Default uncertainty estimates
3. Conservative parameter sets
4. Standard cycle assumption

## Memory and Performance

### Optimization Features
- Efficient numpy operations
- Vectorized calculations where possible
- Smart data windowing
- Caching of intermediate results

### Resource Usage
Typical requirements for 10-year dataset:
- Memory: ~100MB
- CPU: ~2-5 seconds per projection
- Storage: Negligible

### Parallelization
- Multiprocessing support for backtests
- Independent path simulation
- Multiple period analysis
- Backtest parallelization

# Development History & Evolution

## Major Versions

### Version 1.0 (Initial Implementation)
- Basic log return analysis
- Fixed volatility windows
- Simple cycle position calculation
- Base Monte Carlo simulation

### Version 2.0 (Market Structure)
- Added era-based adjustments
- Improved cycle handling
- Multiple timeframe volatility
- Enhanced Monte Carlo engine

### Version 3.0 (Current)
- Adaptive volatility windows
- Dynamic uncertainty scaling
- Improved regime detection
- Enhanced confidence interval calibration

## Key Improvements

### Volatility Estimation
1. **Fixed → Adaptive Windows**
   - Initial: Fixed 30/90/180 day windows
   - Current: Dynamic sizing based on regime
   - Result: Better regime transition handling

2. **Uncertainty Calibration**
   - Initial: Fixed scaling factors
   - Current: Market-aware dynamic scaling
   - Result: More reliable confidence intervals

3. **Era Recognition**
   - Initial: Single model for all periods
   - Current: Era-specific adjustments
   - Result: Better handling of market evolution

### Simulation Engine
1. **Path Generation**
   - Initial: Basic random walks
   - Current: Regime-aware path simulation
   - Result: More realistic price trajectories

2. **Confidence Intervals**
   - Initial: Fixed width
   - Current: Dynamic, asymmetric intervals
   - Result: Better calibrated uncertainty

## Failed Experiments

### 1. Complex Regime Detection
- Attempted multiple indicator fusion
- Added excessive complexity
- Reduced model stability
- Reverted to simpler approach

### 2. Machine Learning Integration
- Tested neural network components
- Reduced interpretability
- Inconsistent improvements
- Kept traditional statistical approach

### 3. External Factor Integration
- Tried incorporating macro indicators
- Added noise to projections
- Complicated parameter estimation
- Maintained focus on price dynamics

## Recent Improvements (2024)

### Adaptive Volatility Windows
- Implementation: Dynamic window sizing
- Purpose: Better regime handling
- Results:
  - Improved 95% CI coverage to ~95%
  - Better stress period handling
  - More reliable uncertainty estimates

### Performance Metrics
Normal Periods:
- MAPE: 39.9%
- RMSE: $12,007
- 95% CI Coverage: 95.9%
- 68% CI Coverage: 72.5%

Stress Periods:
- MAPE: 32.8%
- RMSE: $12,794
- 95% CI Coverage: 95.2%
- 68% CI Coverage: 76.1%

## Future Directions

### Short Term
1. Fine-tune adaptive parameters
2. Improve transition period handling
3. Enhanced backtest framework
4. Additional regime indicators

### Medium Term
1. Cycle strength indicators
2. Volume analysis integration
3. Improved documentation
4. Performance optimization

### Long Term
1. Real-time adaptation framework
2. Advanced regime detection
3. Market microstructure integration
4. External API integration