Add statistical models comparison report

MODEL_COMPARISON.md

@@ -0,0 +1,76 @@
+# M5 Forecasting: Statistical Models Comparison
+
+## Performance Summary
+
+| Model          | RMSE      | MAE       | MAPE    | Model Size | Repo |
+|----------------|-----------|-----------|---------|------------|------|
+| **SARIMAX**    | 2759.70 | 2260.25 | 4.98% | 85.30 MB | [rishini/NPN-sarimax](https://huggingface.co/rishini/NPN-sarimax) |
+| **Prophet**    | 4860.67 | 4038.73 | 8.73% | 0.18 MB | [rishini/NPN-prophet](https://huggingface.co/rishini/NPN-prophet) |
+| **ARIMA**      | 6459.70 | 4852.62 | 10.48% | 6.56 MB | [rishini/NPN-arima](https://huggingface.co/rishini/NPN-arima) |
+| **LightGBM** (per-series) | N/A (WRMSSE=145.56) | | | 106.1 MB | [rishini/NPN](https://huggingface.co/rishini/NPN) |
+
+## Key Findings
+
+### 1. SARIMAX Wins (Best Accuracy)
+- **Lowest RMSE**: 2,759.70 (4.98% MAPE)
+- **Best fit**: SARIMAX(2,1,1)(1,1,1,7) captures weekly seasonality
+- **Exogenous boost**: SNAP indicators and event dummies improve predictions
+- **Trade-off**: Largest model file (85MB) due to complex state space representation
+
+### 2. Prophet (Best Interpretability)
+- **RMSE**: 4,860.67 (8.73% MAPE)
+- **Strengths**: Fast training, automatic seasonality detection, built-in uncertainty intervals
+- **Weaknesses**: Underperforms on aggregate-level predictions
+- **Trade-off**: Smallest model (180KB), fastest to deploy
+
+### 3. ARIMA (Baseline Simplicity)
+- **RMSE**: 6,459.70 (10.48% MAPE)
+- **Best config**: ARIMA(3,1,1) with deterministic trend
+- **Strengths**: Simple, interpretable, smallest non-Prophet model
+- **Weaknesses**: No seasonality, no exogenous variables, poorest fit
+
+### 4. LightGBM (Per-Series Champion)
+- **WRMSSE**: 145.56 (beats naive baselines by 55-69%)
+- **Advantage**: Predicts all 30,490 series individually
+- **Trade-off**: Not directly comparable (different granularity)
+
+## Why SARIMAX Outperforms?
+
+1. **Weekly Seasonality**: Retail demand has strong 7-day cycles (weekends higher)
+2. **Exogenous Signals**: SNAP eligibility and events directly impact demand
+3. **Autocorrelation**: Captures persistence in sales patterns
+4. **Differencing**: (d=1) removes trend, focusing on changes
+
+## Why These Models Don't Beat LightGBM?
+
+| Aspect | LightGBM | Statistical Models |
+|--------|----------|-------------------|
+| Granularity | 30,490 individual series | 1 aggregate series |
+| Features | 34 engineered features | 6-12 basic features |
+| Flexibility | Non-linear relationships | Linear/AR/MA assumptions |
+| Cross-series learning | Store/dept/item interactions | No sharing across series |
+| Deployment | 40 per-store models | 3 aggregate models |
+
+The statistical models operate on **aggregate** sales (total ~34,000/day), while LightGBM models each of the **30,490 individual series** with specialized features. The statistical approach serves as a solid baseline but cannot match the per-series precision of gradient boosting.
+
+## Model Selection Guide
+
+Use **SARIMAX** when:
+- You need the best statistical baseline
+- Exogenous variables (events, promotions) are important
+- Weekly seasonality dominates
+
+Use **Prophet** when:
+- Fast experimentation is needed
+- Interpretability is key
+- Multiple seasonalities exist
+
+Use **ARIMA** when:
+- Simple baseline is sufficient
+- No strong seasonality
+- Minimal computational budget
+
+Use **LightGBM** when:
+- Maximum accuracy is required
+- Per-series predictions needed
+- GPU is available