Update model card: leak-free training, honest backtest metrics

README.md

@@ -7,79 +7,62 @@ tags:
 - lstm
 - multi-task
 - multi-domain
-- text-generation
-pipeline_tag: text-generation
+pipeline_tag: time-series-forecasting
 ---
 
 # Future Prediction Models (Multi-Domain LSTM)
 
-Trained PyTorch LSTM checkpoints forecasting 7 daily time-series domains (AI/NVIDIA, Programming/npm, Finance/BTC, Sports/ATP Elo, Weather/Chennai, Economy/S&P500, Energy/WTI).
+Trained PyTorch LSTM checkpoints forecasting 7 daily time-series domains (AI/NVIDIA, Programming/npm, Finance/BTC, Sports/ATP Elo, Weather, Economy/S&P500, Energy/WTI).
+
+Trained with a leak-free protocol: chronological TRAIN/VALIDATION/TEST splits, validation-only early stopping and tuning, untouched test set, and walk-forward backtesting as the headline metric.
 
 ## Checkpoints
 
-All models predict 7 steps from a 60-step window of base-normalized values `x_t = value_t / value_{t-1} - 1`.
+All models predict 7 steps from a 60-step window. Inputs are 7 causal features per timestep: base-relative value, 1-step return, MA7 ratio, MA30 ratio, 7-step volatility, day-of-week sin/cos.
 
-| File | Parameters (hidden, layers, dropout) |
+| File | Architecture |
 |---|---|
-| `unified_model.pt` | Multi-task: shared LSTM (128, 2) + domain embedding (16) + 7 heads; domains: ai, programming, finance, sports, weather, economy, energy |
-| `model_ai.pt` | 128 hidden, 3 layers, dropout 0.1 |
-| `model_finance.pt` | 128 hidden, 3 layers, dropout 0.1 |
-| `model_programming.pt` | 128 hidden, 3 layers, dropout 0.1 |
-| `model_sports.pt` | 128 hidden, 3 layers, dropout 0.1 |
-| `model_economy.pt` | 64 hidden, 2 layers, dropout 0.1 |
-| `model_energy.pt` | 64 hidden, 2 layers, dropout 0.1 |
-| `model_weather.pt` | 64 hidden, 2 layers, dropout 0.1 |
-
-## Model architecture (reference)
+| `unified_model.pt` | Multi-task: shared LSTM (128 hidden, 2 layers) + domain embedding (16) + 7 heads; domains: ai, programming, finance, sports, weather, economy, energy |
+| `model_ai.pt` | LSTM 64 hidden, 2 layers |
+| `model_programming.pt` | LSTM 64 hidden, 2 layers |
+| `model_finance.pt` | LSTM 64 hidden, 2 layers |
+| `model_sports.pt` | LSTM 64 hidden, 2 layers |
+| `model_weather.pt` | LSTM 64 hidden, 2 layers |
+| `model_economy.pt` | LSTM 64 hidden, 2 layers |
+| `model_energy.pt` | LSTM 64 hidden, 2 layers |
 
-```python
-class LSTMForecaster(nn.Module):
-    def __init__(self, input_size=1, hidden_size=128, num_layers=3, dropout=0.1, horizon=7):
-        ...
-        self.lstm = nn.LSTM(input_size, hidden_size, num_layers, batch_first=True, dropout=...)
-        self.head = nn.Sequential(nn.Linear(hidden_size, hidden_size), nn.ReLU(),
-                                  nn.Dropout(dropout), nn.Linear(hidden_size, horizon))
-    def forward(self, x):  # x: (B, 60, 1)
-        out, _ = self.lstm(x)
-        return self.head(out[:, -1, :])  # (B, 7)
-```
+## Honest performance (walk-forward backtest vs naive persistence)
 
-The unified model shares the LSTM across all 7 domains, adds a 16-dim domain embedding, and keeps one output head per domain (input and output shapes are the same as above).
+Positive = model beats "tomorrow equals today" baseline.
 
-## Validated accuracy (held-out test split, 1-day-ahead MAPE)
+| Topic | vs naive (separate) | vs naive (unified) | Verdict |
+|---|---|---|---|
+| Programming | +71% | +68% | Real edge (weekly seasonality) |
+| AI | +2.3% | +1.6% | Small edge |
+| Energy | +1.8% | +0.9% | Small edge |
+| Economy | +0.3% | +2.0% | Marginal |
+| Weather | -1.3% | -0.2% | No edge |
+| Finance | -4.3% | -2.3% | Trails naive |
+| Sports | -11.9% | -22.8% | Trails naive |
 
-| Topic | Separate | Unified |
-|---|---|---|
-| AI | 1.86% | 2.36% |
-| Programming | 7.12% | 5.42% |
-| Finance | 1.52% | 1.74% |
-| Sports | 0.06% | 0.04% |
-| Weather | 0.18% | 0.18% |
-| Economy | 0.64% | 0.75% |
-| Energy | 2.83% | 1.90% |
+Directional accuracy is ~50-55% on financial series (barely better than chance). Sports direction is mostly undefined because Elo is flat on no-match days.
 
-## Loading and inference
+## Loading
 
 ```python
 import torch
 
 model = torch.load("model_ai.pt", weights_only=False)
 model.eval()
-x = torch.randn(1, 60, 1)          # last 60 normalized returns
+x = torch.randn(1, 60, 7)          # last 60 steps of the 7 causal features
 pred = model(x)                    # (1, 7) predicted returns over next 7 steps
 ```
 
-Example — reconstruct absolute values from predicted returns:
-
-```python
-preds = model(x)[0]                                  # 7 predicted returns
-forecast = current_value * torch.cumprod(1 + preds, 0)  # 7-day forecast
-```
-
 ## Notes
 
-- Forecasts are statistical estimates validated on held-out data; no model can predict the future with 100% accuracy.
-- Trained on CPU (PyTorch); goal was accuracy, not speed.
+- Forecasts are statistical estimates; no model predicts the future reliably.
+- Uncertainty bands shipped with predictions come from validation residual std (+/-1.96 sigma), not invented confidence.
+- Where the model does not beat the persistence baseline, that is reported rather than hidden.
 
 ## License