Merge pull request #33 from ParallelLLC/dev

Merge Claude updates from dev: algotrader 2.0 and Yahoo ingest fixes

.github/workflows/sync-hf-space.yml

@@ -0,0 +1,52 @@
+name: Sync Hugging Face Space
+
+# Publishes app.py + the algotrader package to a Hugging Face Space.
+#
+# Setup (one time):
+#   1. Create the Space at https://huggingface.co/new-space with SDK "Gradio".
+#   2. Add repository secret HF_TOKEN — a write token from
+#      https://huggingface.co/settings/tokens
+#   3. Add repository variable HF_SPACE_ID, e.g. "your-username/backtest-reality-check".
+#
+# Without those set the job is skipped, so forks and PRs are unaffected.
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - 'app.py'
+      - 'algotrader/**'
+      - 'SPACE_README.md'
+      - 'requirements-space.txt'
+      - 'tests/test_v2_*.py'
+      - '.github/workflows/sync-hf-space.yml'
+  workflow_dispatch:
+
+jobs:
+  sync:
+    runs-on: ubuntu-latest
+    if: vars.HF_SPACE_ID != ''
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+
+      - name: Verify the Space actually runs before publishing it
+        run: |
+          pip install --quiet -r requirements-space.txt pytest
+          python -m pytest tests/test_v2_engine.py tests/test_v2_validation.py \
+                           tests/test_v2_strategies.py tests/test_v2_portfolio.py -q
+          ALGOTRADER_OFFLINE=1 python -c "import app; app.build_app(); print('Space builds OK')"
+
+      - name: Push to the Space
+        env:
+          HF_TOKEN: ${{ secrets.HF_TOKEN }}
+        run: |
+          if [ -z "$HF_TOKEN" ]; then
+            echo "HF_TOKEN secret is not set; skipping publish." >&2
+            exit 0
+          fi
+          chmod +x scripts/deploy_hf_space.sh
+          ./scripts/deploy_hf_space.sh "${{ vars.HF_SPACE_ID }}"

README.md

@@ -1,127 +1,195 @@
 # Algorithmic Trading
 
-FinRL reinforcement-learning trading with Alpaca execution, plus optional Yahoo Finance OHLCV for unlabeled real-price research. Parallel LLC.
+Parallel LLC. Two layers in one repository:
 
-This is **research and paper-trading infrastructure**. Live capital requires a separate evaluation contract, feature-parity tests, and a rewritten execution path. Do not treat `paper_trading: false` as a promotion gate.
+1. **algotrader 2.0** (`algotrader/`, `app.py`): a backtester that tries to prove a rule was luck (permutation, deflated Sharpe, PBO, walk-forward, cost stress).
+2. **Agentic v1** (`agentic_ai_system/`): FinRL policies, Yahoo or Alpaca ingest, paper/live execution, Streamlit/Dash/Jupyter UIs, Docker.
+
+Default market data is **Yahoo Finance** (`yfinance>=1.0`), not simulated prices. The simulator exists for offline tests (`--source synthetic` or `ALGOTRADER_OFFLINE=1` with `source=auto`). Live capital still needs a separate evaluation contract. This is research tooling, not investment advice.
 
 ---
 
 ## 1. Title and Summary
 
 **Algorithmic Trading**  
-Northwestern-trained data-engineering practice applied to a trading loop: ingest OHLCV, compute indicators or train a FinRL policy, size orders under position and drawdown caps, route to paper or live Alpaca.
+Ingest real OHLCV, test whether a timing or cross-sectional rule survives a hostile null, optionally train a FinRL policy, size orders under position and drawdown caps, route to paper or live Alpaca.
 
-GitHub `main` is the FinRL / Docker / Streamlit tree. `dev` is the integration branch. Yahoo is an additive `data_source.type`, not a replacement for Alpaca or FinRL.
+GitHub keeps two branches: `main` (protected) and `dev` (integration).
 
 **Design themes**
 
-* Four ingest paths: CSV replay, synthetic GBM, Alpaca REST, Yahoo (`yfinance>=1.0`)
-* FinRL policies (PPO, A2C, DDPG, TD3) on a Gymnasium-style environment
-* Alpaca for authenticated market data and order routing (paper by default)
-* Yahoo for delayed public bars when no broker key is available
-* Secrets from environment (`ALPACA_API_KEY`, `ALPACA_SECRET_KEY`), never committed
-* Tests and Docker/CI as already present on this tree
+* Yahoo as the default public tape (delayed, unofficial, lookback-limited)
+* Validation before belief: permutation, DSR, PBO/CSCV, walk-forward, 3× cost stress
+* FinRL (PPO, A2C, DDPG, TD3) unchanged on the v1 path
+* Alpaca optional for authenticated bars and orders; keys from the environment
+* Synthetic GBM / regime simulator only when requested
+* Secrets never in git
 
 ---
 
-## 2. Concepts and Methods
+## 2. Quick start
+
+```bash
+git clone https://github.com/ParallelLLC/algorithmic_trading.git
+cd algorithmic_trading
+python -m venv .venv && source .venv/bin/activate
+pip install -r requirements-space.txt   # algotrader + Gradio
+# or: pip install -r requirements.txt   # full v1 stack (FinRL, Dash, Docker CI)
+```
+
+```bash
+python app.py                                      # Gradio, localhost:7860, Yahoo by default
+python -m algotrader.cli lab --symbol SPY --strategy sma_cross
+python -m algotrader.cli lab --symbol NVDA --strategy rsi_reversion --permutations 500
+python -m agentic_ai_system.main --mode backtest --start-date 2024-01-01 --end-date 2024-12-31
+```
+
+`config.yaml` defaults:
 
-### Market data
+```yaml
+data_source:
+  type: 'yahoo'
+trading:
+  symbol: 'AAPL'
+  timeframe: '1d'    # Yahoo 1m history is ~7 days; use 1d for multi-year windows
+yahoo:
+  auto_adjust: true  # raw Close turns splits into fake crashes
+```
 
-| Source | When to use | Failure modes |
-| ------ | ----------- | ------------- |
-| **CSV** | Offline replay; default in `config.yaml` | Missing path or OHLCV columns → `None` |
-| **Synthetic** | Unit tests and demos | GBM is not tradable edge |
-| **Alpaca** | Authenticated bars and live/paper orders | Auth, feed, and rate-limit failures |
-| **Yahoo** | Real Close without a broker account | Unofficial API, ~15 min delay, interval lookback caps (1m ≈ 7 days). Pin `yfinance>=1.0`; 0.2.x fails against the current chart API |
+Alpaca is opt-in: `ALPACA_API_KEY` / `ALPACA_SECRET_KEY` and `data_source.type: alpaca` or `execution.broker_api: alpaca_paper`.
+
+---
+
+## 3. algotrader 2.0 (validation lab)
+
+Most backtests answer "how much would this have made?" This one asks **how much of that was luck?**
+
+### Two labs
+
+**The Lab** validates a timing rule on one asset. **The Portfolio Lab** validates a cross-sectional book that ranks many names.
+
+### The four ways a backtest lies
+
+| The lie | The test | Where |
+| --- | --- | --- |
+| The market had no structure to find | Monte-Carlo permutation (shuffle bar order, keep gap/high/low/body/volume) | `algotrader/validation/permutation.py` |
+| You tried 200 things and reported the best | Deflated Sharpe Ratio | `algotrader/validation/deflated_sharpe.py` |
+| Parameters were fitted to the past | PBO (CSCV) and walk-forward | `algotrader/validation/pbo.py`, `walkforward.py` |
+| The edge is smaller than the costs | Cost stress at 3× friction | `algotrader/lab.py` |
+
+Reality Score (0–100, grades A–F): significance 30%, selection 25%, walk-forward 20%, overfitting 15%, robustness 10%. The scale is harsh on purpose. Buy-and-hold and a coin-flip stay in the arena as controls.
+
+Cross-sectional books use a **within-date weight permutation** so market correlation survives; path-shuffle is the wrong null for a long-short ranker. Survivorship is measured. Style regression (market, momentum, low-vol, reversal, liquidity) with White standard errors.
+
+Look-ahead: `position[t] = target[t - lag]` with `lag >= 1`. Turnover is measured against drifted weights, not `|target[t]-target[t-1]|`.
+
+```python
+from algotrader import LabConfig, run_lab
+
+report = run_lab(LabConfig(
+    symbol="SPY",
+    start="2015-01-01",
+    strategy="sma_cross",
+    params={"fast": 20, "slow": 100},
+    source="yahoo",
+    n_permutations=500,
+))
+print(report.verdict["grade"], report.permutation.p_value, report.dsr["dsr"])
+```
+
+```bash
+python -m algotrader.cli strategies
+python -m algotrader.cli lab --symbol SPY --source yahoo
+python -m algotrader.cli portfolio --symbols SPY,QQQ,AAPL,MSFT,NVDA --strategy xs_momentum
+python -m algotrader.cli lab --source synthetic   # offline tests only
+```
 
-`load_data` dispatches on `data_source.type`. Existing `alpaca` / `csv` / `synthetic` branches are unchanged.
+Single-asset zoo: `buy_and_hold`, `sma_cross`, `ema_cross`, `macd_trend`, `rsi_reversion`, `bollinger_reversion`, `donchian_breakout`, `momentum`, `vol_target_momentum`, `channel_trend`, `coin_flip`.
 
-### Strategy and FinRL
+Cross-sectional: `equal_weight`, `xs_momentum`, `xs_reversal`, `low_volatility`, `xs_value_proxy`, `xs_random`.
 
-* `StrategyAgent`: SMA, RSI, Bollinger, MACD on Close; teaching rule, not an alpha claim
-* `FinRLAgent`: PPO / A2C / DDPG / TD3 via Stable-Baselines3; persist under `models/`
-* `ExecutionAgent` / `AlpacaBroker`: paper simulation or Alpaca market/limit orders
+**Data:** `load_ohlcv(..., source="yahoo")` downloads from Yahoo and **raises** if the download is empty. `source="auto"` is the Space fallback (cache, then simulator). `ALGOTRADER_OFFLINE=1` disables the network.
 
-Backtests in this repo are in-sample passes unless you add a purged walk-forward yourself. Leakage is the null hypothesis.
+**HF Space:** `HF_TOKEN=hf_xxx ./scripts/deploy_hf_space.sh <user>/backtest-reality-check`. Card is `SPACE_README.md`. Tests: `python -m pytest tests/test_v2_*.py -q`.
+
+References: Bailey & López de Prado (2014) DSR; Bailey et al. (2016) PBO; Masters (2018) permutation tests for trading systems.
 
 ---
 
-## 3. Stack
+## 4. Concepts and methods (v1 ingest and execution)
+
+| Source | Default? | Failure modes |
+| ------ | -------- | ------------- |
+| **Yahoo** | Yes (`config.yaml`, algotrader CLI, Gradio) | Unofficial API, ~15 min delay, 1m ≈ 7 days, split-adjustment required (`auto_adjust: true`) |
+| **Alpaca** | Optional | Auth, feed, rate limits |
+| **CSV** | Replay | Missing path or OHLCV columns |
+| **Synthetic** | Tests / `--source synthetic` | Not tradable edge |
+
+`agentic_ai_system.data_ingestion.load_data` dispatches on `data_source.type`. Yahoo stream: `yahoo_data_stream.py` (clamped lookback, no incomplete bars by default).
+
+* `StrategyAgent`: SMA, RSI, Bollinger, MACD on Close (teaching rule, not an alpha claim)
+* `FinRLAgent`: PPO / A2C / DDPG / TD3 via Stable-Baselines3
+* `ExecutionAgent` / `AlpacaBroker`: paper simulation or Alpaca orders
+
+v1 `run_backtest` is a single in-sample pass unless you use algotrader walk-forward. Leakage is the null hypothesis.
+
+---
+
+## 5. Stack
 
 | Layer | Tools |
 | ----- | ----- |
-| Language | Python 3.11 (CI); 3.8+ stated for local |
+| Language | Python 3.11 (CI) |
+| Validation | algotrader (permutation, DSR, PBO, walk-forward) |
 | RL | FinRL / Stable-Baselines3, Gym/Gymnasium, PyTorch |
-| Broker | alpaca-py |
-| Market data | Alpaca REST; yfinance ≥ 1.0 (Yahoo) |
+| Market data | yfinance ≥ 1.0 (default); alpaca-py optional |
 | Tabular | pandas, NumPy, scikit-learn |
-| UI | Streamlit, Dash, Jupyter widgets |
-| Deploy | Docker Compose, GitHub Actions |
-| Tests | pytest, pytest-cov |
+| UI | Gradio (`app.py`); Streamlit, Dash, Jupyter (v1) |
+| Deploy | Docker Compose, GitHub Actions, Hugging Face Space |
+| Tests | pytest |
 
 ---
 
-## 4. Structure
+## 6. Structure
 
 ```
 algorithmic_trading/
-├── agentic_ai_system/     # ingest, strategy, FinRL, Alpaca, Yahoo
-├── ui/                    # Streamlit, Dash, Jupyter, WebSocket
+├── algotrader/                 # 2.0 lab, engine, validation, strategies
+├── app.py                      # Gradio Reality Check
+├── agentic_ai_system/          # v1 FinRL, Yahoo/Alpaca ingest, execution
+├── ui/                         # Streamlit, Dash, Jupyter, WebSocket
 ├── tests/
-├── models/                # trained artifacts (gitignored bodies)
-├── data/                  # generated CSV (gitignored)
-├── scripts/               # Docker / deploy helpers
-├─��� .github/workflows/     # CI/CD, release, backtesting
-├── config.yaml
-├── requirements.txt
-├── Dockerfile
-└── docker-compose*.yml
+├── docs/AGENTIC_SYSTEM_V1.md   # v1 notes
+├── config.yaml                 # default data_source.type: yahoo
+├── requirements-space.txt      # Space / algotrader
+├── requirements.txt            # full v1 + CI
+└── scripts/deploy_hf_space.sh
 ```
 
-Branch policy: **`main`** (protected) and **`dev`** only. Do not re-enable Dependabot or the Monday `dependency-updates` workflow; those created extra branches.
-
 ---
 
-## 5. Quick start
+## 7. Configuration
 
-```bash
-git clone https://github.com/ParallelLLC/algorithmic_trading.git
-cd algorithmic_trading
-python -m venv .venv && source .venv/bin/activate
-pip install -r requirements.txt
-cp .env.example .env   # Alpaca keys if using alpaca ingest or orders
-```
+| Key | Meaning |
+| --- | ------- |
+| `data_source.type` | `yahoo` (default) \| `csv` \| `synthetic` \| `alpaca` |
+| `trading.timeframe` | Mapped to Yahoo intervals; use `1d` for multi-year history |
+| `yahoo.auto_adjust` | Split/dividend adjust (keep true) |
+| `yahoo.emit_incomplete_bars` | Default false; forming bars are not closes |
+| `execution.broker_api` | `paper` \| `alpaca_paper` \| `alpaca_live` |
+| `finrl.algorithm` | PPO, A2C, DDPG, TD3 |
+| algotrader `--source` | `yahoo` (default) \| `auto` \| `cache` \| `synthetic` |
 
-Default ingest is CSV. For Yahoo daily bars without a broker:
+---
 
-```yaml
-data_source:
-  type: 'yahoo'
-trading:
-  symbol: 'AAPL'
-  timeframe: '1d'
-```
+## 8. Tests and ops
 
 ```bash
-python demo.py
-python -m agentic_ai_system.main --mode backtest --start-date 2024-01-01 --end-date 2024-12-31
-pytest tests/ -q
+python -m pytest tests/test_v2_*.py -q
+python -m pytest tests/test_yahoo_data_stream.py tests/test_data_ingestion.py -q
 ```
 
-UI launchers and Docker are documented in `UI_SETUP.md` and `DOCKER_HUB_SETUP.md`. Paper-trade before live. Yahoo is not a SIP tape.
-
----
-
-## 6. Configuration (additive Yahoo keys)
-
-| Key | Meaning |
-| --- | ------- |
-| `data_source.type` | `csv` \| `synthetic` \| `alpaca` \| `yahoo` |
-| `yahoo.start_date` / `end_date` | Historical window; clamped per Yahoo interval limits |
-| `yahoo.auto_adjust` | Passed to `yfinance` |
-| `execution.broker_api` | `paper` \| `alpaca_paper` \| `alpaca_live` |
-| `finrl.algorithm` | PPO, A2C, DDPG, TD3 |
+UI launchers and Docker: `UI_SETUP.md`, `DOCKER_HUB_SETUP.md`. Branch policy: `main` and `dev` only. Do not re-enable Dependabot.
 
 ---
 

SPACE_README.md

@@ -0,0 +1,114 @@
+---
+title: Backtest Reality Check
+emoji: 🎲
+colorFrom: blue
+colorTo: gray
+sdk: gradio
+sdk_version: 5.49.1
+app_file: app.py
+pinned: true
+license: apache-2.0
+short_description: Your backtest is probably lying to you. This proves it.
+tags:
+  - finance
+  - quantitative-finance
+  - algorithmic-trading
+  - backtesting
+  - statistics
+  - time-series
+---
+
+# Backtest Reality Check
+
+**Your backtest is probably lying to you.**
+
+Pick a market and a trading rule. This Space runs the backtest — and then spends
+the rest of its effort trying to prove the result was luck.
+
+Most backtesting tools answer *"how much would this have made?"*. That is the easy
+question, and the answer is almost always flattering. This one answers the question
+you need before risking money: **how much of that was luck?**
+
+## The four ways a backtest lies, and the test for each
+
+| The lie | The test |
+|---|---|
+| The market had no structure to find | **Permutation test** — re-run your rule on hundreds of shuffled markets |
+| You tried 200 things and reported the best | **Deflated Sharpe Ratio** — charge for every variant you tried |
+| The parameters were fitted to the past | **PBO + walk-forward** — does the in-sample winner keep winning? |
+| The edge is smaller than the costs | **Cost stress test** — triple the friction and see what survives |
+
+Each contributes to a single **Reality Score** out of 100, with a grade from A to F.
+The scale is deliberately harsh. Most strategies people post online score below 40.
+
+## Two labs
+
+**The Lab** validates a timing rule on one asset. **The Portfolio Lab** validates a
+book that ranks many names — and it gets a harder null: we keep every date's gross
+exposure, net exposure and position count exactly as they were and randomise only
+**which name got which weight**. A book that beats that is picking names. One that
+doesn't was being paid for style exposure you can buy in an ETF, which the factor
+regression measures directly.
+
+It also measures survivorship rather than assuming it away. A universe where every
+name is still trading after ten years was chosen after the fact, and every number
+computed on it is an upper bound.
+
+## Try this first
+
+Run the **Arena** tab on `SPY`. On most markets and most date ranges, plain
+**buy & hold** tops the leaderboard, and the **coin flip** control out-ranks
+several respectable-looking strategies. That is not a bug in the app — it is the
+finding.
+
+## How the permutation test works
+
+We take the real price series and shuffle it. Each bar's gap, high, low, body and
+volume are kept intact, but their **order** is destroyed. The result is a market
+with the same volatility and the same fat tails, and no exploitable structure at
+all. Then we re-run *your exact rule* on hundreds of these shuffled markets.
+
+If your Sharpe ratio sits comfortably inside that cloud, your rule found nothing
+a coin-flip market would not also have handed it.
+
+## No look-ahead, by construction
+
+A strategy emits a target exposure at each bar's close using only data up to that
+bar. The engine holds `position[t] = target[t - lag]` with `lag >= 1`, so a signal
+computed on Tuesday's close cannot earn Tuesday's move. That is the single line
+where look-ahead could enter, and the test suite asserts it directly.
+
+## Use it from Python
+
+```python
+from algotrader import LabConfig, run_lab
+
+report = run_lab(LabConfig(symbol="SPY", strategy="sma_cross"))
+print(report.verdict["grade"], report.verdict["score"])
+print(report.permutation.p_value, report.dsr["dsr"], report.pbo["pbo"])
+```
+
+Or from the command line:
+
+```bash
+python -m algotrader.cli lab --symbol SPY --strategy donchian_breakout --permutations 500
+python -m algotrader.cli arena --symbol BTC-USD
+```
+
+## Data
+
+Live prices come from Yahoo Finance. When the network is unavailable or rate-limited,
+the app falls back to a deterministic market simulator with regime switching, fat
+tails and volatility clustering — and says so, clearly, on every result. The
+statistics remain valid; they are just measured on a simulated market.
+
+## References
+
+- Bailey & López de Prado (2014), *The Deflated Sharpe Ratio*
+- Bailey, Borwein, López de Prado & Zhu (2016), *The Probability of Backtest Overfitting*
+- Masters (2018), *Permutation and Randomization Tests for Trading System Development*
+
+---
+
+Apache-2.0. Research tooling, not investment advice. Nothing here is a
+recommendation to trade.

agentic_ai_system/yahoo_data_stream.py

@@ -1,4 +1,5 @@
 import logging
+import random
 import threading
 import time
 from typing import Any, Callable, Dict, List, Optional
@@ -41,6 +42,26 @@ _MAX_LOOKBACK = {
     '3mo': None,
 }
 
+# How long each bar covers. Used to tell a finished bar from the one still
+# forming right now -- see _drop_incomplete.
+_INTERVAL_DURATION = {
+    '1m': pd.Timedelta(minutes=1),
+    '2m': pd.Timedelta(minutes=2),
+    '5m': pd.Timedelta(minutes=5),
+    '15m': pd.Timedelta(minutes=15),
+    '30m': pd.Timedelta(minutes=30),
+    '60m': pd.Timedelta(hours=1),
+    '90m': pd.Timedelta(minutes=90),
+    '1h': pd.Timedelta(hours=1),
+    '1d': pd.Timedelta(days=1),
+    '5d': pd.Timedelta(days=5),
+    '1wk': pd.Timedelta(weeks=1),
+}
+
+# A daily-or-slower bar that moves more than this is almost always an
+# unadjusted split rather than a real move (NVDA's 2024 10:1 shows up as -90%).
+_SPLIT_SUSPECT_MOVE = 0.35
+
 
 class YahooDataStream:
     """
@@ -62,8 +83,13 @@ class YahooDataStream:
             self.symbols = ['AAPL']
         yahoo_cfg = config.get('yahoo', {})
         self.poll_interval = int(yahoo_cfg.get('poll_interval_seconds', 60))
-        self.auto_adjust = bool(yahoo_cfg.get('auto_adjust', False))
+        # Adjusted by default. With auto_adjust off, Yahoo returns raw Close and
+        # every split reads as a crash: NVDA's June 2024 10:1 becomes a -90% bar.
+        self.auto_adjust = bool(yahoo_cfg.get('auto_adjust', True))
+        self.emit_incomplete_bars = bool(yahoo_cfg.get('emit_incomplete_bars', False))
+        self.max_backoff = int(yahoo_cfg.get('max_backoff_seconds', 900))
         self.interval = self._map_interval(config.get('trading', {}).get('timeframe', '1d'))
+        self._consecutive_failures = 0
         self.data_callbacks: List[Callable] = []
         self.is_connected = False
         self.data_buffer: Dict[str, Dict[str, Any]] = {}
@@ -80,11 +106,21 @@ class YahooDataStream:
                 'latest_bar': None,
             }
 
+        if not self.auto_adjust:
+            logger.warning(
+                "yahoo.auto_adjust is false: prices are NOT split- or dividend-adjusted. "
+                "Every split will appear as a large single-bar loss and any backtest "
+                "spanning one will be wrong."
+            )
+
         logger.info(
-            "Initialized YahooDataStream symbols=%s interval=%s poll_interval=%ss",
+            "Initialized YahooDataStream symbols=%s interval=%s poll_interval=%ss "
+            "auto_adjust=%s emit_incomplete_bars=%s",
             self.symbols,
             self.interval,
             self.poll_interval,
+            self.auto_adjust,
+            self.emit_incomplete_bars,
         )
 
     @staticmethod
@@ -102,7 +138,9 @@ class YahooDataStream:
             return
 
         self._stop_event.clear()
-        self._poll_once()
+        # Seed the backoff from the first attempt: if we are already being
+        # throttled, the loop should start backed off rather than hammering.
+        self._consecutive_failures = 0 if self._poll_once() else 1
         self._poll_thread = threading.Thread(target=self._poll_loop, name='yahoo-poll', daemon=True)
         self._poll_thread.start()
         self.is_connected = True
@@ -137,7 +175,8 @@ class YahooDataStream:
         start, end = self._clamp_window(start_date, end_date, self.interval)
         try:
             raw = self._download(symbol, start=start, end=end, interval=self.interval)
-            df = self._normalize_ohlcv(raw)
+            df = self._drop_incomplete(self._normalize_ohlcv(raw))
+            self._warn_if_unadjusted(symbol, df)
             if df.empty:
                 logger.warning("No Yahoo historical data for %s between %s and %s", symbol, start, end)
             else:
@@ -173,8 +212,6 @@ class YahooDataStream:
         }
 
     def generate_simulated_data(self, symbol: str) -> Dict[str, Any]:
-        import random
-
         latest_data = self.get_latest_data(symbol)
         base_price = 150.0
         if latest_data.get('latest_bar'):
@@ -197,13 +234,40 @@ class YahooDataStream:
         return simulated_bar
 
     def _poll_loop(self) -> None:
-        while not self._stop_event.wait(self.poll_interval):
+        delay = self.poll_interval
+        while not self._stop_event.wait(delay):
             try:
-                self._poll_once()
+                succeeded = self._poll_once()
             except Exception as e:
                 logger.error("Yahoo poll loop error: %s", e, exc_info=True)
-
-    def _poll_once(self) -> None:
+                succeeded = False
+            self._consecutive_failures = 0 if succeeded else self._consecutive_failures + 1
+            delay = self._next_delay()
+
+    def _next_delay(self) -> float:
+        """Poll interval, backed off exponentially while Yahoo is refusing us.
+
+        Yahoo rate-limits aggressively and an unofficial API gives no
+        Retry-After, so a fixed interval just keeps you throttled. Jitter stops
+        several symbols (or several deployments) resynchronising after an outage.
+        """
+        if self._consecutive_failures == 0:
+            base = float(self.poll_interval)
+        else:
+            base = min(
+                self.poll_interval * (2 ** self._consecutive_failures),
+                float(self.max_backoff),
+            )
+            logger.warning(
+                "Yahoo poll failed %s time(s) in a row; next attempt in ~%.0fs",
+                self._consecutive_failures,
+                base,
+            )
+        return max(1.0, base * random.uniform(0.8, 1.2))
+
+    def _poll_once(self) -> bool:
+        """Fetch and ingest one round of bars. Returns True if any symbol succeeded."""
+        any_success = False
         for symbol in self.symbols:
             try:
                 raw = self._download(symbol, period='5d', interval=self.interval)
@@ -212,14 +276,60 @@ class YahooDataStream:
                     logger.warning("Yahoo poll returned no bars for %s", symbol)
                     continue
                 self._ingest_new_bars(symbol, df)
+                any_success = True
             except Exception as e:
                 logger.error("Yahoo poll failed for %s: %s", symbol, e)
+        return any_success
+
+    def _warn_if_unadjusted(self, symbol: str, df: pd.DataFrame) -> int:
+        """Flag single-bar moves that look like unadjusted corporate actions.
+
+        This is a backstop rather than the fix -- the fix is auto_adjust. But a
+        split slipping through silently corrupts every downstream number, so it
+        is worth naming the dates rather than letting a strategy trade them.
+        Returns the number of suspicious bars found.
+        """
+        duration = _INTERVAL_DURATION.get(self.interval)
+        if df.empty or len(df) < 2 or duration is None or duration < pd.Timedelta(days=1):
+            return 0
+        moves = df['close'].pct_change()
+        suspects = df.loc[moves.abs() > _SPLIT_SUSPECT_MOVE, 'timestamp']
+        if len(suspects):
+            dates = ', '.join(str(pd.Timestamp(t).date()) for t in suspects.head(5))
+            logger.warning(
+                "%s has %s bar(s) moving more than %.0f%% (%s). On a liquid name that is "
+                "usually an unadjusted split, not a real move — check yahoo.auto_adjust.",
+                symbol,
+                len(suspects),
+                _SPLIT_SUSPECT_MOVE * 100,
+                dates,
+            )
+        return int(len(suspects))
+
+    def _drop_incomplete(self, df: pd.DataFrame) -> pd.DataFrame:
+        """Remove the bar that is still forming.
+
+        Yahoo returns the in-progress period as an ordinary row. Emitting it
+        would hand the strategy a close that has not happened yet, and because
+        the watermark advances past it, the finished version never arrives.
+        """
+        if self.emit_incomplete_bars or df.empty:
+            return df
+        duration = _INTERVAL_DURATION.get(self.interval)
+        if duration is None:
+            return df
+        now = pd.Timestamp.now(tz='UTC').tz_convert(None)
+        complete = df[df['timestamp'] + duration <= now]
+        dropped = len(df) - len(complete)
+        if dropped:
+            logger.debug("Dropped %s in-progress %s bar(s)", dropped, self.interval)
+        return complete
 
     def _ingest_new_bars(self, symbol: str, df: pd.DataFrame) -> None:
+        rows = self._drop_incomplete(df)
         last_ts = self._last_bar_ts.get(symbol)
-        rows = df
         if last_ts is not None:
-            rows = df[df['timestamp'] > last_ts]
+            rows = rows[rows['timestamp'] > last_ts]
         if rows.empty:
             return
 

algotrader/__init__.py

@@ -0,0 +1,63 @@
+"""algotrader 2.0 — a backtester that tries to prove itself wrong.
+
+Most backtesting libraries answer "how much would this have made?". This one
+answers the question that actually matters before you risk money: "how much of
+that was luck?"
+
+Quick start::
+
+    from algotrader import LabConfig, run_lab
+
+    report = run_lab(LabConfig(symbol="SPY", strategy="sma_cross"))
+    print(report.verdict["verdict"])
+"""
+
+from .attribution import build_style_factors, factor_attribution
+from .cross_sectional import XS_REGISTRY, get_xs_strategy, list_xs_strategies
+from .data import load_ohlcv, simulate_ohlcv
+from .engine import run_backtest
+from .lab import LabConfig, LabReport, run_arena, run_lab
+from .metrics import compute_metrics
+from .panel import Panel, load_panel
+from .portfolio import rebalance_schedule, run_portfolio_backtest
+from .portfolio_lab import PortfolioLabConfig, PortfolioLabReport, run_portfolio_arena, run_portfolio_lab
+from .strategies import REGISTRY, get_strategy, list_strategies
+from .types import BacktestResult, CostModel, MarketData
+from .verdict import reality_score
+
+__version__ = "2.1.0"
+
+__all__ = [
+    "__version__",
+    # single asset
+    "LabConfig",
+    "LabReport",
+    "run_lab",
+    "run_arena",
+    "run_backtest",
+    "get_strategy",
+    "list_strategies",
+    "REGISTRY",
+    # multi asset
+    "Panel",
+    "load_panel",
+    "run_portfolio_backtest",
+    "rebalance_schedule",
+    "PortfolioLabConfig",
+    "PortfolioLabReport",
+    "run_portfolio_lab",
+    "run_portfolio_arena",
+    "get_xs_strategy",
+    "list_xs_strategies",
+    "XS_REGISTRY",
+    "build_style_factors",
+    "factor_attribution",
+    # shared
+    "compute_metrics",
+    "load_ohlcv",
+    "simulate_ohlcv",
+    "BacktestResult",
+    "CostModel",
+    "MarketData",
+    "reality_score",
+]

algotrader/attribution.py

@@ -0,0 +1,148 @@
+"""Style attribution: is this alpha, or is it beta you could have bought cheaply?
+
+The most common way a strategy is oversold is not fraud or overfitting — it is
+that the "edge" is a well-known risk premium wearing a new name. A book that
+loads on market beta, or on momentum, or on low-volatility, will produce a
+respectable Sharpe and an exciting story, and you can buy the same exposure in
+an ETF for a few basis points.
+
+So we regress the strategy's returns on style factors built from the panel
+itself and ask what is left over. If the intercept is not distinguishable from
+zero, the strategy has no alpha — however good its Sharpe looked.
+
+Factors are constructed from the universe under test rather than downloaded,
+which keeps this offline and self-consistent. The tradeoff is that they are
+proxies: without fundamentals there is no true value or size factor, so those
+are labelled honestly as what they are.
+"""
+
+from __future__ import annotations
+
+from typing import Dict, Optional
+
+import numpy as np
+import pandas as pd
+
+from .panel import Panel
+from .portfolio import run_portfolio_backtest
+from .types import CostModel
+
+__all__ = ["build_style_factors", "factor_attribution"]
+
+_FREE = CostModel(0.0, 0.0, 0.0)  # factors are theoretical portfolios
+
+
+def _long_short(panel: Panel, scores: pd.DataFrame, rebalance: int = 21) -> pd.Series:
+    from .cross_sectional import _rebalance_hold, scores_to_weights
+
+    weights = _rebalance_hold(scores_to_weights(scores), rebalance)
+    return run_portfolio_backtest(panel, weights, costs=_FREE).returns
+
+
+def build_style_factors(panel: Panel, rebalance: int = 21) -> pd.DataFrame:
+    """Construct market and style factor returns from the panel itself."""
+    close = panel.close
+    listed = close.notna()
+    counts = listed.sum(axis=1).replace(0, np.nan)
+
+    equal = listed.astype(float).div(counts, axis=0).fillna(0.0)
+    market = run_portfolio_backtest(panel, equal, costs=_FREE).returns
+
+    factors = {
+        "market": market,
+        "momentum": _long_short(panel, close.shift(20) / close.shift(250) - 1.0, rebalance),
+        "low_vol": _long_short(panel, -close.pct_change().rolling(60, min_periods=60).std(), rebalance),
+        "reversal": _long_short(panel, -close.pct_change(5), rebalance),
+        # Dollar volume is a liquidity proxy, not market cap. Named accordingly.
+        "liquidity": _long_short(panel, -np.log(panel.dollar_volume().replace(0, np.nan)), rebalance),
+    }
+    return pd.DataFrame(factors).reindex(panel.index).fillna(0.0)
+
+
+def _white_standard_errors(x: np.ndarray, residuals: np.ndarray, xtx_inv: np.ndarray) -> np.ndarray:
+    """Heteroskedasticity-robust (White) standard errors.
+
+    Return series are famously heteroskedastic — volatility clusters — and
+    classical standard errors would overstate the significance of alpha.
+    """
+    meat = x.T @ (x * (residuals**2)[:, None])
+    covariance = xtx_inv @ meat @ xtx_inv
+    return np.sqrt(np.maximum(np.diag(covariance), 0.0))
+
+
+def factor_attribution(
+    returns: pd.Series,
+    factors: pd.DataFrame,
+    periods_per_year: int = 252,
+    alpha_t_threshold: float = 2.0,
+) -> Dict[str, object]:
+    """Regress strategy returns on style factors; report annualised alpha and betas."""
+    aligned = pd.concat([returns.rename("strategy"), factors], axis=1).dropna()
+    if len(aligned) < 60 or factors.shape[1] == 0:
+        return {"available": False, "note": "Not enough overlapping observations for attribution."}
+
+    y = aligned["strategy"].to_numpy(dtype=float)
+    names = list(factors.columns)
+    x = np.column_stack([np.ones(len(aligned))] + [aligned[c].to_numpy(dtype=float) for c in names])
+
+    # Drop factors that are constant or collinear; a singular fit is worse than
+    # a smaller one.
+    keep = [0] + [i + 1 for i, c in enumerate(names) if aligned[c].std() > 1e-12]
+    x = x[:, keep]
+    names = [names[i - 1] for i in keep[1:]]
+
+    try:
+        xtx_inv = np.linalg.pinv(x.T @ x)
+    except np.linalg.LinAlgError:  # pragma: no cover - pinv rarely fails
+        return {"available": False, "note": "Factor matrix is singular."}
+
+    beta = xtx_inv @ x.T @ y
+    fitted = x @ beta
+    residuals = y - fitted
+    errors = _white_standard_errors(x, residuals, xtx_inv)
+    t_stats = np.divide(beta, errors, out=np.zeros_like(beta), where=errors > 0)
+
+    ss_res = float(residuals @ residuals)
+    ss_tot = float(((y - y.mean()) ** 2).sum())
+    r_squared = 1.0 - ss_res / ss_tot if ss_tot > 0 else 0.0
+
+    alpha_period = float(beta[0])
+    alpha_annual = alpha_period * periods_per_year
+    alpha_t = float(t_stats[0])
+    significant = bool(abs(alpha_t) >= alpha_t_threshold and alpha_annual > 0)
+
+    betas = {name: float(b) for name, b in zip(names, beta[1:])}
+    beta_ts = {name: float(t) for name, t in zip(names, t_stats[1:])}
+    dominant = max(betas, key=lambda k: abs(betas[k])) if betas else None
+
+    if significant:
+        note = (
+            f"Alpha of {alpha_annual:.1%} a year survives the style regression "
+            f"(t = {alpha_t:.1f}). Something here is not explained by market, momentum, "
+            "low-volatility, reversal or liquidity exposure."
+        )
+    else:
+        explained = (
+            f" Most of the variation is {dominant} exposure (beta {betas[dominant]:.2f})."
+            if dominant
+            else ""
+        )
+        note = (
+            f"Alpha is {alpha_annual:.1%} a year with t = {alpha_t:.1f}, which is not "
+            f"distinguishable from zero. The style factors explain {r_squared:.0%} of the "
+            f"returns.{explained} You can buy that exposure far more cheaply than by "
+            "running this strategy."
+        )
+
+    return {
+        "available": True,
+        "alpha_annual": alpha_annual,
+        "alpha_t_stat": alpha_t,
+        "alpha_significant": significant,
+        "betas": betas,
+        "beta_t_stats": beta_ts,
+        "r_squared": float(r_squared),
+        "dominant_factor": dominant,
+        "n_obs": int(len(aligned)),
+        "note": note,
+    }

algotrader/charts.py

@@ -0,0 +1,393 @@
+"""Plotly figures for the Lab.
+
+Colour system (dark surface, validated for CVD separation):
+
+* **blue** is always *your strategy's honest result* — the realised equity
+  curve, the out-of-sample fold, the observed Sharpe.
+* **orange** is always *the thing it is measured against* — buy & hold, the
+  in-sample fold, the null distribution.
+
+Holding that mapping across every figure means a reader learns it once.
+"""
+
+from __future__ import annotations
+
+from typing import Dict, Optional
+
+import numpy as np
+import pandas as pd
+import plotly.graph_objects as go
+
+SURFACE = "#1a1a19"
+PAGE = "#0d0d0d"
+INK = "#ffffff"
+INK_SECONDARY = "#c3c2b7"
+INK_MUTED = "#898781"
+GRID = "#2c2c2a"
+AXIS = "#383835"
+
+SUBJECT = "#3987e5"  # categorical slot 1
+REFERENCE = "#d95926"  # categorical slot 2
+NEGATIVE = "#e66767"  # negative arm of the diverging pair (drawdowns)
+
+FONT = 'system-ui, -apple-system, "Segoe UI", sans-serif'
+
+_EMPTY_NOTE = "Run an analysis to populate this chart."
+
+
+def _base_layout(title: str, height: int = 340, **kwargs) -> dict:
+    return dict(
+        title=dict(text=title, font=dict(size=15, color=INK), x=0, xanchor="left", pad=dict(b=8)),
+        paper_bgcolor=PAGE,
+        plot_bgcolor=SURFACE,
+        font=dict(family=FONT, size=12, color=INK_SECONDARY),
+        height=height,
+        margin=dict(l=56, r=24, t=48, b=40),
+        hovermode="x unified",
+        hoverlabel=dict(bgcolor=SURFACE, bordercolor=AXIS, font=dict(color=INK, family=FONT)),
+        xaxis=dict(gridcolor=GRID, linecolor=AXIS, zeroline=False, tickfont=dict(color=INK_MUTED)),
+        yaxis=dict(gridcolor=GRID, linecolor=AXIS, zeroline=False, tickfont=dict(color=INK_MUTED)),
+        legend=dict(
+            orientation="h", yanchor="bottom", y=1.02, xanchor="left", x=0,
+            font=dict(color=INK_SECONDARY, size=11), bgcolor="rgba(0,0,0,0)",
+        ),
+        **kwargs,
+    )
+
+
+def empty_figure(message: str = _EMPTY_NOTE, height: int = 340) -> go.Figure:
+    fig = go.Figure()
+    fig.update_layout(**_base_layout("", height=height))
+    fig.update_xaxes(visible=False)
+    fig.update_yaxes(visible=False)
+    fig.add_annotation(
+        text=message, showarrow=False, xref="paper", yref="paper", x=0.5, y=0.5,
+        font=dict(color=INK_MUTED, size=13),
+    )
+    return fig
+
+
+def equity_chart(report, benchmark_label: str = "Buy & hold") -> go.Figure:
+    """Strategy equity against its benchmark, both indexed to the same start."""
+    bt = report.backtest
+    strat = bt.equity / bt.equity.iloc[0] * 100.0
+    bench = bt.benchmark_equity / bt.benchmark_equity.iloc[0] * 100.0
+
+    fig = go.Figure()
+    fig.add_trace(
+        go.Scatter(
+            x=bench.index, y=bench.to_numpy(), name=benchmark_label, mode="lines",
+            line=dict(color=REFERENCE, width=2, dash="dash"),
+            hovertemplate=benchmark_label + "  %{y:.1f}<extra></extra>",
+        )
+    )
+    fig.add_trace(
+        go.Scatter(
+            x=strat.index, y=strat.to_numpy(), name=report.strategy.name, mode="lines",
+            line=dict(color=SUBJECT, width=2),
+            hovertemplate=report.strategy.name + "  %{y:.1f}<extra></extra>",
+        )
+    )
+
+    # Direct-label the two endpoints; the axis and tooltip carry everything else.
+    for series, color, label in ((strat, SUBJECT, report.strategy.name), (bench, REFERENCE, benchmark_label)):
+        fig.add_annotation(
+            x=series.index[-1], y=float(series.iloc[-1]),
+            text=f"  {label}: {series.iloc[-1]:.0f}", showarrow=False,
+            xanchor="left", font=dict(color=color, size=11),
+        )
+
+    fig.update_layout(**_base_layout("Growth of 100 (net of costs)", height=360))
+    fig.update_layout(margin=dict(l=56, r=140, t=48, b=40))
+    return fig
+
+
+def drawdown_chart(report) -> go.Figure:
+    """Underwater plot — how deep, and for how long."""
+    from .metrics import drawdown_series
+
+    dd = drawdown_series(report.backtest.equity) * 100.0
+    fig = go.Figure(
+        go.Scatter(
+            x=dd.index, y=dd.to_numpy(), mode="lines", name="Drawdown",
+            line=dict(color=NEGATIVE, width=2), fill="tozeroy",
+            fillcolor="rgba(230,103,103,0.18)",
+            hovertemplate="Drawdown %{y:.1f}%<extra></extra>",
+        )
+    )
+    trough = float(dd.min())
+    fig.add_annotation(
+        x=dd.idxmin(), y=trough, text=f"worst {trough:.1f}%", showarrow=True,
+        arrowhead=0, arrowcolor=AXIS, ay=24, font=dict(color=INK_SECONDARY, size=11),
+    )
+    fig.update_layout(**_base_layout("Drawdown", height=240, showlegend=False))
+    fig.update_yaxes(ticksuffix="%")
+    return fig
+
+
+def permutation_chart(report) -> go.Figure:
+    """The headline chart: your Sharpe against Sharpes from shuffled markets."""
+    perm = report.permutation
+    if perm is None or perm.null.size == 0:
+        return empty_figure("Permutation test was skipped.", height=320)
+
+    null = perm.null
+    fig = go.Figure()
+    fig.add_trace(
+        go.Histogram(
+            x=null, name="Shuffled markets (no real edge)", nbinsx=44,
+            marker=dict(color="rgba(217,89,38,0.55)", line=dict(color=REFERENCE, width=1)),
+            hovertemplate="Sharpe %{x:.2f}<br>%{y} shuffles<extra></extra>",
+        )
+    )
+
+    top = np.histogram(null, bins=44)[0].max() if null.size else 1
+    fig.add_trace(
+        go.Scatter(
+            x=[perm.observed, perm.observed], y=[0, top * 1.08], mode="lines",
+            name="Your strategy", line=dict(color=SUBJECT, width=2),
+            hovertemplate="Your Sharpe %{x:.2f}<extra></extra>",
+        )
+    )
+    fig.add_annotation(
+        x=perm.observed, y=top * 1.08, text=f"  your Sharpe {perm.observed:.2f}",
+        showarrow=False, xanchor="left", font=dict(color=SUBJECT, size=11),
+    )
+
+    beats = (null >= perm.observed).mean() * 100.0
+    fig.update_layout(
+        **_base_layout(
+            f"Permutation test — {beats:.0f}% of structure-free markets did this well or better "
+            f"(p = {perm.p_value:.3f})",
+            height=320,
+        )
+    )
+    fig.update_layout(hovermode="closest", bargap=0.02)
+    fig.update_xaxes(title=dict(text="Annualised Sharpe ratio", font=dict(color=INK_MUTED, size=11)))
+    # Headroom so the "your Sharpe" label never collides with the plot edge.
+    fig.update_yaxes(
+        title=dict(text="Shuffled markets", font=dict(color=INK_MUTED, size=11)),
+        range=[0, top * 1.28],
+    )
+    return fig
+
+
+def walkforward_chart(report) -> go.Figure:
+    """In-sample vs out-of-sample Sharpe, fold by fold."""
+    folds = report.walkforward.get("folds") or []
+    if not folds:
+        return empty_figure(report.walkforward.get("note") or _EMPTY_NOTE, height=300)
+
+    labels = [f"Fold {f['fold']}<br><span style='font-size:10px'>{f['test_start'][:7]}</span>" for f in folds]
+    fig = go.Figure()
+    fig.add_trace(
+        go.Bar(
+            x=labels, y=[f["is_sharpe"] for f in folds], name="In-sample (tuned)",
+            marker=dict(color=REFERENCE, line=dict(color=SURFACE, width=2)),
+            hovertemplate="In-sample Sharpe %{y:.2f}<extra></extra>",
+        )
+    )
+    fig.add_trace(
+        go.Bar(
+            x=labels, y=[f["oos_sharpe"] for f in folds], name="Out-of-sample (blind)",
+            marker=dict(color=SUBJECT, line=dict(color=SURFACE, width=2)),
+            hovertemplate="Out-of-sample Sharpe %{y:.2f}<extra></extra>",
+        )
+    )
+    eff = report.walkforward.get("efficiency", 0.0)
+    fig.update_layout(
+        **_base_layout(f"Walk-forward — {eff:.0%} of the tuned Sharpe survived out of sample", height=300)
+    )
+    fig.update_layout(barmode="group", bargap=0.35, bargroupgap=0.08, hovermode="x unified")
+    fig.add_hline(y=0, line=dict(color=AXIS, width=1))
+    return fig
+
+
+def score_chart(verdict: Dict[str, object], significance_label: str = "Beats shuffled markets") -> go.Figure:
+    """The five components behind the Reality Score."""
+    components = verdict.get("components") or {}
+    if not components:
+        return empty_figure(height=260)
+
+    pretty = {
+        "significance": significance_label,
+        "selection": "Survives selection bias",
+        "walk_forward": "Holds up walking forward",
+        "overfitting": "Not overfit (PBO)",
+        "robustness": "Survives 3x costs",
+    }
+    keys = list(pretty)
+    values = [float(components.get(k, 0.0)) for k in keys]
+
+    fig = go.Figure(
+        go.Bar(
+            x=values, y=[pretty[k] for k in keys], orientation="h",
+            marker=dict(color=SUBJECT, line=dict(color=SURFACE, width=2)),
+            text=[f"{v:.0f}" for v in values], textposition="outside",
+            textfont=dict(color=INK_SECONDARY, size=11),
+            hovertemplate="%{y}: %{x:.0f}/100<extra></extra>",
+        )
+    )
+    fig.update_layout(**_base_layout("Where the score comes from", height=260, showlegend=False))
+    fig.update_layout(margin=dict(l=190, r=48, t=48, b=32), hovermode="closest")
+    fig.update_xaxes(range=[0, 108], tickvals=[0, 25, 50, 75, 100])
+    fig.update_yaxes(autorange="reversed")
+    return fig
+
+
+def arena_chart(table: pd.DataFrame) -> go.Figure:
+    """Leaderboard bars. One measure, one colour — the table carries the rest."""
+    if table is None or table.empty:
+        return empty_figure(height=380)
+
+    ordered = table.iloc[::-1]
+    fig = go.Figure(
+        go.Bar(
+            x=ordered["Sharpe"].to_numpy(), y=ordered["Strategy"].tolist(), orientation="h",
+            marker=dict(color=SUBJECT, line=dict(color=SURFACE, width=2)),
+            customdata=np.column_stack([ordered["p-value"].to_numpy(), ordered["DSR"].to_numpy()]),
+            hovertemplate="%{y}<br>Sharpe %{x:.2f}<br>p = %{customdata[0]:.3f}"
+                          "<br>Deflated Sharpe %{customdata[1]:.2f}<extra></extra>",
+        )
+    )
+    # Direct-label only what matters: the ones that actually cleared significance.
+    for _, row in ordered.iterrows():
+        if np.isfinite(row["p-value"]) and row["p-value"] < 0.05:
+            fig.add_annotation(
+                x=row["Sharpe"], y=row["Strategy"], text="  p &lt; 0.05", showarrow=False,
+                xanchor="left" if row["Sharpe"] >= 0 else "right",
+                font=dict(color=INK_SECONDARY, size=10),
+            )
+    fig.update_layout(
+        **_base_layout(
+            "Strategy arena — Sharpe ratio, ordered by strength of evidence",
+            height=max(300, 42 * len(table)),
+            showlegend=False,
+        )
+    )
+    fig.update_layout(margin=dict(l=180, r=96, t=48, b=32), hovermode="closest")
+    fig.add_vline(x=0, line=dict(color=AXIS, width=1))
+    return fig
+
+
+def cross_permutation_chart(report) -> go.Figure:
+    """Sharpe against books of identical shape holding randomly chosen names."""
+    perm = report.permutation
+    if perm is None or perm.null.size == 0:
+        return empty_figure("Name-shuffle test was skipped.", height=320)
+
+    fig = go.Figure()
+    fig.add_trace(
+        go.Histogram(
+            x=perm.null, name="Same book, random names", nbinsx=40,
+            marker=dict(color="rgba(217,89,38,0.55)", line=dict(color=REFERENCE, width=1)),
+            hovertemplate="Sharpe %{x:.2f}<br>%{y} shuffles<extra></extra>",
+        )
+    )
+    top = np.histogram(perm.null, bins=40)[0].max() if perm.null.size else 1
+    fig.add_trace(
+        go.Scatter(
+            x=[perm.observed, perm.observed], y=[0, top * 1.08], mode="lines",
+            name="Your book", line=dict(color=SUBJECT, width=2),
+            hovertemplate="Your Sharpe %{x:.2f}<extra></extra>",
+        )
+    )
+    fig.add_annotation(
+        x=perm.observed, y=top * 1.08, text=f"  your Sharpe {perm.observed:.2f}",
+        showarrow=False, xanchor="left", font=dict(color=SUBJECT, size=11),
+    )
+    beats = (perm.null >= perm.observed).mean() * 100.0
+    fig.update_layout(
+        **_base_layout(
+            f"Name-shuffle test — {beats:.0f}% of books with the same shape but random names "
+            f"did this well or better (p = {perm.p_value:.3f})",
+            height=320,
+        )
+    )
+    fig.update_layout(hovermode="closest", bargap=0.02)
+    fig.update_xaxes(title=dict(text="Annualised Sharpe ratio", font=dict(color=INK_MUTED, size=11)))
+    fig.update_yaxes(
+        title=dict(text="Shuffled books", font=dict(color=INK_MUTED, size=11)),
+        range=[0, top * 1.28],
+    )
+    return fig
+
+
+def attribution_chart(attribution: Dict[str, object]) -> go.Figure:
+    """Factor betas. One measure across categories, so one colour."""
+    if not attribution or not attribution.get("available"):
+        return empty_figure((attribution or {}).get("note", _EMPTY_NOTE), height=280)
+
+    betas = attribution.get("betas") or {}
+    if not betas:
+        return empty_figure("No factor exposures to show.", height=280)
+
+    names = list(betas)
+    values = [betas[n] for n in names]
+    fig = go.Figure(
+        go.Bar(
+            x=values, y=[n.replace("_", " ") for n in names], orientation="h",
+            marker=dict(color=SUBJECT, line=dict(color=SURFACE, width=2)),
+            text=[f"{v:+.2f}" for v in values], textposition="outside",
+            textfont=dict(color=INK_SECONDARY, size=11),
+            hovertemplate="%{y} beta %{x:.2f}<extra></extra>",
+        )
+    )
+    alpha = attribution.get("alpha_annual", 0.0)
+    t_stat = attribution.get("alpha_t_stat", 0.0)
+    fig.update_layout(
+        **_base_layout(
+            f"Style exposure — alpha {alpha:+.1%}/yr (t = {t_stat:.1f}), "
+            f"R² {attribution.get('r_squared', 0):.0%}",
+            height=280,
+            showlegend=False,
+        )
+    )
+    fig.update_layout(margin=dict(l=120, r=88, t=48, b=32), hovermode="closest")
+    # Outside labels need room or the widest beta reads as "+0".
+    span = max(abs(min(values)), abs(max(values)), 0.1)
+    fig.update_xaxes(range=[min(0, min(values)) - 0.25 * span, max(0, max(values)) + 0.35 * span])
+    fig.add_vline(x=0, line=dict(color=AXIS, width=1))
+    return fig
+
+
+def weights_chart(report) -> go.Figure:
+    """Gross and net exposure over time — is the book actually neutral?"""
+    held = report.backtest.held
+    gross = held.abs().sum(axis=1)
+    net = held.sum(axis=1)
+
+    fig = go.Figure()
+    fig.add_trace(
+        go.Scatter(
+            x=gross.index, y=gross.to_numpy(), name="Gross", mode="lines",
+            line=dict(color=REFERENCE, width=2, dash="dash"),
+            hovertemplate="Gross %{y:.2f}x<extra></extra>",
+        )
+    )
+    fig.add_trace(
+        go.Scatter(
+            x=net.index, y=net.to_numpy(), name="Net", mode="lines",
+            line=dict(color=SUBJECT, width=2),
+            hovertemplate="Net %{y:.2f}x<extra></extra>",
+        )
+    )
+    fig.update_layout(**_base_layout("Book exposure", height=240))
+    fig.add_hline(y=0, line=dict(color=AXIS, width=1))
+    return fig
+
+
+def exposure_chart(report) -> go.Figure:
+    """What the strategy was actually holding, over time."""
+    pos = report.backtest.position
+    fig = go.Figure(
+        go.Scatter(
+            x=pos.index, y=pos.to_numpy(), mode="lines", name="Exposure",
+            line=dict(color=SUBJECT, width=2, shape="hv"), fill="tozeroy",
+            fillcolor="rgba(57,135,229,0.16)",
+            hovertemplate="Exposure %{y:.2f}x<extra></extra>",
+        )
+    )
+    fig.update_layout(**_base_layout("Position held", height=200, showlegend=False))
+    fig.add_hline(y=0, line=dict(color=AXIS, width=1))
+    return fig

algotrader/cli.py

@@ -0,0 +1,290 @@
+"""Command-line interface.
+
+    python -m algotrader.cli lab --symbol SPY --strategy sma_cross
+    python -m algotrader.cli arena --symbol BTC-USD --start 2018-01-01
+    python -m algotrader.cli strategies
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from typing import Dict, List
+
+from . import __version__
+from .lab import LabConfig, run_arena, run_lab
+from .strategies import REGISTRY, get_strategy
+
+
+def _parse_params(pairs: List[str] | None) -> Dict[str, float]:
+    params: Dict[str, float] = {}
+    for pair in pairs or []:
+        if "=" not in pair:
+            raise SystemExit(f"--param expects name=value, got '{pair}'")
+        name, _, value = pair.partition("=")
+        params[name.strip()] = float(value)
+    return params
+
+
+def _add_common(parser: argparse.ArgumentParser) -> None:
+    parser.add_argument("--symbol", default="SPY")
+    parser.add_argument("--start", default="2015-01-01")
+    parser.add_argument("--end", default=None)
+    parser.add_argument("--interval", default="1d")
+    parser.add_argument(
+        "--source", default="yahoo", choices=["yahoo", "live", "auto", "cache", "synthetic"],
+        help="'yahoo' requires a Yahoo download. 'synthetic' is offline tests only. 'auto' falls back to the simulator.",
+    )
+    parser.add_argument("--commission-bps", type=float, default=1.0)
+    parser.add_argument("--slippage-bps", type=float, default=2.0)
+    parser.add_argument("--no-short", action="store_true", help="Long/flat only.")
+
+
+def _config_from(args: argparse.Namespace, **overrides) -> LabConfig:
+    return LabConfig(
+        symbol=args.symbol,
+        start=args.start,
+        end=args.end,
+        interval=args.interval,
+        source=args.source,
+        commission_bps=args.commission_bps,
+        slippage_bps=args.slippage_bps,
+        allow_short=not args.no_short,
+        **overrides,
+    )
+
+
+def _cmd_lab(args: argparse.Namespace) -> int:
+    cfg = _config_from(
+        args,
+        strategy=args.strategy,
+        params=_parse_params(args.param),
+        n_permutations=args.permutations,
+        permutation_method=args.null,
+        wf_folds=args.folds,
+    )
+    progress = None if args.quiet else (lambda f, m: print(f"  [{f:5.0%}] {m}", file=sys.stderr))
+    report = run_lab(cfg, progress=progress)
+
+    if args.json:
+        payload = {
+            "symbol": report.market.symbol,
+            "source": report.market.source,
+            "strategy": report.strategy.key,
+            "params": report.params,
+            "metrics": report.backtest.metrics,
+            "benchmark_metrics": report.backtest.benchmark_metrics,
+            "p_value": report.permutation.p_value if report.permutation else None,
+            "deflated_sharpe": report.dsr.get("dsr"),
+            "pbo": report.pbo.get("pbo"),
+            "walkforward_efficiency": report.walkforward.get("efficiency"),
+            "cost_stress": report.cost_stress,
+            "verdict": {k: v for k, v in report.verdict.items()},
+        }
+        print(json.dumps(payload, indent=2, default=str))
+        return 0
+
+    v, m, b = report.verdict, report.backtest.metrics, report.backtest.benchmark_metrics
+    bar = "=" * 66
+    print(f"\n{bar}")
+    print(f"  {report.strategy.name} on {report.market.symbol}   [{report.market.source} data]")
+    print(f"  {report.market.start.date()} to {report.market.end.date()}  ·  {len(report.market.df):,} bars")
+    print(bar)
+    print(f"  REALITY SCORE   {v['score']:.1f} / 100      GRADE  {v['grade']}")
+    print(f"  {v['headline']}")
+    print(bar)
+    print(f"  Total return      {m['total_return']:>9.1%}    buy & hold {b['total_return']:>8.1%}")
+    print(f"  CAGR              {m['cagr']:>9.1%}    buy & hold {b['cagr']:>8.1%}")
+    print(f"  Sharpe            {m['sharpe']:>9.2f}    buy & hold {b['sharpe']:>8.2f}")
+    print(f"  Max drawdown      {m['max_drawdown']:>9.1%}")
+    print(f"  Trades            {int(m.get('n_trades', 0)):>9,}")
+    print(bar)
+    if report.permutation:
+        print(f"  Permutation p     {report.permutation.p_value:>9.3f}    ({report.permutation.n_permutations} shuffled markets)")
+    print(f"  Deflated Sharpe   {report.dsr.get('dsr', 0):>9.2f}    (after {report.trials.get('n', 1)} variants)")
+    pbo = report.pbo.get("pbo")
+    print(f"  Overfit prob.     {pbo:>9.2f}" if pbo == pbo else "  Overfit prob.           n/a")
+    print(f"  Walk-forward eff. {report.walkforward.get('efficiency', 0):>9.2f}")
+    print(f"  Sharpe at 3x cost {report.cost_stress.get('sharpe_3x', 0):>9.2f}")
+    print(bar)
+    for flag in v["flags"]:
+        print(f"  ! {flag}")
+    if v["flags"]:
+        print(bar)
+    print(f"  {v['verdict']}\n")
+    return 0
+
+
+def _cmd_arena(args: argparse.Namespace) -> int:
+    cfg = _config_from(args)
+    progress = None if args.quiet else (lambda f, m: print(f"  [{f:5.0%}] {m}", file=sys.stderr))
+    table, market, _ = run_arena(cfg, n_permutations=args.permutations, progress=progress)
+
+    if args.json:
+        print(table.to_json(orient="records", indent=2))
+        return 0
+
+    print(f"\n  {market.symbol}  [{market.source} data]  "
+          f"{market.start.date()} to {market.end.date()}\n")
+    display = table.drop(columns=["key"]).copy()
+    for col in ("Return", "CAGR", "MaxDD"):
+        display[col] = display[col].map("{:.1%}".format)
+    for col in ("Sharpe", "DSR", "Evidence"):
+        display[col] = display[col].map("{:.2f}".format)
+    display["p-value"] = display["p-value"].map(lambda v: "—" if v != v else f"{v:.3f}")
+    print(display.to_string(index=False))
+    print("\n  Ranked by evidence = (1 - p) x deflated Sharpe, not by return.\n")
+    return 0
+
+
+def _cmd_portfolio(args: argparse.Namespace) -> int:
+    from .portfolio_lab import DEFAULT_UNIVERSE, PortfolioLabConfig, run_portfolio_lab
+
+    symbols = [s.strip() for s in args.symbols.split(",") if s.strip()] if args.symbols else DEFAULT_UNIVERSE
+    cfg = PortfolioLabConfig(
+        symbols=symbols,
+        start=args.start,
+        end=args.end,
+        interval=args.interval,
+        source=args.source,
+        strategy=args.strategy,
+        params=_parse_params(args.param),
+        commission_bps=args.commission_bps,
+        slippage_bps=args.slippage_bps,
+        allow_short=not args.no_short,
+        rebalance=args.rebalance,
+        n_permutations=args.permutations,
+        wf_folds=args.folds,
+    )
+    progress = None if args.quiet else (lambda f, m: print(f"  [{f:5.0%}] {m}", file=sys.stderr))
+    report = run_portfolio_lab(cfg, progress=progress)
+
+    if args.json:
+        print(json.dumps({
+            "symbols": report.panel.symbols,
+            "strategy": report.strategy.key,
+            "params": report.params,
+            "metrics": report.backtest.metrics,
+            "p_value": report.permutation.p_value if report.permutation else None,
+            "deflated_sharpe": report.dsr.get("dsr"),
+            "pbo": report.pbo.get("pbo"),
+            "walkforward_efficiency": report.walkforward.get("efficiency"),
+            "attribution": report.attribution,
+            "survivorship": report.survivorship.__dict__,
+            "verdict": dict(report.verdict),
+        }, indent=2, default=str))
+        return 0
+
+    v, m, b = report.verdict, report.backtest.metrics, report.backtest.benchmark_metrics
+    bar = "=" * 72
+    print(f"\n{bar}")
+    print(f"  {report.strategy.name} on {len(report.panel.symbols)} symbols   [{report.panel.interval}]")
+    print(f"  {report.panel.index[0].date()} to {report.panel.index[-1].date()}  ·  "
+          f"{len(report.panel):,} bars  ·  rebalance {report.config.rebalance}")
+    print(bar)
+    print(f"  REALITY SCORE   {v['score']:.1f} / 100      GRADE  {v['grade']}")
+    print(f"  {v['headline']}")
+    print(bar)
+    print(f"  Total return      {m['total_return']:>9.1%}    equal weight {b['total_return']:>8.1%}")
+    print(f"  CAGR              {m['cagr']:>9.1%}    equal weight {b['cagr']:>8.1%}")
+    print(f"  Sharpe            {m['sharpe']:>9.2f}    equal weight {b['sharpe']:>8.2f}")
+    print(f"  Max drawdown      {m['max_drawdown']:>9.1%}")
+    print(f"  Gross / net exp.  {m.get('gross_exposure', 0):>9.2f} / {m.get('net_exposure', 0):.2f}")
+    print(f"  Avg positions     {m.get('avg_positions', 0):>9.1f}    turnover {m.get('turnover_ann', 0):.1f}x/yr")
+    print(bar)
+    if report.permutation:
+        print(f"  Name-shuffle p    {report.permutation.p_value:>9.3f}    "
+              f"({report.permutation.n_permutations} shuffles of which names got which weights)")
+    print(f"  Deflated Sharpe   {report.dsr.get('dsr', 0):>9.2f}    (after {report.trials.get('n', 1)} variants)")
+    pbo = report.pbo.get("pbo")
+    print(f"  Overfit prob.     {pbo:>9.2f}" if pbo == pbo else "  Overfit prob.           n/a")
+    print(f"  Walk-forward eff. {report.walkforward.get('efficiency', 0):>9.2f}")
+    if report.attribution.get("available"):
+        print(f"  Style alpha       {report.attribution['alpha_annual']:>9.1%}    "
+              f"t = {report.attribution['alpha_t_stat']:.2f}, R² = {report.attribution['r_squared']:.2f}")
+    print(f"  Survivorship      {report.survivorship.survival_rate:>9.0%}    "
+          f"({report.survivorship.n_delisted} of {report.survivorship.n_symbols} stopped trading)")
+    print(bar)
+    for flag in v["flags"]:
+        print(f"  ! {flag}")
+    if v["flags"]:
+        print(bar)
+    print(f"  {v['verdict']}\n")
+    return 0
+
+
+def _cmd_strategies(args: argparse.Namespace) -> int:
+    from .cross_sectional import XS_REGISTRY
+
+    for title, registry in (("Single asset", REGISTRY), ("Cross-sectional", XS_REGISTRY)):
+        print(f"\n  {title}\n  {'-' * len(title)}")
+        for key, strategy in registry.items():
+            params = ", ".join(f"{p.name}={p.default:g}" for p in strategy.params) or "no parameters"
+            print(f"  {key:<22} {strategy.name:<28} [{strategy.family}]")
+            print(f"  {'':<22} {strategy.description}")
+            print(f"  {'':<22} defaults: {params}\n")
+    return 0
+
+
+def main(argv: List[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(
+        prog="algotrader",
+        description="Backtest a trading rule, then try to prove the result was luck.",
+    )
+    parser.add_argument("--version", action="version", version=f"algotrader {__version__}")
+    sub = parser.add_subparsers(dest="command", required=True)
+
+    lab = sub.add_parser("lab", help="Full reality check for one strategy.")
+    _add_common(lab)
+    lab.add_argument("--strategy", default="sma_cross", choices=sorted(REGISTRY))
+    lab.add_argument("--param", action="append", metavar="NAME=VALUE",
+                     help="Override a strategy parameter. Repeatable.")
+    lab.add_argument("--permutations", type=int, default=250)
+    lab.add_argument("--null", default="permute", choices=["permute", "block"])
+    lab.add_argument("--folds", type=int, default=5)
+    lab.add_argument("--json", action="store_true")
+    lab.add_argument("--quiet", "-q", action="store_true")
+    lab.set_defaults(func=_cmd_lab)
+
+    arena = sub.add_parser("arena", help="Race every strategy on one market.")
+    _add_common(arena)
+    arena.add_argument("--permutations", type=int, default=120)
+    arena.add_argument("--json", action="store_true")
+    arena.add_argument("--quiet", "-q", action="store_true")
+    arena.set_defaults(func=_cmd_arena)
+
+    from .cross_sectional import XS_REGISTRY
+
+    portfolio = sub.add_parser(
+        "portfolio", help="Reality check for a cross-sectional (multi-asset) strategy."
+    )
+    _add_common(portfolio)
+    portfolio.add_argument(
+        "--symbols", default=None,
+        help="Comma-separated universe, e.g. SPY,QQQ,AAPL. Defaults to a 12-name universe.",
+    )
+    portfolio.add_argument("--strategy", default="xs_momentum", choices=sorted(XS_REGISTRY))
+    portfolio.add_argument("--param", action="append", metavar="NAME=VALUE")
+    portfolio.add_argument("--rebalance", default="M", help="D, W, M, Q, or a number of bars.")
+    portfolio.add_argument("--permutations", type=int, default=150)
+    portfolio.add_argument("--folds", type=int, default=4)
+    portfolio.add_argument("--json", action="store_true")
+    portfolio.add_argument("--quiet", "-q", action="store_true")
+    portfolio.set_defaults(func=_cmd_portfolio)
+
+    listing = sub.add_parser("strategies", help="List the strategy zoo.")
+    listing.set_defaults(func=_cmd_strategies)
+
+    args = parser.parse_args(argv)
+    try:
+        return args.func(args)
+    except KeyboardInterrupt:
+        return 130
+    except Exception as exc:  # noqa: BLE001
+        print(f"error: {exc}", file=sys.stderr)
+        return 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

algotrader/cross_sectional.py

@@ -0,0 +1,290 @@
+"""Cross-sectional strategies.
+
+Where a single-asset rule asks "should I be long this thing?", a
+cross-sectional rule asks "which of these things should I be long, and which
+short?". That difference matters for validation: a long-short book that ranks
+names is exposed to entirely different failure modes than a timing rule, and it
+needs its own null (see :mod:`algotrader.validation.cross_permutation`).
+
+Each strategy is a pure function ``(panel, **params) -> T x N weights``, causal
+by construction, with gross exposure of at most 1.
+"""
+
+from __future__ import annotations
+
+import itertools
+from dataclasses import dataclass
+from typing import Callable, Dict, Iterable, List
+
+import numpy as np
+import pandas as pd
+
+from .panel import Panel
+from .strategies import ParamSpec
+
+__all__ = [
+    "CrossSectionalStrategy",
+    "XS_REGISTRY",
+    "get_xs_strategy",
+    "list_xs_strategies",
+    "scores_to_weights",
+]
+
+MIN_NAMES = 4  # below this, "cross-section" is not a meaningful word
+
+
+def scores_to_weights(
+    scores: pd.DataFrame,
+    long_frac: float = 0.3,
+    short_frac: float = 0.3,
+    long_only: bool = False,
+    min_names: int = MIN_NAMES,
+) -> pd.DataFrame:
+    """Turn a score matrix into a dollar-neutral (or long-only) weight matrix.
+
+    Ranks within each date, takes the top and bottom fractions, and equal-weights
+    each leg. Gross exposure is 1: half per leg when short-selling, all of it in
+    the long leg otherwise.
+    """
+    valid = scores.notna()
+    counts = valid.sum(axis=1)
+    ranks = scores.rank(axis=1, pct=True, na_option="keep")
+
+    longs = (ranks > 1.0 - long_frac) & valid
+    n_long = longs.sum(axis=1).replace(0, np.nan)
+
+    if long_only:
+        weights = longs.astype(float).div(n_long, axis=0)
+    else:
+        shorts = (ranks <= short_frac) & valid
+        n_short = shorts.sum(axis=1).replace(0, np.nan)
+        weights = (
+            longs.astype(float).div(n_long, axis=0) * 0.5
+            - shorts.astype(float).div(n_short, axis=0) * 0.5
+        )
+
+    # A cross-section of two names is not a cross-section.
+    weights = weights.where(counts >= min_names, 0.0)
+    return weights.fillna(0.0)
+
+
+def _rebalance_hold(weights: pd.DataFrame, every: int) -> pd.DataFrame:
+    """Refresh the target only every ``every`` bars, holding it in between."""
+    if every <= 1:
+        return weights
+    out = weights.copy()
+    keep = np.zeros(len(out), dtype=bool)
+    keep[::every] = True
+    out.iloc[~keep] = np.nan
+    return out.ffill().fillna(0.0)
+
+
+# --------------------------------------------------------------------------
+# Strategy implementations
+# --------------------------------------------------------------------------
+
+def _xs_momentum(
+    panel: Panel, lookback: int = 250, skip: int = 20, rebalance: int = 21, long_frac: float = 0.3
+) -> pd.DataFrame:
+    """Classic 12-1 momentum: rank on past return, skipping the most recent month.
+
+    The skip is not decoration -- including the last month mixes in short-term
+    reversal, which points the other way and muddies the signal.
+    """
+    close = panel.close
+    scores = close.shift(skip) / close.shift(lookback) - 1.0
+    return _rebalance_hold(scores_to_weights(scores, long_frac, long_frac), rebalance)
+
+
+def _xs_reversal(
+    panel: Panel, lookback: int = 5, rebalance: int = 5, long_frac: float = 0.3
+) -> pd.DataFrame:
+    """Short-term reversal: buy the recent losers, sell the recent winners."""
+    scores = -(panel.close.pct_change(lookback))
+    return _rebalance_hold(scores_to_weights(scores, long_frac, long_frac), rebalance)
+
+
+def _low_volatility(
+    panel: Panel, window: int = 60, rebalance: int = 21, long_frac: float = 0.3
+) -> pd.DataFrame:
+    """The low-volatility anomaly: long the calm names, short the wild ones."""
+    scores = -(panel.close.pct_change().rolling(window, min_periods=window).std())
+    return _rebalance_hold(scores_to_weights(scores, long_frac, long_frac), rebalance)
+
+
+def _xs_value_proxy(
+    panel: Panel, window: int = 250, rebalance: int = 21, long_frac: float = 0.3
+) -> pd.DataFrame:
+    """Distance below the long-run average price, as a crude cheapness proxy.
+
+    This is not book-to-market -- there are no fundamentals in the panel -- so
+    treat it as mean reversion over a long horizon rather than value investing.
+    """
+    close = panel.close
+    scores = -(close / close.rolling(window, min_periods=window).mean() - 1.0)
+    return _rebalance_hold(scores_to_weights(scores, long_frac, long_frac), rebalance)
+
+
+def _equal_weight(panel: Panel, rebalance: int = 21) -> pd.DataFrame:
+    """Own everything tradable, equally. The bar a stock picker has to clear."""
+    listed = panel.close.notna()
+    counts = listed.sum(axis=1).replace(0, np.nan)
+    return _rebalance_hold(listed.astype(float).div(counts, axis=0).fillna(0.0), rebalance)
+
+
+def _xs_random(panel: Panel, rebalance: int = 21, seed: int = 7) -> pd.DataFrame:
+    """Random long-short book. The control group for cross-sectional claims."""
+    rng = np.random.default_rng(int(seed))
+    scores = pd.DataFrame(
+        rng.standard_normal(panel.close.shape), index=panel.index, columns=panel.symbols
+    ).where(panel.close.notna())
+    return _rebalance_hold(scores_to_weights(scores), rebalance)
+
+
+@dataclass(frozen=True)
+class CrossSectionalStrategy:
+    key: str
+    name: str
+    family: str
+    description: str
+    fn: Callable[..., pd.DataFrame]
+    params: tuple = ()
+
+    def defaults(self) -> Dict[str, float]:
+        return {p.name: p.cast(p.default) for p in self.params}
+
+    def clean(self, params: Dict[str, float] | None) -> Dict[str, float]:
+        merged = self.defaults()
+        for spec in self.params:
+            if params and spec.name in params and params[spec.name] is not None:
+                merged[spec.name] = spec.cast(params[spec.name])
+        return merged
+
+    def generate(self, panel: Panel, params: Dict[str, float] | None = None) -> pd.DataFrame:
+        weights = self.fn(panel, **self.clean(params))
+        return (
+            weights.reindex(index=panel.index, columns=panel.symbols)
+            .astype(float)
+            .fillna(0.0)
+            .clip(-1.0, 1.0)
+        )
+
+    def grid(self, limit: int | None = None) -> List[Dict[str, float]]:
+        if not self.params:
+            return [{}]
+        names = [p.name for p in self.params]
+        combos = [dict(zip(names, v)) for v in itertools.product(*[p.grid for p in self.params])]
+        combos = [c for c in combos if not ("skip" in c and "lookback" in c and c["skip"] >= c["lookback"])]
+        if limit is not None and len(combos) > limit:
+            step = len(combos) / limit
+            combos = [combos[int(i * step)] for i in range(limit)]
+        return combos
+
+
+XS_REGISTRY: Dict[str, CrossSectionalStrategy] = {}
+
+
+def _register(strategy: CrossSectionalStrategy) -> CrossSectionalStrategy:
+    XS_REGISTRY[strategy.key] = strategy
+    return strategy
+
+
+_register(
+    CrossSectionalStrategy(
+        key="equal_weight",
+        name="Equal Weight",
+        family="benchmark",
+        description="Own every name equally. The bar a stock picker has to clear.",
+        fn=_equal_weight,
+        params=(ParamSpec("rebalance", "Rebalance (bars)", 21, (5, 21, 63), "int", 1, 252, 1),),
+    )
+)
+
+_register(
+    CrossSectionalStrategy(
+        key="xs_momentum",
+        name="Cross-Sectional Momentum",
+        family="momentum",
+        description="Long the past winners, short the past losers, skipping the most recent month.",
+        fn=_xs_momentum,
+        params=(
+            ParamSpec("lookback", "Lookback", 250, (60, 120, 250), "int", 20, 750, 1),
+            ParamSpec("skip", "Skip recent", 20, (0, 5, 20), "int", 0, 60, 1),
+            ParamSpec("rebalance", "Rebalance (bars)", 21, (5, 21, 63), "int", 1, 252, 1),
+            ParamSpec("long_frac", "Leg size", 0.3, (0.1, 0.2, 0.3), "float", 0.05, 0.5, 0.05),
+        ),
+    )
+)
+
+_register(
+    CrossSectionalStrategy(
+        key="xs_reversal",
+        name="Short-Term Reversal",
+        family="mean-reversion",
+        description="Buy this week's losers and sell its winners.",
+        fn=_xs_reversal,
+        params=(
+            ParamSpec("lookback", "Lookback", 5, (1, 3, 5, 10, 21), "int", 1, 60, 1),
+            ParamSpec("rebalance", "Rebalance (bars)", 5, (1, 5, 21), "int", 1, 252, 1),
+            ParamSpec("long_frac", "Leg size", 0.3, (0.1, 0.2, 0.3), "float", 0.05, 0.5, 0.05),
+        ),
+    )
+)
+
+_register(
+    CrossSectionalStrategy(
+        key="low_volatility",
+        name="Low Volatility",
+        family="risk",
+        description="Long the calm names, short the volatile ones.",
+        fn=_low_volatility,
+        params=(
+            ParamSpec("window", "Vol window", 60, (20, 60, 120), "int", 5, 252, 1),
+            ParamSpec("rebalance", "Rebalance (bars)", 21, (5, 21, 63), "int", 1, 252, 1),
+            ParamSpec("long_frac", "Leg size", 0.3, (0.1, 0.2, 0.3), "float", 0.05, 0.5, 0.05),
+        ),
+    )
+)
+
+_register(
+    CrossSectionalStrategy(
+        key="xs_value_proxy",
+        name="Long-Horizon Reversion",
+        family="value-ish",
+        description="Long names trading below their long-run average, short those above.",
+        fn=_xs_value_proxy,
+        params=(
+            ParamSpec("window", "Window", 250, (120, 250, 500), "int", 30, 1000, 1),
+            ParamSpec("rebalance", "Rebalance (bars)", 21, (5, 21, 63), "int", 1, 252, 1),
+            ParamSpec("long_frac", "Leg size", 0.3, (0.1, 0.2, 0.3), "float", 0.05, 0.5, 0.05),
+        ),
+    )
+)
+
+_register(
+    CrossSectionalStrategy(
+        key="xs_random",
+        name="Random Book (control)",
+        family="control",
+        description="Random long-short positions. Anything that cannot beat this is noise.",
+        fn=_xs_random,
+        params=(
+            ParamSpec("rebalance", "Rebalance (bars)", 21, (5, 21, 63), "int", 1, 252, 1),
+            ParamSpec("seed", "Seed", 7, (1, 7, 42, 123), "int", 0, 9999, 1),
+        ),
+    )
+)
+
+
+def get_xs_strategy(key: str) -> CrossSectionalStrategy:
+    try:
+        return XS_REGISTRY[key]
+    except KeyError:
+        raise KeyError(
+            f"Unknown cross-sectional strategy '{key}'. Available: {', '.join(sorted(XS_REGISTRY))}"
+        ) from None
+
+
+def list_xs_strategies(exclude: Iterable[str] = ()) -> List[CrossSectionalStrategy]:
+    skip = set(exclude)
+    return [s for k, s in XS_REGISTRY.items() if k not in skip]

algotrader/data.py

@@ -0,0 +1,252 @@
+"""Market data loading with a three-tier fallback.
+
+Order of preference: live Yahoo download -> on-disk cache -> a deterministic
+simulator. The fallback exists because a Hugging Face Space that shows a
+stack trace on the first click is a Space nobody shares. When the simulator is
+used, :class:`~algotrader.types.MarketData` says so and the UI shows it.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import logging
+import os
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Optional
+
+import numpy as np
+import pandas as pd
+
+from .types import OHLCV_COLUMNS, MarketData
+
+logger = logging.getLogger(__name__)
+
+CACHE_DIR = Path(os.environ.get("ALGOTRADER_CACHE", Path.home() / ".cache" / "algotrader"))
+NETWORK_ENABLED = os.environ.get("ALGOTRADER_OFFLINE", "").lower() not in ("1", "true", "yes")
+
+# Popular tickers get hand-set simulation parameters so the offline demo is at
+# least in the right postcode: annual drift, annual vol, and a starting price.
+@dataclass(frozen=True)
+class SimProfile:
+    drift: float
+    vol: float
+    price: float
+
+
+SIM_PROFILES: dict[str, SimProfile] = {
+    "AAPL": SimProfile(0.24, 0.29, 190.0),
+    "MSFT": SimProfile(0.25, 0.27, 410.0),
+    "NVDA": SimProfile(0.55, 0.52, 120.0),
+    "TSLA": SimProfile(0.30, 0.58, 250.0),
+    "AMZN": SimProfile(0.22, 0.33, 180.0),
+    "GOOGL": SimProfile(0.20, 0.31, 170.0),
+    "META": SimProfile(0.26, 0.40, 500.0),
+    "SPY": SimProfile(0.10, 0.16, 550.0),
+    "QQQ": SimProfile(0.14, 0.21, 480.0),
+    "BTC-USD": SimProfile(0.45, 0.65, 65000.0),
+    "ETH-USD": SimProfile(0.35, 0.75, 3000.0),
+    "GLD": SimProfile(0.07, 0.14, 200.0),
+    "TLT": SimProfile(0.01, 0.15, 95.0),
+}
+
+DEFAULT_UNIVERSE = ["SPY", "AAPL", "NVDA", "MSFT", "TSLA", "QQQ", "BTC-USD", "GLD"]
+
+
+def _seed_for(symbol: str) -> int:
+    """Stable per-symbol seed so a given ticker always simulates identically."""
+    digest = hashlib.sha256(symbol.upper().encode()).digest()
+    return int.from_bytes(digest[:4], "big")
+
+
+def _normalise(df: pd.DataFrame) -> pd.DataFrame:
+    """Coerce any loader's output into a clean lowercase OHLCV frame."""
+    if isinstance(df.columns, pd.MultiIndex):
+        df = df.copy()
+        df.columns = [str(c[0]) for c in df.columns]
+    df = df.rename(columns={c: str(c).strip().lower().replace(" ", "_") for c in df.columns})
+    if "adj_close" in df.columns and "close" not in df.columns:
+        df = df.rename(columns={"adj_close": "close"})
+    missing = [c for c in OHLCV_COLUMNS if c not in df.columns]
+    for col in missing:
+        if col == "volume":
+            df["volume"] = 0.0
+        elif "close" in df.columns:
+            df[col] = df["close"]
+        else:
+            raise ValueError(f"Price data is missing required column: {col}")
+    df = df.loc[:, list(OHLCV_COLUMNS)].astype(float)
+    if not isinstance(df.index, pd.DatetimeIndex):
+        df.index = pd.to_datetime(df.index)
+    df.index = df.index.tz_localize(None) if df.index.tz is not None else df.index
+    df = df[~df.index.duplicated(keep="last")].sort_index()
+    df = df[df["close"] > 0].dropna(subset=["close"])
+    return df
+
+
+def _cache_path(symbol: str, interval: str) -> Path:
+    safe = symbol.upper().replace("/", "_")
+    return CACHE_DIR / f"{safe}_{interval}.csv"
+
+
+def _read_cache(symbol: str, interval: str) -> Optional[pd.DataFrame]:
+    path = _cache_path(symbol, interval)
+    if not path.exists():
+        return None
+    try:
+        return _normalise(pd.read_csv(path, index_col=0, parse_dates=True))
+    except Exception as exc:  # pragma: no cover - corrupted cache is not worth failing over
+        logger.warning("Ignoring unreadable cache %s: %s", path, exc)
+        return None
+
+
+def _write_cache(symbol: str, interval: str, df: pd.DataFrame) -> None:
+    try:
+        CACHE_DIR.mkdir(parents=True, exist_ok=True)
+        df.to_csv(_cache_path(symbol, interval))
+    except Exception as exc:  # pragma: no cover - a read-only FS must not break the app
+        logger.warning("Could not write cache for %s: %s", symbol, exc)
+
+
+def _download(symbol: str, start: str, end: str | None, interval: str) -> Optional[pd.DataFrame]:
+    if not NETWORK_ENABLED:
+        return None
+    try:
+        import yfinance as yf
+    except ImportError:
+        logger.info("yfinance not installed; using offline data")
+        return None
+    try:
+        raw = yf.download(
+            symbol,
+            start=start,
+            end=end,
+            interval=interval,
+            progress=False,
+            auto_adjust=True,
+            threads=False,
+        )
+    except Exception as exc:
+        logger.warning("Download failed for %s: %s", symbol, exc)
+        return None
+    if raw is None or len(raw) == 0:
+        logger.warning("Download for %s returned no rows", symbol)
+        return None
+    try:
+        return _normalise(raw)
+    except Exception as exc:
+        logger.warning("Could not normalise download for %s: %s", symbol, exc)
+        return None
+
+
+def simulate_ohlcv(
+    symbol: str = "SIM",
+    start: str = "2015-01-01",
+    end: str | None = None,
+    interval: str = "1d",
+    seed: Optional[int] = None,
+) -> pd.DataFrame:
+    """Generate a deterministic but realistic-looking OHLCV series.
+
+    This is not geometric Brownian motion with a straight face: it uses a
+    two-state (calm / stressed) regime switch, Student-t innovations and
+    GARCH-ish vol persistence, so the resulting series has fat tails and
+    volatility clustering. That matters, because a strategy tested against
+    naive GBM looks far better than it deserves to.
+    """
+    profile = SIM_PROFILES.get(symbol.upper(), SimProfile(0.08, 0.25, 100.0))
+    rng = np.random.default_rng(_seed_for(symbol) if seed is None else seed)
+
+    freq = {"1d": "B", "1wk": "W-FRI", "1h": "h"}.get(interval, "B")
+    index = pd.date_range(start=start, end=end or pd.Timestamp.today().normalize(), freq=freq)
+    n = len(index)
+    if n < 50:
+        raise ValueError("Simulated range is too short to backtest")
+
+    ppy = 252 if freq in ("B", "h") else 52
+    mu = profile.drift / ppy
+    base_vol = profile.vol / np.sqrt(ppy)
+
+    # Regime chain: calm state is sticky, stressed state is short and violent.
+    p_calm_to_stress, p_stress_to_calm = 0.01, 0.06
+    regime = np.zeros(n, dtype=int)
+    for i in range(1, n):
+        flip = rng.random()
+        if regime[i - 1] == 0:
+            regime[i] = 1 if flip < p_calm_to_stress else 0
+        else:
+            regime[i] = 0 if flip < p_stress_to_calm else 1
+
+    # Persistent vol around a regime-dependent level.
+    vol = np.empty(n)
+    level = np.where(regime == 1, base_vol * 2.4, base_vol * 0.9)
+    vol[0] = level[0]
+    for i in range(1, n):
+        vol[i] = 0.92 * vol[i - 1] + 0.08 * level[i]
+
+    shocks = rng.standard_t(df=4, size=n) / np.sqrt(2.0)  # unit-ish variance, fat tails
+    drift = np.where(regime == 1, mu - 3.0 * base_vol**2, mu)
+    log_ret = drift + vol * shocks
+    close = profile.price * np.exp(np.cumsum(log_ret))
+    close = close * (profile.price / close[-1])  # end near the quoted level
+
+    intrabar = vol * rng.uniform(0.3, 1.1, size=n)
+    open_ = close * np.exp(-log_ret * rng.uniform(0.2, 0.8, size=n))
+    high = np.maximum(open_, close) * np.exp(np.abs(intrabar))
+    low = np.minimum(open_, close) * np.exp(-np.abs(intrabar))
+    volume = rng.lognormal(mean=15.5, sigma=0.45, size=n) * (1.0 + 3.0 * regime)
+
+    return _normalise(
+        pd.DataFrame(
+            {"open": open_, "high": high, "low": low, "close": close, "volume": volume},
+            index=index,
+        )
+    )
+
+
+def load_ohlcv(
+    symbol: str = "SPY",
+    start: str = "2015-01-01",
+    end: str | None = None,
+    interval: str = "1d",
+    source: str = "yahoo",
+) -> MarketData:
+    """Load OHLCV for ``symbol``.
+
+    Default ``source='yahoo'`` requires a Yahoo download. ``auto`` still falls
+    back to cache then the simulator (Hugging Face Space). ``synthetic`` is tests only.
+    """
+    symbol = (symbol or "SPY").strip().upper()
+
+    if source == "synthetic":
+        df = simulate_ohlcv(symbol, start, end, interval)
+        return MarketData(symbol, df, "synthetic", interval, "Simulated prices (requested).")
+
+    if source in ("yahoo", "live", "auto"):
+        df = _download(symbol, start, end, interval)
+        if df is not None and len(df) > 50:
+            _write_cache(symbol, interval, df)
+            return MarketData(symbol, df, "yfinance", interval, "Live data from Yahoo Finance.")
+        if source in ("yahoo", "live"):
+            raise RuntimeError(
+                f"Yahoo returned no usable bars for {symbol}. "
+                "Check the ticker, date range, and network. "
+                "Pass source='synthetic' only for offline tests."
+            )
+
+    cached = _read_cache(symbol, interval)
+    if cached is not None and len(cached) > 50:
+        window = cached.loc[str(start) : str(end)] if end else cached.loc[str(start) :]
+        if len(window) > 50:
+            return MarketData(symbol, window, "bundled", interval, "Cached data (network unavailable).")
+
+    df = simulate_ohlcv(symbol, start, end, interval)
+    return MarketData(
+        symbol,
+        df,
+        "synthetic",
+        interval,
+        f"Live data for {symbol} was unavailable, so this run uses a deterministic "
+        "market simulator with fat tails and volatility clustering. The statistics "
+        "below are still valid — they are just measured on a simulated market.",
+    )

algotrader/engine.py

@@ -0,0 +1,140 @@
+"""Vectorised, look-ahead-free backtest engine.
+
+Contract
+--------
+A strategy emits ``target[t]``: the exposure it wants, decided using only
+information available at the close of bar ``t``. The engine holds
+``position[t] = target[t - lag]`` during bar ``t`` and credits it with that
+bar's close-to-close return. With the default ``lag=1`` this means "decide on
+today's close, hold the position through tomorrow" -- the single place where
+look-ahead could sneak in, and it is one line.
+
+Costs are charged on exposure *changes*, so a strategy that flips daily pays
+for it. Short exposure additionally accrues a borrow fee.
+"""
+
+from __future__ import annotations
+
+from typing import Optional
+
+import numpy as np
+import pandas as pd
+
+from .metrics import compute_metrics, infer_periods_per_year
+from .types import BacktestResult, CostModel
+
+__all__ = ["run_backtest", "bars_to_returns"]
+
+
+def bars_to_returns(df: pd.DataFrame) -> pd.Series:
+    """Close-to-close simple returns."""
+    return df["close"].astype(float).pct_change().fillna(0.0)
+
+
+def run_backtest(
+    df: pd.DataFrame,
+    target: pd.Series,
+    costs: CostModel | None = None,
+    lag: int = 1,
+    max_leverage: float = 1.0,
+    allow_short: bool = True,
+    initial_capital: float = 100_000.0,
+    periods_per_year: Optional[int] = None,
+    rf: float = 0.0,
+    meta: Optional[dict] = None,
+) -> BacktestResult:
+    """Run one backtest and return equity, returns and the full metric bundle."""
+    if df.empty:
+        raise ValueError("Cannot backtest an empty price frame")
+    if lag < 1:
+        raise ValueError("lag must be >= 1; lag=0 would trade on unavailable information")
+
+    costs = costs or CostModel()
+    ppy = periods_per_year or infer_periods_per_year(df.index)
+
+    asset_ret = bars_to_returns(df)
+
+    target = target.reindex(df.index).astype(float).fillna(0.0)
+    lower = -max_leverage if allow_short else 0.0
+    target = target.clip(lower, max_leverage)
+
+    position = target.shift(lag).fillna(0.0)
+
+    gross = position * asset_ret
+
+    # Turnover is measured against the *drifted* weight, not the previous
+    # target. Holding a full-notional long needs no rebalancing (the position
+    # and the portfolio grow together), but a short does: lose 10% on a 100%
+    # short and the weight drifts to -82%, so staying at -100% costs a trade.
+    # See portfolio.py for the same formula in matrix form.
+    growth = (1.0 + gross).replace(0.0, np.nan)
+    drifted = (position * (1.0 + asset_ret)) / growth
+    previous = drifted.shift(1).fillna(0.0)
+    traded = position - previous
+    trade_cost = traded.abs() * (costs.one_way_bps / 1e4)
+
+    borrow_cost = position.clip(upper=0.0).abs() * (costs.short_borrow_bps / 1e4) / ppy
+    total_cost = trade_cost + borrow_cost
+
+    net = gross - total_cost
+    equity = initial_capital * (1.0 + net).cumprod()
+    benchmark_equity = initial_capital * (1.0 + asset_ret).cumprod()
+
+    result = BacktestResult(
+        equity=equity,
+        returns=net,
+        gross_returns=gross,
+        position=position,
+        target=target,
+        costs=total_cost,
+        benchmark_equity=benchmark_equity,
+        metrics=compute_metrics(net, equity, position, ppy, rf),
+        benchmark_metrics=compute_metrics(asset_ret, benchmark_equity, None, ppy, rf),
+        meta={
+            "lag": lag,
+            "commission_bps": costs.commission_bps,
+            "slippage_bps": costs.slippage_bps,
+            "short_borrow_bps": costs.short_borrow_bps,
+            "max_leverage": max_leverage,
+            "allow_short": allow_short,
+            "initial_capital": initial_capital,
+            "periods_per_year": ppy,
+            **(meta or {}),
+        },
+    )
+    result.metrics["cost_drag_ann"] = float(total_cost.sum() / max(result.metrics.get("years", 1e-9), 1e-9))
+    result.metrics["gross_sharpe"] = float(
+        compute_metrics(gross, initial_capital * (1.0 + gross).cumprod(), None, ppy, rf).get("sharpe", 0.0)
+    )
+    return result
+
+
+def fast_sharpe(
+    asset_ret: np.ndarray,
+    target: np.ndarray,
+    one_way_bps: float,
+    lag: int,
+    periods_per_year: int,
+) -> float:
+    """Numpy-only Sharpe for hot loops (permutation tests, PBO grids).
+
+    Mirrors :func:`run_backtest` exactly for the no-borrow case; it exists only
+    because building a DataFrame 1000 times is the difference between a Space
+    that answers in 4 seconds and one nobody waits for.
+    """
+    n = asset_ret.size
+    position = np.empty(n, dtype=float)
+    position[:lag] = 0.0
+    position[lag:] = target[:-lag] if lag else target
+    gross = position * asset_ret
+    traded = np.empty(n, dtype=float)
+    traded[0] = position[0]
+    traded[1:] = np.diff(position)
+    net = gross - np.abs(traded) * (one_way_bps / 1e4)
+    net = net[np.isfinite(net)]
+    if net.size < 2:
+        return 0.0
+    sd = net.std(ddof=1)
+    if not np.isfinite(sd) or sd < 1e-12:
+        return 0.0
+    return float(net.mean() / sd * np.sqrt(periods_per_year))

algotrader/indicators.py

@@ -0,0 +1,102 @@
+"""Vectorised technical indicators.
+
+Every function takes and returns pandas objects aligned to the input index, and
+every one of them is causal: the value at bar ``t`` uses only data up to and
+including ``t``. That property is what makes the backtest engine's single
+``shift`` enough to guarantee no look-ahead.
+"""
+
+from __future__ import annotations
+
+import numpy as np
+import pandas as pd
+
+__all__ = [
+    "sma",
+    "ema",
+    "rsi",
+    "macd",
+    "bollinger",
+    "atr",
+    "donchian",
+    "zscore",
+    "roc",
+    "realised_vol",
+]
+
+
+def sma(series: pd.Series, window: int) -> pd.Series:
+    return series.rolling(window, min_periods=window).mean()
+
+
+def ema(series: pd.Series, window: int) -> pd.Series:
+    return series.ewm(span=window, adjust=False, min_periods=window).mean()
+
+
+def rsi(series: pd.Series, window: int = 14) -> pd.Series:
+    """Wilder's RSI."""
+    delta = series.diff()
+    gain = delta.clip(lower=0.0)
+    loss = -delta.clip(upper=0.0)
+    avg_gain = gain.ewm(alpha=1.0 / window, adjust=False, min_periods=window).mean()
+    avg_loss = loss.ewm(alpha=1.0 / window, adjust=False, min_periods=window).mean()
+    rs = avg_gain / avg_loss.replace(0.0, np.nan)
+    out = 100.0 - (100.0 / (1.0 + rs))
+    # avg_loss == 0 leaves rs undefined: an all-gain window is RSI 100, and a
+    # perfectly flat window (no gains either) is RSI 50.
+    flat = (avg_gain == 0.0) & (avg_loss == 0.0)
+    out = out.mask((avg_loss == 0.0) & (avg_gain > 0.0), 100.0)
+    out = out.mask(flat, 50.0)
+    return out.where(avg_gain.notna())
+
+
+def macd(
+    series: pd.Series, fast: int = 12, slow: int = 26, signal: int = 9
+) -> tuple[pd.Series, pd.Series, pd.Series]:
+    """Returns ``(macd_line, signal_line, histogram)``."""
+    macd_line = ema(series, fast) - ema(series, slow)
+    signal_line = macd_line.ewm(span=signal, adjust=False, min_periods=signal).mean()
+    return macd_line, signal_line, macd_line - signal_line
+
+
+def bollinger(
+    series: pd.Series, window: int = 20, k: float = 2.0
+) -> tuple[pd.Series, pd.Series, pd.Series]:
+    """Returns ``(lower, middle, upper)``."""
+    mid = sma(series, window)
+    sd = series.rolling(window, min_periods=window).std(ddof=0)
+    return mid - k * sd, mid, mid + k * sd
+
+
+def atr(df: pd.DataFrame, window: int = 14) -> pd.Series:
+    prev_close = df["close"].shift(1)
+    tr = pd.concat(
+        [
+            df["high"] - df["low"],
+            (df["high"] - prev_close).abs(),
+            (df["low"] - prev_close).abs(),
+        ],
+        axis=1,
+    ).max(axis=1)
+    return tr.ewm(alpha=1.0 / window, adjust=False, min_periods=window).mean()
+
+
+def donchian(df: pd.DataFrame, window: int = 20) -> tuple[pd.Series, pd.Series]:
+    """Rolling channel excluding the current bar, so a breakout test is causal."""
+    upper = df["high"].rolling(window, min_periods=window).max().shift(1)
+    lower = df["low"].rolling(window, min_periods=window).min().shift(1)
+    return lower, upper
+
+
+def zscore(series: pd.Series, window: int = 20) -> pd.Series:
+    mean = series.rolling(window, min_periods=window).mean()
+    sd = series.rolling(window, min_periods=window).std(ddof=0)
+    return (series - mean) / sd.replace(0.0, np.nan)
+
+
+def roc(series: pd.Series, window: int = 20) -> pd.Series:
+    return series.pct_change(window)
+
+
+def realised_vol(returns: pd.Series, window: int = 20, periods_per_year: int = 252) -> pd.Series:
+    return returns.rolling(window, min_periods=window).std(ddof=0) * np.sqrt(periods_per_year)

algotrader/lab.py

@@ -0,0 +1,328 @@
+"""The Lab: one call that runs a backtest and then tries to disprove it.
+
+This is the module both the Gradio Space and the CLI drive. Keeping the whole
+pipeline here means the app and the command line can never disagree about what
+a Reality Score means.
+"""
+
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass, field
+from typing import Callable, Dict, List, Optional
+
+import numpy as np
+import pandas as pd
+
+from .data import load_ohlcv
+from .engine import run_backtest
+from .metrics import infer_periods_per_year
+from .strategies import Strategy, get_strategy, list_strategies
+from .types import BacktestResult, CostModel, MarketData
+from .validation.deflated_sharpe import deflated_sharpe_ratio, min_track_record_length
+from .validation.pbo import probability_of_backtest_overfitting
+from .validation.permutation import PermutationResult, permutation_test
+from .validation.walkforward import walk_forward
+from .verdict import reality_score
+
+logger = logging.getLogger(__name__)
+
+__all__ = ["LabConfig", "LabReport", "run_lab", "run_arena"]
+
+ProgressFn = Optional[Callable[[float, str], None]]
+
+
+@dataclass
+class LabConfig:
+    symbol: str = "SPY"
+    start: str = "2015-01-01"
+    end: Optional[str] = None
+    interval: str = "1d"
+    source: str = "yahoo"
+
+    strategy: str = "sma_cross"
+    params: Dict[str, float] = field(default_factory=dict)
+
+    commission_bps: float = 1.0
+    slippage_bps: float = 2.0
+    short_borrow_bps: float = 50.0
+    lag: int = 1
+    allow_short: bool = True
+    max_leverage: float = 1.0
+    capital: float = 100_000.0
+
+    n_permutations: int = 250
+    permutation_method: str = "permute"
+    block_size: int = 20
+    wf_folds: int = 5
+    pbo_splits: int = 8
+    grid_limit: int = 40
+    seed: int = 0
+
+    def costs(self, multiplier: float = 1.0) -> CostModel:
+        return CostModel(
+            commission_bps=self.commission_bps * multiplier,
+            slippage_bps=self.slippage_bps * multiplier,
+            short_borrow_bps=self.short_borrow_bps * multiplier,
+        )
+
+
+@dataclass
+class LabReport:
+    config: LabConfig
+    market: MarketData
+    strategy: Strategy
+    params: Dict[str, float]
+    backtest: BacktestResult
+    permutation: Optional[PermutationResult] = None
+    dsr: Dict[str, float] = field(default_factory=dict)
+    pbo: Dict[str, object] = field(default_factory=dict)
+    walkforward: Dict[str, object] = field(default_factory=dict)
+    trials: Dict[str, object] = field(default_factory=dict)
+    verdict: Dict[str, object] = field(default_factory=dict)
+    cost_stress: Dict[str, float] = field(default_factory=dict)
+    benchmark_correlation: float = float("nan")
+
+
+def _trial_matrix(
+    df: pd.DataFrame,
+    strategy: Strategy,
+    cfg: LabConfig,
+    progress: ProgressFn = None,
+) -> tuple[np.ndarray, List[float], List[str]]:
+    """Backtest every parameter combination a researcher would plausibly try.
+
+    The resulting ``T x N`` return matrix feeds both the Deflated Sharpe (how
+    many variants were tried, and how spread out were they) and PBO.
+    """
+    grid = strategy.grid(limit=cfg.grid_limit)
+    costs = cfg.costs()
+    columns, sharpes, labels = [], [], []
+
+    for i, params in enumerate(grid):
+        target = strategy.generate(df, params)
+        result = run_backtest(
+            df, target, costs=costs, lag=cfg.lag,
+            max_leverage=cfg.max_leverage, allow_short=cfg.allow_short,
+        )
+        columns.append(result.returns.to_numpy(dtype=float))
+        sharpes.append(result.sharpe)
+        labels.append(", ".join(f"{k}={v}" for k, v in params.items()) or "default")
+        if progress is not None and i % 5 == 0:
+            progress((i + 1) / max(len(grid), 1), f"Variant {i + 1}/{len(grid)}")
+
+    matrix = np.column_stack(columns) if columns else np.zeros((len(df), 0))
+    return matrix, sharpes, labels
+
+
+def run_lab(cfg: LabConfig, progress: ProgressFn = None) -> LabReport:
+    """Run the full honesty pipeline for one strategy on one symbol."""
+
+    def step(fraction: float, message: str) -> None:
+        if progress is not None:
+            progress(min(max(fraction, 0.0), 1.0), message)
+
+    step(0.02, "Loading market data")
+    market = load_ohlcv(cfg.symbol, cfg.start, cfg.end, cfg.interval, cfg.source)
+    df = market.df
+    if len(df) < 120:
+        raise ValueError(
+            f"Only {len(df)} bars available for {cfg.symbol}. "
+            "Widen the date range — anything shorter cannot be validated."
+        )
+
+    strategy = get_strategy(cfg.strategy)
+    params = strategy.clean(cfg.params)
+    ppy = infer_periods_per_year(df.index)
+
+    step(0.10, "Running the backtest")
+    target = strategy.generate(df, params)
+    backtest = run_backtest(
+        df,
+        target,
+        costs=cfg.costs(),
+        lag=cfg.lag,
+        max_leverage=cfg.max_leverage,
+        allow_short=cfg.allow_short,
+        initial_capital=cfg.capital,
+        periods_per_year=ppy,
+        meta={"symbol": market.symbol, "strategy": strategy.key, "params": params},
+    )
+
+    step(0.16, "Stress-testing costs")
+    stressed = run_backtest(
+        df, target, costs=cfg.costs(3.0), lag=cfg.lag,
+        max_leverage=cfg.max_leverage, allow_short=cfg.allow_short,
+        periods_per_year=ppy,
+    )
+    base_sharpe = backtest.sharpe
+    cost_stress_ratio = float(stressed.sharpe / base_sharpe) if base_sharpe > 1e-9 else 0.0
+    cost_stress = {
+        "sharpe_1x": base_sharpe,
+        "sharpe_3x": stressed.sharpe,
+        "ratio": cost_stress_ratio,
+        "return_3x": float(stressed.metrics.get("total_return", 0.0)),
+    }
+
+    step(0.22, "Backtesting every parameter variant")
+    matrix, trial_sharpes, labels = _trial_matrix(
+        df, strategy, cfg, lambda f, m: step(0.22 + 0.18 * f, m)
+    )
+    n_trials = max(len(trial_sharpes), 1)
+
+    step(0.42, "Deflating the Sharpe ratio for selection bias")
+    dsr = deflated_sharpe_ratio(
+        backtest.returns.to_numpy(dtype=float),
+        sharpe_annual=base_sharpe,
+        periods_per_year=ppy,
+        n_trials=n_trials,
+        trial_sharpes=trial_sharpes if n_trials > 1 else None,
+    )
+    mtrl = min_track_record_length(
+        dsr["sr_per_period"], dsr["n_obs"], dsr["skew"], dsr["kurtosis"],
+        benchmark=dsr["threshold_sr_per_period"],
+    )
+    dsr["min_track_record_bars"] = mtrl
+    dsr["min_track_record_years"] = float(mtrl / ppy) if np.isfinite(mtrl) else float("inf")
+
+    step(0.46, "Measuring backtest overfitting")
+    pbo = probability_of_backtest_overfitting(matrix, n_splits=cfg.pbo_splits, labels=labels)
+
+    step(0.50, "Shuffling the market")
+    permutation = None
+    if cfg.n_permutations > 0:
+        permutation = permutation_test(
+            df,
+            lambda frame: strategy.generate(frame, params),
+            n_permutations=cfg.n_permutations,
+            method=cfg.permutation_method,
+            block=cfg.block_size,
+            costs=cfg.costs(),
+            lag=cfg.lag,
+            max_leverage=cfg.max_leverage,
+            allow_short=cfg.allow_short,
+            seed=cfg.seed,
+            observed=base_sharpe,
+            progress=lambda f, m: step(0.50 + 0.32 * f, m),
+        )
+
+    step(0.84, "Walking the strategy forward")
+    wf = walk_forward(
+        df, strategy, n_folds=cfg.wf_folds, costs=cfg.costs(), lag=cfg.lag,
+        max_leverage=cfg.max_leverage, allow_short=cfg.allow_short,
+        grid_limit=min(cfg.grid_limit, 24),
+        progress=lambda f, m: step(0.84 + 0.12 * f, m),
+    )
+
+    bench_corr = float(
+        pd.Series(backtest.returns).corr(backtest.benchmark_equity.pct_change().fillna(0.0))
+    )
+
+    step(0.98, "Grading")
+    verdict = reality_score(
+        metrics=backtest.metrics,
+        benchmark_metrics=backtest.benchmark_metrics,
+        p_value=permutation.p_value if permutation else None,
+        dsr=dsr.get("dsr"),
+        pbo=pbo.get("pbo"),
+        wf_efficiency=wf.get("efficiency"),
+        wf_win_rate=wf.get("oos_win_rate"),
+        cost_stress_ratio=cost_stress_ratio,
+        benchmark_correlation=bench_corr,
+    )
+
+    step(1.0, "Done")
+    return LabReport(
+        config=cfg,
+        market=market,
+        strategy=strategy,
+        params=params,
+        backtest=backtest,
+        permutation=permutation,
+        dsr=dsr,
+        pbo=pbo,
+        walkforward=wf,
+        trials={"n": n_trials, "sharpes": trial_sharpes, "labels": labels, "matrix_shape": matrix.shape},
+        verdict=verdict,
+        cost_stress=cost_stress,
+        benchmark_correlation=bench_corr,
+    )
+
+
+def run_arena(
+    cfg: LabConfig,
+    strategy_keys: Optional[List[str]] = None,
+    n_permutations: int = 120,
+    progress: ProgressFn = None,
+) -> tuple[pd.DataFrame, MarketData, Dict[str, BacktestResult]]:
+    """Race every strategy on the same market, ranked by evidence not returns.
+
+    Buy & hold and the coin flip stay in the field on purpose: a leaderboard
+    without a control group is marketing, not measurement.
+    """
+    market = load_ohlcv(cfg.symbol, cfg.start, cfg.end, cfg.interval, cfg.source)
+    df = market.df
+    ppy = infer_periods_per_year(df.index)
+    costs = cfg.costs()
+
+    keys = strategy_keys or [s.key for s in list_strategies()]
+    rows, curves = [], {}
+
+    for i, key in enumerate(keys):
+        strategy = get_strategy(key)
+        params = strategy.defaults()
+        target = strategy.generate(df, params)
+        result = run_backtest(
+            df, target, costs=costs, lag=cfg.lag, max_leverage=cfg.max_leverage,
+            allow_short=cfg.allow_short, initial_capital=cfg.capital, periods_per_year=ppy,
+        )
+        curves[key] = result
+
+        p_value = None
+        if n_permutations > 0:
+            p_value = permutation_test(
+                df,
+                lambda frame, s=strategy, p=params: s.generate(frame, p),
+                n_permutations=n_permutations,
+                method=cfg.permutation_method,
+                block=cfg.block_size,
+                costs=costs,
+                lag=cfg.lag,
+                max_leverage=cfg.max_leverage,
+                allow_short=cfg.allow_short,
+                seed=cfg.seed,
+                observed=result.sharpe,
+            ).p_value
+
+        grid_size = len(strategy.grid(limit=cfg.grid_limit))
+        dsr = deflated_sharpe_ratio(
+            result.returns.to_numpy(dtype=float),
+            sharpe_annual=result.sharpe,
+            periods_per_year=ppy,
+            n_trials=grid_size,
+        )
+
+        rows.append(
+            {
+                "Strategy": strategy.name,
+                "key": key,
+                "Family": strategy.family,
+                "Return": result.metrics.get("total_return", 0.0),
+                "CAGR": result.metrics.get("cagr", 0.0),
+                "Sharpe": result.sharpe,
+                "MaxDD": result.metrics.get("max_drawdown", 0.0),
+                "Trades": int(result.metrics.get("n_trades", 0)),
+                "p-value": p_value if p_value is not None else float("nan"),
+                "DSR": dsr["dsr"],
+            }
+        )
+        if progress is not None:
+            progress((i + 1) / len(keys), f"{strategy.name} ({i + 1}/{len(keys)})")
+
+    table = pd.DataFrame(rows)
+    if not table.empty:
+        # Rank by evidence: a high Sharpe with a p-value of 0.4 is not a win.
+        table["Evidence"] = (1.0 - table["p-value"].fillna(0.5)) * table["DSR"]
+        table = table.sort_values("Evidence", ascending=False).reset_index(drop=True)
+        table.insert(0, "#", table.index + 1)
+    return table, market, curves

algotrader/metrics.py

@@ -0,0 +1,157 @@
+"""Performance and risk metrics.
+
+All ratios are computed from *net* per-bar returns and annualised with the
+periodicity inferred from the index, so daily / hourly / minute series all get
+comparable numbers.
+"""
+
+from __future__ import annotations
+
+from typing import Dict
+
+import numpy as np
+import pandas as pd
+
+__all__ = [
+    "infer_periods_per_year",
+    "sharpe_ratio",
+    "sortino_ratio",
+    "max_drawdown",
+    "drawdown_series",
+    "compute_metrics",
+]
+
+_SECONDS_PER_YEAR = 365.25 * 24 * 3600
+_TRADING_DAYS = 252
+
+# A return stream with dispersion below this is constant to floating-point
+# noise. Without an absolute floor, a flat series divides by ~1e-19 and reports
+# a Sharpe of 1e16 -- the exact kind of nonsense number this project exists to
+# catch, so it must not originate here.
+_DEGENERATE_SD = 1e-12
+
+
+def infer_periods_per_year(index: pd.Index) -> int:
+    """Guess bars-per-year from an index, defaulting to daily trading bars."""
+    if not isinstance(index, pd.DatetimeIndex) or len(index) < 3:
+        return _TRADING_DAYS
+    nanos = index.to_numpy(dtype="datetime64[ns]").astype("int64")
+    deltas = np.diff(nanos) / 1e9  # seconds
+    deltas = deltas[deltas > 0]
+    if deltas.size == 0:
+        return _TRADING_DAYS
+    step = float(np.median(deltas))
+    if step >= 20 * 3600:  # daily or slower -> use trading-day convention
+        days = step / 86400.0
+        return max(1, int(round(_TRADING_DAYS / max(days / 1.4, 1.0))))
+    # Intraday: assume a 6.5h session, 252 days a year.
+    bars_per_session = (6.5 * 3600) / step
+    return max(1, int(round(bars_per_session * _TRADING_DAYS)))
+
+
+def _clean(returns: pd.Series) -> np.ndarray:
+    arr = np.asarray(returns, dtype=float)
+    return arr[np.isfinite(arr)]
+
+
+def sharpe_ratio(returns: pd.Series, periods_per_year: int, rf: float = 0.0) -> float:
+    """Annualised Sharpe. ``rf`` is an annual risk-free rate."""
+    arr = _clean(returns)
+    if arr.size < 2:
+        return 0.0
+    excess = arr - rf / periods_per_year
+    sd = excess.std(ddof=1)
+    if not np.isfinite(sd) or sd < _DEGENERATE_SD:
+        return 0.0
+    return float(excess.mean() / sd * np.sqrt(periods_per_year))
+
+
+def sortino_ratio(returns: pd.Series, periods_per_year: int, rf: float = 0.0) -> float:
+    arr = _clean(returns)
+    if arr.size < 2:
+        return 0.0
+    excess = arr - rf / periods_per_year
+    downside = excess[excess < 0]
+    if downside.size == 0:
+        return float("inf") if excess.mean() > 0 else 0.0
+    dd = np.sqrt(np.mean(downside**2))
+    if not np.isfinite(dd) or dd < _DEGENERATE_SD:
+        return 0.0
+    return float(excess.mean() / dd * np.sqrt(periods_per_year))
+
+
+def drawdown_series(equity: pd.Series) -> pd.Series:
+    peak = equity.cummax()
+    return equity / peak - 1.0
+
+
+def max_drawdown(equity: pd.Series) -> float:
+    if equity.empty:
+        return 0.0
+    return float(drawdown_series(equity).min())
+
+
+def _time_under_water(equity: pd.Series, periods_per_year: int) -> float:
+    """Longest stretch below a prior peak, in years."""
+    if equity.empty:
+        return 0.0
+    dd = drawdown_series(equity).to_numpy()
+    longest = current = 0
+    for value in dd:
+        current = current + 1 if value < 0 else 0
+        longest = max(longest, current)
+    return longest / periods_per_year
+
+
+def compute_metrics(
+    returns: pd.Series,
+    equity: pd.Series,
+    position: pd.Series | None = None,
+    periods_per_year: int | None = None,
+    rf: float = 0.0,
+) -> Dict[str, float]:
+    """Full metric bundle for one equity curve."""
+    ppy = periods_per_year or infer_periods_per_year(returns.index)
+    arr = _clean(returns)
+    n = arr.size
+    if n == 0 or equity.empty:
+        return {"periods_per_year": float(ppy)}
+
+    years = n / ppy
+    total_return = float(equity.iloc[-1] / equity.iloc[0] - 1.0)
+    cagr = float((equity.iloc[-1] / equity.iloc[0]) ** (1.0 / years) - 1.0) if years > 0 else 0.0
+    vol = float(arr.std(ddof=1) * np.sqrt(ppy))
+    mdd = max_drawdown(equity)
+    sr = sharpe_ratio(returns, ppy, rf)
+
+    out: Dict[str, float] = {
+        "total_return": total_return,
+        "cagr": cagr,
+        "ann_vol": vol,
+        "sharpe": sr,
+        "sortino": sortino_ratio(returns, ppy, rf),
+        "calmar": float(cagr / abs(mdd)) if mdd < 0 else 0.0,
+        "max_drawdown": mdd,
+        "time_under_water_yrs": _time_under_water(equity, ppy),
+        "hit_rate": float((arr > 0).mean()),
+        "skew": float(pd.Series(arr).skew()) if n > 2 else 0.0,
+        "kurtosis": float(pd.Series(arr).kurtosis()) if n > 3 else 0.0,
+        "var_95": float(np.percentile(arr, 5)),
+        "cvar_95": float(arr[arr <= np.percentile(arr, 5)].mean()) if n > 20 else 0.0,
+        "best_bar": float(arr.max()),
+        "worst_bar": float(arr.min()),
+        "n_bars": float(n),
+        "years": float(years),
+        "periods_per_year": float(ppy),
+    }
+
+    if position is not None and not position.empty:
+        pos = position.fillna(0.0)
+        turnover = pos.diff().abs().fillna(pos.abs().iloc[0] if len(pos) else 0.0)
+        out["exposure"] = float(pos.abs().mean())
+        out["long_share"] = float((pos > 0).mean())
+        out["short_share"] = float((pos < 0).mean())
+        out["turnover_ann"] = float(turnover.sum() / years) if years > 0 else 0.0
+        # A "trade" is any change in sign or size of exposure.
+        out["n_trades"] = float((turnover > 1e-9).sum())
+    return out

algotrader/panel.py

@@ -0,0 +1,283 @@
+"""Multi-asset price panels.
+
+A :class:`Panel` is a set of aligned ``T x N`` frames -- one per OHLCV field,
+one column per symbol. That is the shape cross-sectional work actually needs,
+and it is what the portfolio engine consumes.
+
+The important design choice here is that **missing data stays missing**. It is
+tempting to forward-fill a symbol through the days it did not trade, but that
+invents liquidity that never existed and quietly lets a strategy hold a
+delisted stock forever. Instead the panel tracks exactly when each symbol was
+tradable, which is also what makes survivorship measurable rather than assumed.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Dict, Iterable, List, Mapping, Optional, Sequence
+
+import numpy as np
+import pandas as pd
+
+from .data import load_ohlcv
+from .types import OHLCV_COLUMNS
+
+__all__ = ["Panel", "load_panel", "SurvivorshipReport"]
+
+
+@dataclass(frozen=True)
+class SurvivorshipReport:
+    """How much of this universe is made of winners we already know survived."""
+
+    n_symbols: int
+    n_alive_at_end: int
+    n_delisted: int
+    delisted_symbols: List[str]
+    late_starters: List[str]
+    survival_rate: float
+    biased: bool
+    note: str
+
+    def as_flag(self) -> Optional[str]:
+        return self.note if self.biased else None
+
+
+@dataclass(frozen=True)
+class Panel:
+    """Aligned multi-asset OHLCV."""
+
+    fields: Mapping[str, pd.DataFrame]
+    sources: Mapping[str, str] = field(default_factory=dict)
+    interval: str = "1d"
+    note: str = ""
+
+    def __post_init__(self) -> None:
+        missing = [c for c in OHLCV_COLUMNS if c not in self.fields]
+        if missing:
+            raise ValueError(f"Panel is missing field(s): {', '.join(missing)}")
+        reference = self.fields["close"]
+        for name, frame in self.fields.items():
+            if not frame.index.equals(reference.index) or list(frame.columns) != list(reference.columns):
+                raise ValueError(f"Panel field '{name}' is not aligned with 'close'")
+
+    # -- accessors ---------------------------------------------------------
+    @property
+    def close(self) -> pd.DataFrame:
+        return self.fields["close"]
+
+    @property
+    def open(self) -> pd.DataFrame:
+        return self.fields["open"]
+
+    @property
+    def high(self) -> pd.DataFrame:
+        return self.fields["high"]
+
+    @property
+    def low(self) -> pd.DataFrame:
+        return self.fields["low"]
+
+    @property
+    def volume(self) -> pd.DataFrame:
+        return self.fields["volume"]
+
+    @property
+    def symbols(self) -> List[str]:
+        return list(self.close.columns)
+
+    @property
+    def index(self) -> pd.DatetimeIndex:
+        return self.close.index
+
+    @property
+    def is_real(self) -> bool:
+        return all(s in ("yfinance", "bundled") for s in self.sources.values())
+
+    def __len__(self) -> int:
+        return len(self.close)
+
+    @property
+    def shape(self) -> tuple:
+        return self.close.shape
+
+    # -- derived -----------------------------------------------------------
+    def returns(self) -> pd.DataFrame:
+        """Per-asset close-to-close returns, NaN where the asset was untradable."""
+        rets = self.close.pct_change()
+        return rets.where(self.tradable())
+
+    def tradable(self) -> pd.DataFrame:
+        """True where the asset had a price on this bar *and* the one before.
+
+        A position can only be held over a bar whose return is defined, so this
+        is the mask the engine uses to zero out impossible weights.
+        """
+        listed = self.close.notna()
+        return listed & listed.shift(1, fill_value=False)
+
+    def dollar_volume(self) -> pd.DataFrame:
+        return (self.close * self.volume).where(self.close.notna())
+
+    def first_valid(self) -> pd.Series:
+        return self.close.apply(lambda col: col.first_valid_index())
+
+    def last_valid(self) -> pd.Series:
+        return self.close.apply(lambda col: col.last_valid_index())
+
+    # -- survivorship ------------------------------------------------------
+    def survivorship(self, tolerance_bars: int = 5) -> SurvivorshipReport:
+        """Measure how many names survived to the end of the sample.
+
+        A universe picked today and backfilled contains only survivors, and
+        every backtest run on it is flattered by the companies that failed and
+        were quietly excluded. We cannot fix that here, but we can refuse to
+        hide it: if every single name is still trading at the end of a long
+        sample, that is itself the evidence.
+        """
+        if not len(self):
+            return SurvivorshipReport(0, 0, 0, [], [], 1.0, False, "Empty panel.")
+
+        last = self.last_valid()
+        first = self.first_valid()
+        end = self.index[-1]
+        start = self.index[0]
+        cutoff = self.index[max(0, len(self) - 1 - tolerance_bars)]
+        entry_cutoff = self.index[min(len(self) - 1, tolerance_bars)]
+
+        delisted = sorted(str(s) for s in last.index[last < cutoff])
+        late = sorted(str(s) for s in first.index[first > entry_cutoff])
+        n = len(self.symbols)
+        alive = n - len(delisted)
+        rate = alive / n if n else 1.0
+
+        years = len(self) / 252.0
+        biased = rate >= 1.0 and years >= 3 and n >= 5
+        if biased:
+            note = (
+                f"All {n} symbols were still trading at the end of a {years:.1f}-year sample. "
+                "A universe with no failures in it was almost certainly chosen after the fact, "
+                "which means these results exclude every name that went to zero. Treat the "
+                "returns below as an upper bound."
+            )
+        elif n == 0:
+            note = "Empty panel."
+        else:
+            note = (
+                f"{len(delisted)} of {n} symbols stopped trading before the end of the sample "
+                f"({rate:.0%} survived), so the universe is not made purely of winners."
+            )
+
+        return SurvivorshipReport(
+            n_symbols=n,
+            n_alive_at_end=alive,
+            n_delisted=len(delisted),
+            delisted_symbols=delisted[:25],
+            late_starters=late[:25],
+            survival_rate=float(rate),
+            biased=bool(biased),
+            note=note,
+        )
+
+    # -- construction ------------------------------------------------------
+    @classmethod
+    def from_frames(
+        cls,
+        frames: Mapping[str, pd.DataFrame],
+        sources: Optional[Mapping[str, str]] = None,
+        interval: str = "1d",
+        note: str = "",
+        min_bars: int = 2,
+    ) -> "Panel":
+        """Build a panel from ``{symbol: ohlcv_frame}``, aligning on the union index."""
+        usable = {
+            str(symbol): frame
+            for symbol, frame in frames.items()
+            if frame is not None and len(frame) >= min_bars
+        }
+        if not usable:
+            raise ValueError("No symbol had enough data to build a panel")
+
+        index = pd.DatetimeIndex([])
+        for frame in usable.values():
+            index = index.union(pd.DatetimeIndex(frame.index))
+        index = index.sort_values()
+
+        fields: Dict[str, pd.DataFrame] = {}
+        for column in OHLCV_COLUMNS:
+            fields[column] = pd.DataFrame(
+                {
+                    symbol: pd.to_numeric(frame[column], errors="coerce").reindex(index)
+                    for symbol, frame in usable.items()
+                },
+                index=index,
+            )
+
+        return cls(
+            fields=fields,
+            sources=dict(sources or {s: "unknown" for s in usable}),
+            interval=interval,
+            note=note,
+        )
+
+    def select(self, symbols: Sequence[str]) -> "Panel":
+        keep = [s for s in symbols if s in self.close.columns]
+        if not keep:
+            raise ValueError("None of the requested symbols are in this panel")
+        return Panel(
+            fields={name: frame.loc[:, keep] for name, frame in self.fields.items()},
+            sources={s: self.sources.get(s, "unknown") for s in keep},
+            interval=self.interval,
+            note=self.note,
+        )
+
+    def slice(self, start=None, end=None) -> "Panel":
+        return Panel(
+            fields={name: frame.loc[start:end] for name, frame in self.fields.items()},
+            sources=dict(self.sources),
+            interval=self.interval,
+            note=self.note,
+        )
+
+
+def load_panel(
+    symbols: Iterable[str],
+    start: str = "2015-01-01",
+    end: Optional[str] = None,
+    interval: str = "1d",
+    source: str = "auto",
+    min_bars: int = 120,
+) -> Panel:
+    """Load a panel for ``symbols``, skipping any that cannot supply enough history."""
+    symbols = [str(s).strip().upper() for s in symbols if str(s).strip()]
+    if not symbols:
+        raise ValueError("No symbols requested")
+
+    frames: Dict[str, pd.DataFrame] = {}
+    sources: Dict[str, str] = {}
+    skipped: List[str] = []
+
+    for symbol in dict.fromkeys(symbols):  # de-duplicate, keep order
+        market = load_ohlcv(symbol, start, end, interval, source)
+        if len(market.df) < min_bars:
+            skipped.append(symbol)
+            continue
+        frames[symbol] = market.df
+        sources[symbol] = market.source
+
+    if not frames:
+        raise ValueError(
+            f"None of {len(symbols)} symbols returned at least {min_bars} bars."
+        )
+
+    simulated = sorted(s for s, src in sources.items() if src == "synthetic")
+    note = ""
+    if simulated:
+        note = (
+            f"{len(simulated)} of {len(frames)} symbols fell back to the market simulator "
+            f"({', '.join(simulated[:6])}{'...' if len(simulated) > 6 else ''}). "
+            "The statistics are still valid; they are measured on a simulated market."
+        )
+    if skipped:
+        note = (note + " " if note else "") + f"Skipped for insufficient history: {', '.join(skipped[:6])}."
+
+    return Panel.from_frames(frames, sources, interval=interval, note=note.strip())

algotrader/portfolio.py

@@ -0,0 +1,257 @@
+"""Matrix portfolio engine.
+
+The single-asset engine treats turnover as ``|target[t] - target[t-1]|``. That
+is wrong the moment weights are fractional, because a position you did not
+touch still *drifts*: hold 50% of your book in a name that doubles and you are
+now at 67% without trading. Charging costs against the previous target instead
+of the previous *actual* weight understates the cost of doing nothing and
+overstates the cost of rebalancing.
+
+This module models the drift explicitly and closes the gap:
+
+    w_start[t] = target[t - lag]                     what we want to hold
+    r_p[t]     = sum(w_start[t] * R[t])              portfolio return that bar
+    w_end[t]   = w_start[t] * (1 + R[t]) / (1 + r_p[t])   drifted by the bar
+    turnover[t] = sum |w_start[t] - w_end[t - 1]|    what we actually traded
+
+Every step is a function of the current bar and the one before, so the whole
+thing stays vectorised -- no Python loop over time.
+"""
+
+from __future__ import annotations
+
+from typing import Optional
+
+import numpy as np
+import pandas as pd
+
+from .metrics import compute_metrics, infer_periods_per_year
+from .panel import Panel
+from .types import BacktestResult, CostModel
+
+__all__ = ["run_portfolio_backtest", "PortfolioResult", "normalise_weights"]
+
+
+class PortfolioResult(BacktestResult):
+    """A :class:`BacktestResult` that also keeps the per-asset weight history."""
+
+    def __init__(self, *args, weights: pd.DataFrame, held: pd.DataFrame, panel: Panel, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.weights = weights  # requested, post-constraint
+        self.held = held  # actually held during each bar
+        self.panel = panel
+
+    def attribution(self) -> pd.Series:
+        """Total return contribution per symbol, largest first."""
+        contrib = (self.held * self.panel.returns().fillna(0.0)).sum(axis=0)
+        return contrib.sort_values(ascending=False)
+
+
+def normalise_weights(
+    weights: pd.DataFrame,
+    listed: pd.DataFrame,
+    gross_leverage: float = 1.0,
+    max_weight: Optional[float] = None,
+    allow_short: bool = True,
+) -> pd.DataFrame:
+    """Apply the constraints a real book has, in the order a real book applies them.
+
+    ``listed`` is "did this name have a price when the decision was made" --
+    deliberately not the stricter "is a return defined over this bar" mask. A
+    decision taken at Monday's close only needs Monday's price to exist; whether
+    the position can actually be carried is enforced after the lag shift.
+    """
+    w = weights.reindex(index=listed.index, columns=listed.columns).astype(float).fillna(0.0)
+
+    # You cannot ask for exposure to something that is not listed yet.
+    w = w.where(listed, 0.0)
+
+    if not allow_short:
+        w = w.clip(lower=0.0)
+    if max_weight is not None:
+        w = w.clip(-abs(max_weight), abs(max_weight))
+
+    # Scale down (never up) so gross exposure respects the leverage cap.
+    gross = w.abs().sum(axis=1)
+    scale = np.minimum(1.0, gross_leverage / gross.replace(0.0, np.nan))
+    return w.mul(scale.fillna(1.0), axis=0)
+
+
+def run_portfolio_backtest(
+    panel: Panel,
+    weights: pd.DataFrame,
+    costs: Optional[CostModel] = None,
+    lag: int = 1,
+    gross_leverage: float = 1.0,
+    max_weight: Optional[float] = None,
+    allow_short: bool = True,
+    initial_capital: float = 100_000.0,
+    periods_per_year: Optional[int] = None,
+    rf: float = 0.0,
+    benchmark: Optional[pd.Series] = None,
+    rebalance_on: Optional[pd.Series] = None,
+    meta: Optional[dict] = None,
+) -> PortfolioResult:
+    """Backtest a ``T x N`` weight matrix against a panel.
+
+    ``weights[t]`` is the exposure decided using information up to the close of
+    bar ``t``; it is held from bar ``t + lag``. The default benchmark is the
+    equal-weight universe, which is a far more honest comparison for a
+    cross-sectional strategy than any single ticker.
+    """
+    if len(panel) < 2:
+        raise ValueError("Cannot backtest a panel with fewer than two bars")
+    if lag < 1:
+        raise ValueError("lag must be >= 1; lag=0 would trade on unavailable information")
+
+    costs = costs or CostModel()
+    ppy = periods_per_year or infer_periods_per_year(panel.index)
+
+    asset_returns = panel.returns().fillna(0.0)
+    tradable = panel.tradable()
+    listed = panel.close.notna()
+
+    target = normalise_weights(weights, listed, gross_leverage, max_weight, allow_short)
+    held = target.shift(lag).fillna(0.0)
+    # Re-apply tradability after the shift: a name can delist between the
+    # decision and the fill, and we must not be holding it when it does.
+    held = held.where(tradable, 0.0)
+
+    if rebalance_on is not None:
+        held = _apply_rebalance_schedule(held, asset_returns, tradable, rebalance_on)
+
+    gross_return = (held * asset_returns).sum(axis=1)
+
+    # Weights after the bar's move, renormalised to the new portfolio value.
+    growth = (1.0 + gross_return).replace(0.0, np.nan)
+    drifted = (held * (1.0 + asset_returns)).div(growth, axis=0).fillna(0.0)
+    previous = drifted.shift(1).fillna(0.0)
+
+    traded = (held - previous).abs().sum(axis=1)
+    trade_cost = traded * (costs.one_way_bps / 1e4)
+    borrow_cost = held.clip(upper=0.0).abs().sum(axis=1) * (costs.short_borrow_bps / 1e4) / ppy
+    total_cost = trade_cost + borrow_cost
+
+    net = gross_return - total_cost
+    equity = initial_capital * (1.0 + net).cumprod()
+
+    if benchmark is None:
+        # Equal weight across whatever was tradable on each bar.
+        counts = tradable.sum(axis=1).replace(0, np.nan)
+        equal = tradable.astype(float).div(counts, axis=0).fillna(0.0)
+        benchmark = (equal.shift(lag).fillna(0.0) * asset_returns).sum(axis=1)
+    benchmark = benchmark.reindex(panel.index).fillna(0.0)
+    benchmark_equity = initial_capital * (1.0 + benchmark).cumprod()
+
+    exposure = held.abs().sum(axis=1)
+    metrics = compute_metrics(net, equity, exposure, ppy, rf)
+    metrics.update(_portfolio_metrics(held, traded, metrics.get("years", 1.0)))
+
+    survivorship = panel.survivorship()
+
+    result = PortfolioResult(
+        equity=equity,
+        returns=net,
+        gross_returns=gross_return,
+        position=exposure,
+        target=target.abs().sum(axis=1),
+        costs=total_cost,
+        benchmark_equity=benchmark_equity,
+        metrics=metrics,
+        benchmark_metrics=compute_metrics(benchmark, benchmark_equity, None, ppy, rf),
+        meta={
+            "lag": lag,
+            "gross_leverage": gross_leverage,
+            "max_weight": max_weight,
+            "allow_short": allow_short,
+            "initial_capital": initial_capital,
+            "periods_per_year": ppy,
+            "n_symbols": len(panel.symbols),
+            "survivorship": survivorship,
+            **(meta or {}),
+        },
+        weights=target,
+        held=held,
+        panel=panel,
+    )
+    return result
+
+
+def _apply_rebalance_schedule(
+    held: pd.DataFrame,
+    asset_returns: pd.DataFrame,
+    tradable: pd.DataFrame,
+    rebalance_on: pd.Series,
+) -> pd.DataFrame:
+    """Trade only on rebalance bars; let the book drift in between.
+
+    Without this, a monthly strategy whose target is constant between
+    rebalances gets charged turnover every single bar for holding still --
+    the model would be paying to *prevent* drift that a real book simply lets
+    happen. This is the one genuinely recursive step in the engine: today's
+    holding depends on yesterday's drifted holding.
+    """
+    schedule = rebalance_on.reindex(held.index).fillna(False).to_numpy(dtype=bool)
+    target = held.to_numpy(dtype=float)
+    returns = asset_returns.to_numpy(dtype=float)
+    can_hold = tradable.to_numpy(dtype=bool)
+
+    n_bars, n_assets = target.shape
+    out = np.zeros((n_bars, n_assets), dtype=float)
+    carried = np.zeros(n_assets, dtype=float)
+
+    for t in range(n_bars):
+        current = target[t] if schedule[t] else carried
+        current = np.where(can_hold[t], current, 0.0)
+        out[t] = current
+        # Drift into the next bar, renormalised to the new portfolio value.
+        portfolio_return = float(current @ returns[t])
+        growth = 1.0 + portfolio_return
+        carried = current * (1.0 + returns[t]) / growth if abs(growth) > 1e-12 else current
+
+    return pd.DataFrame(out, index=held.index, columns=held.columns)
+
+
+def rebalance_schedule(index: pd.DatetimeIndex, frequency: str = "M") -> pd.Series:
+    """Boolean per-bar mask marking rebalance dates.
+
+    ``frequency`` is ``D`` (every bar), ``W``, ``M``, ``Q``, or an integer
+    number of bars as a string.
+    """
+    frequency = str(frequency).upper().strip()
+    if frequency in ("D", "1", "B", ""):
+        return pd.Series(True, index=index)
+    if frequency.isdigit():
+        step = max(1, int(frequency))
+        mask = np.zeros(len(index), dtype=bool)
+        mask[::step] = True
+        return pd.Series(mask, index=index)
+
+    periods = {"W": index.to_period("W"), "M": index.to_period("M"), "Q": index.to_period("Q")}
+    if frequency not in periods:
+        raise ValueError(f"Unknown rebalance frequency '{frequency}'")
+    period = periods[frequency]
+    # First bar of each period -- known at the time, unlike the last bar.
+    return pd.Series(period != pd.Series(period, index=index).shift(1).to_numpy(), index=index)
+
+
+def _portfolio_metrics(held: pd.DataFrame, traded: pd.Series, years: float) -> dict:
+    """Book-level statistics a portfolio manager will look for first."""
+    absolute = held.abs()
+    gross = absolute.sum(axis=1)
+    active = (absolute > 1e-9).sum(axis=1)
+
+    # Herfindahl on the gross book: 1.0 is everything in one name, 1/n is even.
+    shares = absolute.div(gross.replace(0.0, np.nan), axis=0)
+    hhi = (shares**2).sum(axis=1)
+
+    return {
+        "gross_exposure": float(gross.mean()),
+        "net_exposure": float(held.sum(axis=1).mean()),
+        "max_gross_exposure": float(gross.max()),
+        "avg_positions": float(active.mean()),
+        "max_positions": float(active.max()),
+        "concentration_hhi": float(hhi.mean(skipna=True)) if hhi.notna().any() else float("nan"),
+        "turnover_ann": float(traded.sum() / years) if years > 0 else 0.0,
+        "n_trades": float((traded > 1e-9).sum()),
+    }

algotrader/portfolio_lab.py

@@ -0,0 +1,319 @@
+"""The Portfolio Lab: the honesty pipeline for cross-sectional strategies.
+
+Same idea as :mod:`algotrader.lab`, but the questions change when you move from
+one asset to many. A timing rule has to prove the market had structure. A book
+that ranks names has to prove three harder things:
+
+1. it picked the right names (cross-sectional permutation);
+2. what it picked is not just a style you could buy in an ETF (attribution);
+3. the universe it picked from contains the losers as well as the winners
+   (survivorship).
+
+All three are wired into the Reality Score alongside the usual selection-bias
+and walk-forward machinery.
+"""
+
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass, field
+from typing import Callable, Dict, List, Optional, Sequence
+
+import numpy as np
+import pandas as pd
+
+from .attribution import build_style_factors, factor_attribution
+from .cross_sectional import CrossSectionalStrategy, get_xs_strategy, list_xs_strategies
+from .metrics import infer_periods_per_year
+from .panel import Panel, load_panel
+from .portfolio import PortfolioResult, rebalance_schedule, run_portfolio_backtest
+from .types import CostModel
+from .validation.cross_permutation import CrossPermutationResult, cross_sectional_permutation_test
+from .validation.deflated_sharpe import deflated_sharpe_ratio
+from .validation.pbo import probability_of_backtest_overfitting
+from .validation.walkforward import walk_forward_panel
+from .verdict import reality_score
+
+logger = logging.getLogger(__name__)
+
+__all__ = ["PortfolioLabConfig", "PortfolioLabReport", "run_portfolio_lab", "run_portfolio_arena"]
+
+ProgressFn = Optional[Callable[[float, str], None]]
+
+DEFAULT_UNIVERSE = [
+    "SPY", "QQQ", "AAPL", "MSFT", "NVDA", "AMZN",
+    "META", "TSLA", "GOOGL", "GLD", "TLT", "BTC-USD",
+]
+
+
+@dataclass
+class PortfolioLabConfig:
+    symbols: Sequence[str] = tuple(DEFAULT_UNIVERSE)
+    start: str = "2015-01-01"
+    end: Optional[str] = None
+    interval: str = "1d"
+    source: str = "yahoo"
+
+    strategy: str = "xs_momentum"
+    params: Dict[str, float] = field(default_factory=dict)
+
+    commission_bps: float = 1.0
+    slippage_bps: float = 2.0
+    short_borrow_bps: float = 50.0
+    lag: int = 1
+    gross_leverage: float = 1.0
+    max_weight: Optional[float] = 0.25
+    allow_short: bool = True
+    rebalance: str = "M"
+    capital: float = 1_000_000.0
+
+    n_permutations: int = 150
+    wf_folds: int = 4
+    pbo_splits: int = 8
+    grid_limit: int = 24
+    seed: int = 0
+
+    def costs(self, multiplier: float = 1.0) -> CostModel:
+        return CostModel(
+            commission_bps=self.commission_bps * multiplier,
+            slippage_bps=self.slippage_bps * multiplier,
+            short_borrow_bps=self.short_borrow_bps * multiplier,
+        )
+
+
+@dataclass
+class PortfolioLabReport:
+    config: PortfolioLabConfig
+    panel: Panel
+    strategy: CrossSectionalStrategy
+    params: Dict[str, float]
+    backtest: PortfolioResult
+    permutation: Optional[CrossPermutationResult] = None
+    dsr: Dict[str, float] = field(default_factory=dict)
+    pbo: Dict[str, object] = field(default_factory=dict)
+    walkforward: Dict[str, object] = field(default_factory=dict)
+    attribution: Dict[str, object] = field(default_factory=dict)
+    trials: Dict[str, object] = field(default_factory=dict)
+    verdict: Dict[str, object] = field(default_factory=dict)
+    cost_stress: Dict[str, float] = field(default_factory=dict)
+
+    @property
+    def survivorship(self):
+        return self.panel.survivorship()
+
+
+def _trial_matrix(
+    panel: Panel,
+    strategy: CrossSectionalStrategy,
+    cfg: PortfolioLabConfig,
+    schedule: pd.Series,
+    progress: ProgressFn = None,
+) -> tuple[np.ndarray, List[float], List[str]]:
+    """Backtest every parameter variant, for the Deflated Sharpe and PBO inputs."""
+    grid = strategy.grid(limit=cfg.grid_limit)
+    costs = cfg.costs()
+    columns, sharpes, labels = [], [], []
+
+    for i, params in enumerate(grid):
+        weights = strategy.generate(panel, params)
+        result = run_portfolio_backtest(
+            panel, weights, costs=costs, lag=cfg.lag,
+            gross_leverage=cfg.gross_leverage, max_weight=cfg.max_weight,
+            allow_short=cfg.allow_short, rebalance_on=schedule,
+        )
+        columns.append(result.returns.to_numpy(dtype=float))
+        sharpes.append(result.sharpe)
+        labels.append(", ".join(f"{k}={v}" for k, v in params.items()) or "default")
+        if progress is not None and i % 3 == 0:
+            progress((i + 1) / max(len(grid), 1), f"Variant {i + 1}/{len(grid)}")
+
+    matrix = np.column_stack(columns) if columns else np.zeros((len(panel), 0))
+    return matrix, sharpes, labels
+
+
+def run_portfolio_lab(cfg: PortfolioLabConfig, progress: ProgressFn = None) -> PortfolioLabReport:
+    """Run the full cross-sectional honesty pipeline."""
+
+    def step(fraction: float, message: str) -> None:
+        if progress is not None:
+            progress(min(max(fraction, 0.0), 1.0), message)
+
+    step(0.02, f"Loading {len(cfg.symbols)} symbols")
+    panel = load_panel(cfg.symbols, cfg.start, cfg.end, cfg.interval, cfg.source)
+    if len(panel) < 250:
+        raise ValueError(
+            f"Only {len(panel)} bars available. Widen the date range — a cross-sectional "
+            "book cannot be validated on less than a year of data."
+        )
+
+    strategy = get_xs_strategy(cfg.strategy)
+    params = strategy.clean(cfg.params)
+    ppy = infer_periods_per_year(panel.index)
+    schedule = rebalance_schedule(panel.index, cfg.rebalance)
+
+    step(0.10, "Running the backtest")
+    weights = strategy.generate(panel, params)
+    backtest = run_portfolio_backtest(
+        panel, weights, costs=cfg.costs(), lag=cfg.lag,
+        gross_leverage=cfg.gross_leverage, max_weight=cfg.max_weight,
+        allow_short=cfg.allow_short, initial_capital=cfg.capital,
+        periods_per_year=ppy, rebalance_on=schedule,
+        meta={"strategy": strategy.key, "params": params, "rebalance": cfg.rebalance},
+    )
+
+    step(0.16, "Stress-testing costs")
+    stressed = run_portfolio_backtest(
+        panel, weights, costs=cfg.costs(3.0), lag=cfg.lag,
+        gross_leverage=cfg.gross_leverage, max_weight=cfg.max_weight,
+        allow_short=cfg.allow_short, periods_per_year=ppy, rebalance_on=schedule,
+    )
+    base_sharpe = backtest.sharpe
+    cost_stress = {
+        "sharpe_1x": base_sharpe,
+        "sharpe_3x": stressed.sharpe,
+        "ratio": float(stressed.sharpe / base_sharpe) if base_sharpe > 1e-9 else 0.0,
+        "return_3x": float(stressed.metrics.get("total_return", 0.0)),
+    }
+
+    step(0.22, "Backtesting every parameter variant")
+    matrix, trial_sharpes, labels = _trial_matrix(
+        panel, strategy, cfg, schedule, lambda f, m: step(0.22 + 0.14 * f, m)
+    )
+    n_trials = max(len(trial_sharpes), 1)
+
+    step(0.38, "Deflating the Sharpe ratio for selection bias")
+    dsr = deflated_sharpe_ratio(
+        backtest.returns.to_numpy(dtype=float),
+        sharpe_annual=base_sharpe,
+        periods_per_year=ppy,
+        n_trials=n_trials,
+        trial_sharpes=trial_sharpes if n_trials > 1 else None,
+    )
+
+    step(0.42, "Measuring backtest overfitting")
+    pbo = probability_of_backtest_overfitting(matrix, n_splits=cfg.pbo_splits, labels=labels)
+
+    step(0.46, "Shuffling names within each date")
+    permutation = None
+    if cfg.n_permutations > 0:
+        permutation = cross_sectional_permutation_test(
+            panel, weights, n_permutations=cfg.n_permutations, lag=cfg.lag,
+            gross_leverage=cfg.gross_leverage, max_weight=cfg.max_weight,
+            allow_short=cfg.allow_short, rebalance_on=schedule, seed=cfg.seed,
+            progress=lambda f, m: step(0.46 + 0.30 * f, m),
+        )
+
+    step(0.78, "Attributing returns to style factors")
+    try:
+        factors = build_style_factors(panel)
+        attribution = factor_attribution(backtest.returns, factors, ppy)
+    except Exception as exc:  # noqa: BLE001 - attribution must never sink a run
+        logger.warning("Attribution failed: %s", exc)
+        attribution = {"available": False, "note": f"Attribution unavailable: {exc}"}
+
+    step(0.86, "Walking the strategy forward")
+    wf = walk_forward_panel(
+        panel, strategy, n_folds=cfg.wf_folds, costs=cfg.costs(), lag=cfg.lag,
+        gross_leverage=cfg.gross_leverage, allow_short=cfg.allow_short,
+        rebalance=cfg.rebalance, grid_limit=min(cfg.grid_limit, 12),
+        progress=lambda f, m: step(0.86 + 0.10 * f, m),
+    )
+
+    step(0.98, "Grading")
+    survivorship = panel.survivorship()
+    verdict = reality_score(
+        metrics=backtest.metrics,
+        benchmark_metrics=backtest.benchmark_metrics,
+        p_value=permutation.p_value if permutation else None,
+        dsr=dsr.get("dsr"),
+        pbo=pbo.get("pbo"),
+        wf_efficiency=wf.get("efficiency"),
+        wf_win_rate=wf.get("oos_win_rate"),
+        cost_stress_ratio=cost_stress["ratio"],
+        attribution=attribution,
+        survivorship=survivorship,
+        benchmark_name="The equal-weight universe",
+        permutation_label="books with the same shape but randomly chosen names",
+    )
+
+    step(1.0, "Done")
+    return PortfolioLabReport(
+        config=cfg,
+        panel=panel,
+        strategy=strategy,
+        params=params,
+        backtest=backtest,
+        permutation=permutation,
+        dsr=dsr,
+        pbo=pbo,
+        walkforward=wf,
+        attribution=attribution,
+        trials={"n": n_trials, "sharpes": trial_sharpes, "labels": labels},
+        verdict=verdict,
+        cost_stress=cost_stress,
+    )
+
+
+def run_portfolio_arena(
+    cfg: PortfolioLabConfig,
+    strategy_keys: Optional[List[str]] = None,
+    n_permutations: int = 80,
+    progress: ProgressFn = None,
+) -> tuple[pd.DataFrame, Panel, Dict[str, PortfolioResult]]:
+    """Race every cross-sectional strategy on one universe, ranked by evidence."""
+    panel = load_panel(cfg.symbols, cfg.start, cfg.end, cfg.interval, cfg.source)
+    ppy = infer_periods_per_year(panel.index)
+    schedule = rebalance_schedule(panel.index, cfg.rebalance)
+    costs = cfg.costs()
+
+    keys = strategy_keys or [s.key for s in list_xs_strategies()]
+    factors = build_style_factors(panel)
+    rows, books = [], {}
+
+    for i, key in enumerate(keys):
+        strategy = get_xs_strategy(key)
+        weights = strategy.generate(panel, strategy.defaults())
+        result = run_portfolio_backtest(
+            panel, weights, costs=costs, lag=cfg.lag, gross_leverage=cfg.gross_leverage,
+            max_weight=cfg.max_weight, allow_short=cfg.allow_short,
+            initial_capital=cfg.capital, periods_per_year=ppy, rebalance_on=schedule,
+        )
+        books[key] = result
+
+        p_value = float("nan")
+        if n_permutations > 0:
+            p_value = cross_sectional_permutation_test(
+                panel, weights, n_permutations=n_permutations, lag=cfg.lag,
+                gross_leverage=cfg.gross_leverage, max_weight=cfg.max_weight,
+                allow_short=cfg.allow_short, rebalance_on=schedule, seed=cfg.seed,
+                observed=result.sharpe,
+            ).p_value
+
+        dsr = deflated_sharpe_ratio(
+            result.returns.to_numpy(dtype=float), sharpe_annual=result.sharpe,
+            periods_per_year=ppy, n_trials=len(strategy.grid(limit=cfg.grid_limit)),
+        )
+        attr = factor_attribution(result.returns, factors, ppy)
+
+        rows.append({
+            "Strategy": strategy.name,
+            "key": key,
+            "Family": strategy.family,
+            "Return": result.metrics.get("total_return", 0.0),
+            "CAGR": result.metrics.get("cagr", 0.0),
+            "Sharpe": result.sharpe,
+            "MaxDD": result.metrics.get("max_drawdown", 0.0),
+            "Turnover": result.metrics.get("turnover_ann", 0.0),
+            "p-value": p_value,
+            "DSR": dsr["dsr"],
+            "Alpha t": attr.get("alpha_t_stat", float("nan")) if attr.get("available") else float("nan"),
+        })
+        if progress is not None:
+            progress((i + 1) / len(keys), f"{strategy.name} ({i + 1}/{len(keys)})")
+
+    table = pd.DataFrame(rows)
+    if not table.empty:
+        table["Evidence"] = (1.0 - table["p-value"].fillna(0.5)) * table["DSR"]
+        table = table.sort_values("Evidence", ascending=False).reset_index(drop=True)
+        table.insert(0, "#", table.index + 1)
+    return table, panel, books

algotrader/strategies.py

@@ -0,0 +1,345 @@
+"""The strategy zoo.
+
+Every strategy is a pure function ``(df, **params) -> target exposure series``
+in ``[-1, 1]``, causal by construction. They never see costs, capital or
+execution — that is the engine's job — which is what lets the same function be
+re-run thousands of times inside the permutation and PBO machinery.
+"""
+
+from __future__ import annotations
+
+import itertools
+from dataclasses import dataclass
+from typing import Callable, Dict, Iterable, List, Sequence
+
+import numpy as np
+import pandas as pd
+
+from . import indicators as ind
+
+__all__ = ["Strategy", "ParamSpec", "REGISTRY", "get_strategy", "list_strategies"]
+
+
+@dataclass(frozen=True)
+class ParamSpec:
+    name: str
+    label: str
+    default: float
+    grid: Sequence[float]
+    kind: str = "int"
+    minimum: float | None = None
+    maximum: float | None = None
+    step: float | None = None
+
+    def cast(self, value):
+        return int(value) if self.kind == "int" else float(value)
+
+
+@dataclass(frozen=True)
+class Strategy:
+    key: str
+    name: str
+    family: str
+    description: str
+    fn: Callable[..., pd.Series]
+    params: tuple[ParamSpec, ...] = ()
+
+    def defaults(self) -> Dict[str, float]:
+        return {p.name: p.cast(p.default) for p in self.params}
+
+    def clean(self, params: Dict[str, float] | None) -> Dict[str, float]:
+        """Fill in missing params and coerce types, ignoring unknown keys."""
+        merged = self.defaults()
+        for spec in self.params:
+            if params and spec.name in params and params[spec.name] is not None:
+                merged[spec.name] = spec.cast(params[spec.name])
+        return merged
+
+    def generate(self, df: pd.DataFrame, params: Dict[str, float] | None = None) -> pd.Series:
+        target = self.fn(df, **self.clean(params))
+        return target.reindex(df.index).astype(float).fillna(0.0).clip(-1.0, 1.0)
+
+    def grid(self, limit: int | None = None) -> List[Dict[str, float]]:
+        """Cartesian product of the per-parameter grids (the 'trials' a
+        researcher would realistically run before picking a winner)."""
+        if not self.params:
+            return [{}]
+        names = [p.name for p in self.params]
+        combos = [
+            dict(zip(names, values))
+            for values in itertools.product(*[p.grid for p in self.params])
+        ]
+        combos = [c for c in combos if self._valid(c)]
+        if limit is not None and len(combos) > limit:
+            step = len(combos) / limit
+            combos = [combos[int(i * step)] for i in range(limit)]
+        return combos
+
+    def _valid(self, combo: Dict[str, float]) -> bool:
+        """Reject nonsensical combinations (a fast MA slower than the slow one)."""
+        if "fast" in combo and "slow" in combo and combo["fast"] >= combo["slow"]:
+            return False
+        if "lower" in combo and "upper" in combo and combo["lower"] >= combo["upper"]:
+            return False
+        return True
+
+
+def _hold_until_flip(raw: pd.Series) -> pd.Series:
+    """Turn sparse entry/exit signals into a continuously held position."""
+    return raw.ffill().fillna(0.0)
+
+
+# --------------------------------------------------------------------------
+# Strategy implementations
+# --------------------------------------------------------------------------
+
+def _buy_and_hold(df: pd.DataFrame) -> pd.Series:
+    return pd.Series(1.0, index=df.index)
+
+
+def _sma_cross(df: pd.DataFrame, fast: int = 20, slow: int = 100) -> pd.Series:
+    f, s = ind.sma(df["close"], fast), ind.sma(df["close"], slow)
+    return pd.Series(np.where(f > s, 1.0, -1.0), index=df.index).where(s.notna())
+
+
+def _ema_cross(df: pd.DataFrame, fast: int = 12, slow: int = 50) -> pd.Series:
+    f, s = ind.ema(df["close"], fast), ind.ema(df["close"], slow)
+    return pd.Series(np.where(f > s, 1.0, -1.0), index=df.index).where(s.notna())
+
+
+def _macd_trend(df: pd.DataFrame, fast: int = 12, slow: int = 26, signal: int = 9) -> pd.Series:
+    _, _, hist = ind.macd(df["close"], fast, slow, signal)
+    return pd.Series(np.sign(hist), index=df.index).where(hist.notna())
+
+
+def _rsi_reversion(df: pd.DataFrame, window: int = 14, lower: int = 30, upper: int = 70) -> pd.Series:
+    r = ind.rsi(df["close"], window)
+    raw = pd.Series(np.nan, index=df.index)
+    raw[r < lower] = 1.0
+    raw[r > upper] = -1.0
+    raw[(r > 45) & (r < 55)] = 0.0  # flatten in the middle of the range
+    return _hold_until_flip(raw).where(r.notna())
+
+
+def _bollinger_reversion(df: pd.DataFrame, window: int = 20, k: float = 2.0) -> pd.Series:
+    low, mid, high = ind.bollinger(df["close"], window, k)
+    close = df["close"]
+    raw = pd.Series(np.nan, index=df.index)
+    raw[close < low] = 1.0
+    raw[close > high] = -1.0
+    raw[(close - mid).abs() < 0.1 * (high - mid)] = 0.0
+    return _hold_until_flip(raw).where(mid.notna())
+
+
+def _donchian_breakout(df: pd.DataFrame, window: int = 20) -> pd.Series:
+    low, high = ind.donchian(df, window)
+    raw = pd.Series(np.nan, index=df.index)
+    raw[df["close"] > high] = 1.0
+    raw[df["close"] < low] = -1.0
+    return _hold_until_flip(raw).where(high.notna())
+
+
+def _momentum(df: pd.DataFrame, lookback: int = 60) -> pd.Series:
+    return np.sign(ind.roc(df["close"], lookback))
+
+
+def _vol_target_momentum(
+    df: pd.DataFrame, lookback: int = 60, vol_window: int = 20, target_vol: float = 15
+) -> pd.Series:
+    """Momentum sized inversely to recent volatility (targets ``target_vol`` %)."""
+    signal = np.sign(ind.roc(df["close"], lookback))
+    rv = ind.realised_vol(df["close"].pct_change(), vol_window)
+    scale = (target_vol / 100.0) / rv.replace(0.0, np.nan)
+    return (signal * scale.clip(upper=1.0)).where(rv.notna())
+
+
+def _channel_trend(df: pd.DataFrame, window: int = 50, atr_window: int = 14, mult: float = 1.0) -> pd.Series:
+    """Long above an ATR band around the mean, short below it, flat inside."""
+    mid = ind.sma(df["close"], window)
+    band = ind.atr(df, atr_window) * mult
+    raw = pd.Series(np.nan, index=df.index)
+    raw[df["close"] > mid + band] = 1.0
+    raw[df["close"] < mid - band] = -1.0
+    raw[(df["close"] - mid).abs() < 0.25 * band] = 0.0
+    return _hold_until_flip(raw).where(mid.notna() & band.notna())
+
+
+def _coin_flip(df: pd.DataFrame, hold: int = 5, seed: int = 7) -> pd.Series:
+    """A deliberately worthless strategy: the control group.
+
+    If your clever rule cannot beat this on the validation panel, that is the
+    single most useful thing this app can tell you.
+    """
+    rng = np.random.default_rng(int(seed))
+    n = len(df)
+    draws = rng.choice([-1.0, 1.0], size=int(np.ceil(n / max(hold, 1))))
+    return pd.Series(np.repeat(draws, max(hold, 1))[:n], index=df.index)
+
+
+REGISTRY: Dict[str, Strategy] = {}
+
+
+def _register(strategy: Strategy) -> Strategy:
+    REGISTRY[strategy.key] = strategy
+    return strategy
+
+
+_register(
+    Strategy(
+        key="buy_and_hold",
+        name="Buy & Hold",
+        family="benchmark",
+        description="Own the asset, do nothing. The bar every other strategy has to clear.",
+        fn=_buy_and_hold,
+    )
+)
+
+_register(
+    Strategy(
+        key="sma_cross",
+        name="SMA Crossover",
+        family="trend",
+        description="Long when the fast simple moving average is above the slow one, short when below.",
+        fn=_sma_cross,
+        params=(
+            ParamSpec("fast", "Fast MA", 20, (5, 10, 20, 30, 50), "int", 2, 100, 1),
+            ParamSpec("slow", "Slow MA", 100, (50, 100, 150, 200), "int", 10, 300, 5),
+        ),
+    )
+)
+
+_register(
+    Strategy(
+        key="ema_cross",
+        name="EMA Crossover",
+        family="trend",
+        description="Same idea as the SMA cross but with exponential averages, so it turns faster.",
+        fn=_ema_cross,
+        params=(
+            ParamSpec("fast", "Fast EMA", 12, (5, 8, 12, 21, 34), "int", 2, 100, 1),
+            ParamSpec("slow", "Slow EMA", 50, (34, 50, 89, 144, 200), "int", 10, 300, 1),
+        ),
+    )
+)
+
+_register(
+    Strategy(
+        key="macd_trend",
+        name="MACD Trend",
+        family="trend",
+        description="Follow the sign of the MACD histogram.",
+        fn=_macd_trend,
+        params=(
+            ParamSpec("fast", "Fast", 12, (8, 12, 16), "int", 2, 60, 1),
+            ParamSpec("slow", "Slow", 26, (21, 26, 34, 50), "int", 10, 200, 1),
+            ParamSpec("signal", "Signal", 9, (5, 9, 13), "int", 2, 50, 1),
+        ),
+    )
+)
+
+_register(
+    Strategy(
+        key="rsi_reversion",
+        name="RSI Mean Reversion",
+        family="mean-reversion",
+        description="Buy oversold, sell overbought, flatten in the middle of the range.",
+        fn=_rsi_reversion,
+        params=(
+            ParamSpec("window", "RSI window", 14, (7, 14, 21), "int", 2, 60, 1),
+            ParamSpec("lower", "Oversold", 30, (20, 25, 30, 35), "int", 5, 49, 1),
+            ParamSpec("upper", "Overbought", 70, (65, 70, 75, 80), "int", 51, 95, 1),
+        ),
+    )
+)
+
+_register(
+    Strategy(
+        key="bollinger_reversion",
+        name="Bollinger Reversion",
+        family="mean-reversion",
+        description="Fade moves outside the Bollinger bands and exit back at the middle band.",
+        fn=_bollinger_reversion,
+        params=(
+            ParamSpec("window", "Window", 20, (10, 20, 30, 50), "int", 5, 120, 1),
+            ParamSpec("k", "Band width (σ)", 2.0, (1.5, 2.0, 2.5, 3.0), "float", 0.5, 4.0, 0.1),
+        ),
+    )
+)
+
+_register(
+    Strategy(
+        key="donchian_breakout",
+        name="Donchian Breakout",
+        family="breakout",
+        description="The classic turtle rule: buy new highs, sell new lows.",
+        fn=_donchian_breakout,
+        params=(ParamSpec("window", "Channel", 20, (10, 20, 40, 55, 100), "int", 5, 250, 1),),
+    )
+)
+
+_register(
+    Strategy(
+        key="momentum",
+        name="Time-Series Momentum",
+        family="momentum",
+        description="Hold long if the asset is up over the lookback, short if it is down.",
+        fn=_momentum,
+        params=(ParamSpec("lookback", "Lookback", 60, (5, 10, 20, 60, 120, 250), "int", 2, 500, 1),),
+    )
+)
+
+_register(
+    Strategy(
+        key="vol_target_momentum",
+        name="Vol-Targeted Momentum",
+        family="momentum",
+        description="Momentum sized down when markets get volatile, so risk stays roughly constant.",
+        fn=_vol_target_momentum,
+        params=(
+            ParamSpec("lookback", "Lookback", 60, (20, 60, 120, 250), "int", 5, 500, 1),
+            ParamSpec("vol_window", "Vol window", 20, (10, 20, 60), "int", 5, 120, 1),
+            ParamSpec("target_vol", "Target vol %", 15, (10, 15, 20), "float", 2, 60, 1),
+        ),
+    )
+)
+
+_register(
+    Strategy(
+        key="channel_trend",
+        name="ATR Channel Trend",
+        family="trend",
+        description="Trade with the trend only once price clears an ATR band around its mean.",
+        fn=_channel_trend,
+        params=(
+            ParamSpec("window", "Mean window", 50, (20, 50, 100, 200), "int", 5, 300, 1),
+            ParamSpec("atr_window", "ATR window", 14, (7, 14, 28), "int", 2, 60, 1),
+            ParamSpec("mult", "ATR multiple", 1.0, (0.5, 1.0, 1.5, 2.0), "float", 0.1, 5.0, 0.1),
+        ),
+    )
+)
+
+_register(
+    Strategy(
+        key="coin_flip",
+        name="Coin Flip (control)",
+        family="control",
+        description="Random positions. The control group — anything that cannot beat this is noise.",
+        fn=_coin_flip,
+        params=(
+            ParamSpec("hold", "Bars per flip", 5, (1, 5, 10, 20), "int", 1, 60, 1),
+            ParamSpec("seed", "Seed", 7, (1, 7, 42, 123), "int", 0, 9999, 1),
+        ),
+    )
+)
+
+
+def get_strategy(key: str) -> Strategy:
+    try:
+        return REGISTRY[key]
+    except KeyError:
+        raise KeyError(f"Unknown strategy '{key}'. Available: {', '.join(sorted(REGISTRY))}") from None
+
+
+def list_strategies(exclude: Iterable[str] = ()) -> List[Strategy]:
+    skip = set(exclude)
+    return [s for k, s in REGISTRY.items() if k not in skip]

algotrader/types.py

@@ -0,0 +1,100 @@
+"""Core data types shared across the algotrader 2.0 stack.
+
+Everything downstream (engine, validation, UI) speaks these types, so they are
+deliberately small, immutable-ish and free of framework dependencies.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any, Dict, Optional
+
+import pandas as pd
+
+OHLCV_COLUMNS = ("open", "high", "low", "close", "volume")
+
+
+@dataclass(frozen=True)
+class MarketData:
+    """A validated OHLCV series plus provenance.
+
+    Provenance matters here: the app is about honesty, so the UI always tells
+    the user whether they are looking at real prices or a simulation.
+    """
+
+    symbol: str
+    df: pd.DataFrame
+    source: str  # "yfinance" | "bundled" | "synthetic"
+    interval: str = "1d"
+    note: str = ""
+
+    @property
+    def is_real(self) -> bool:
+        return self.source in ("yfinance", "bundled")
+
+    @property
+    def start(self) -> pd.Timestamp:
+        return self.df.index[0]
+
+    @property
+    def end(self) -> pd.Timestamp:
+        return self.df.index[-1]
+
+    def __len__(self) -> int:  # pragma: no cover - trivial
+        return len(self.df)
+
+
+@dataclass(frozen=True)
+class CostModel:
+    """Round-trip friction. All values are one-way, in basis points."""
+
+    commission_bps: float = 1.0
+    slippage_bps: float = 2.0
+    short_borrow_bps: float = 50.0  # annualised, charged on short exposure
+
+    @property
+    def one_way_bps(self) -> float:
+        return self.commission_bps + self.slippage_bps
+
+
+@dataclass
+class BacktestResult:
+    """Output of a single backtest run."""
+
+    equity: pd.Series
+    returns: pd.Series  # net of costs
+    gross_returns: pd.Series
+    position: pd.Series  # exposure actually held during each bar
+    target: pd.Series  # exposure requested by the strategy
+    costs: pd.Series
+    benchmark_equity: pd.Series
+    metrics: Dict[str, float] = field(default_factory=dict)
+    benchmark_metrics: Dict[str, float] = field(default_factory=dict)
+    meta: Dict[str, Any] = field(default_factory=dict)
+
+    @property
+    def sharpe(self) -> float:
+        return float(self.metrics.get("sharpe", 0.0))
+
+    @property
+    def n_trades(self) -> int:
+        return int(self.metrics.get("n_trades", 0))
+
+
+@dataclass
+class ValidationReport:
+    """Everything we know about how much of a backtest is luck."""
+
+    permutation_p_value: Optional[float] = None
+    permutation_null: Optional[Any] = None  # np.ndarray of null Sharpes
+    deflated_sharpe: Optional[float] = None
+    probabilistic_sharpe: Optional[float] = None
+    min_track_record_years: Optional[float] = None
+    n_trials: int = 1
+    pbo: Optional[float] = None
+    pbo_detail: Dict[str, Any] = field(default_factory=dict)
+    walkforward: Dict[str, Any] = field(default_factory=dict)
+    reality_score: float = 0.0
+    grade: str = "?"
+    verdict: str = ""
+    flags: list = field(default_factory=list)

algotrader/validation/__init__.py

@@ -0,0 +1,15 @@
+"""Statistical tests that ask "is this edge real, or did we just get lucky?"."""
+
+from .deflated_sharpe import deflated_sharpe_ratio, min_track_record_length, probabilistic_sharpe_ratio
+from .pbo import probability_of_backtest_overfitting
+from .permutation import permutation_test
+from .walkforward import walk_forward
+
+__all__ = [
+    "deflated_sharpe_ratio",
+    "probabilistic_sharpe_ratio",
+    "min_track_record_length",
+    "probability_of_backtest_overfitting",
+    "permutation_test",
+    "walk_forward",
+]

algotrader/validation/cross_permutation.py

@@ -0,0 +1,146 @@
+"""The cross-sectional null.
+
+For a timing rule, shuffling the price path is the right null. For a rule that
+*ranks names*, it is the wrong test entirely: shuffling time destroys the
+market's whole correlation structure, and the resulting null is so weak that
+almost any long-short book clears it.
+
+The question a cross-sectional strategy has to answer is narrower. Not "does
+this market have structure?" but: **given these dates, these assets and this
+book's shape, does the strategy put its weight on the right names?**
+
+So we permute the *weights across assets within each date*. Every calendar
+effect survives. Every correlation between names survives. The gross and net
+exposure of the book on each date survives exactly. The one thing destroyed is
+the link between the strategy's choice and the asset it chose.
+
+A momentum book that beats this null is picking names. One that does not was
+being paid for its market exposure, its sector tilt, or the calendar -- all of
+which are available far more cheaply.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Callable, Optional
+
+import numpy as np
+import pandas as pd
+
+from ..panel import Panel
+from ..portfolio import run_portfolio_backtest
+from ..types import CostModel
+
+__all__ = ["cross_sectional_permutation_test", "permute_within_dates", "CrossPermutationResult"]
+
+
+@dataclass
+class CrossPermutationResult:
+    observed: float
+    null: np.ndarray
+    p_value: float
+    n_permutations: int
+
+    @property
+    def null_mean(self) -> float:
+        return float(np.mean(self.null)) if self.null.size else 0.0
+
+    @property
+    def percentile(self) -> float:
+        if not self.null.size:
+            return 50.0
+        return float((self.null < self.observed).mean() * 100.0)
+
+
+def permute_within_dates(
+    weights: pd.DataFrame,
+    investable: pd.DataFrame,
+    rng: np.random.Generator,
+) -> pd.DataFrame:
+    """Reassign each date's weights among that date's investable assets.
+
+    The multiset of weights on every row is preserved exactly -- so gross
+    exposure, net exposure, leg sizes and position counts are all identical to
+    the real book -- but which asset receives which weight is randomised.
+
+    Vectorised across all dates at once: sorting each row puts the investable
+    weights first and pushes non-investable slots to NaN, then a random rank per
+    investable slot picks each weight exactly once.
+    """
+    values = weights.to_numpy(dtype=float, copy=True)
+    mask = investable.to_numpy(dtype=bool)
+
+    masked = np.where(mask, values, np.nan)
+    # NaNs sort last, so the first k entries of each row are that row's real weights.
+    ordered = np.sort(masked, axis=1)
+
+    noise = np.where(mask, rng.random(values.shape), np.inf)
+    # Double argsort turns random values into ranks 0..k-1 for investable slots,
+    # and k..n-1 for the rest, which then index into the NaN tail.
+    random_rank = np.argsort(np.argsort(noise, axis=1), axis=1)
+
+    shuffled = np.take_along_axis(ordered, random_rank, axis=1)
+    return pd.DataFrame(
+        np.nan_to_num(shuffled, nan=0.0), index=weights.index, columns=weights.columns
+    )
+
+
+def cross_sectional_permutation_test(
+    panel: Panel,
+    weights: pd.DataFrame,
+    n_permutations: int = 200,
+    costs: Optional[CostModel] = None,
+    lag: int = 1,
+    gross_leverage: float = 1.0,
+    max_weight: Optional[float] = None,
+    allow_short: bool = True,
+    rebalance_on: Optional[pd.Series] = None,
+    seed: int = 0,
+    observed: Optional[float] = None,
+    neutralise_costs: bool = True,
+    progress: Optional[Callable[[float, str], None]] = None,
+) -> CrossPermutationResult:
+    """Test whether a book's Sharpe survives randomising which names it picked.
+
+    ``neutralise_costs`` defaults to True, and it matters more than it looks.
+    A real momentum book holds many of the same names from one rebalance to the
+    next, so it churns slowly. A shuffled book reassigns names at random every
+    date, so it churns furiously and pays for it. Charging costs would penalise
+    the null for turnover the strategy never had, and the strategy would look
+    good by comparison for reasons that have nothing to do with skill.
+
+    So this test asks only "did it pick the right names?" and leaves "can you
+    afford to trade it?" to the cost stress test, which measures that directly.
+    """
+    costs = CostModel(0.0, 0.0, 0.0) if neutralise_costs else (costs or CostModel())
+    rng = np.random.default_rng(seed)
+    investable = panel.close.notna()
+
+    def sharpe_of(w: pd.DataFrame) -> float:
+        return run_portfolio_backtest(
+            panel,
+            w,
+            costs=costs,
+            lag=lag,
+            gross_leverage=gross_leverage,
+            max_weight=max_weight,
+            allow_short=allow_short,
+            rebalance_on=rebalance_on,
+        ).sharpe
+
+    if observed is None:
+        observed = sharpe_of(weights)
+
+    null = np.empty(n_permutations, dtype=float)
+    for i in range(n_permutations):
+        null[i] = sharpe_of(permute_within_dates(weights, investable, rng))
+        if progress is not None and (i % 10 == 0 or i == n_permutations - 1):
+            progress((i + 1) / n_permutations, f"Cross-sectional shuffle {i + 1}/{n_permutations}")
+
+    p_value = float((1 + np.sum(null >= observed)) / (n_permutations + 1))
+    return CrossPermutationResult(
+        observed=float(observed),
+        null=null,
+        p_value=p_value,
+        n_permutations=n_permutations,
+    )

algotrader/validation/deflated_sharpe.py

@@ -0,0 +1,145 @@
+"""Probabilistic and Deflated Sharpe Ratios.
+
+Bailey & López de Prado (2014), "The Deflated Sharpe Ratio: Correcting for
+Selection Bias, Backtest Overfitting and Non-Normality".
+
+The intuition: if you try 200 strategy variants, the best one will show a
+handsome Sharpe *even when none of them has any edge*. The Deflated Sharpe
+Ratio asks whether the winner beats what the luckiest of 200 coin-flippers
+would have produced, and it charges extra for fat tails and negative skew --
+exactly the return shapes that make naive Sharpe ratios flatter.
+"""
+
+from __future__ import annotations
+
+import numpy as np
+from scipy import stats
+
+__all__ = [
+    "probabilistic_sharpe_ratio",
+    "expected_max_sharpe",
+    "deflated_sharpe_ratio",
+    "min_track_record_length",
+]
+
+_EULER = 0.5772156649015329
+
+
+def _moments(returns: np.ndarray) -> tuple[float, float]:
+    """Sample skew and *non-excess* kurtosis, as the PSR formula expects."""
+    arr = np.asarray(returns, dtype=float)
+    arr = arr[np.isfinite(arr)]
+    if arr.size < 4:
+        return 0.0, 3.0
+    return float(stats.skew(arr, bias=False)), float(stats.kurtosis(arr, bias=False) + 3.0)
+
+
+def probabilistic_sharpe_ratio(
+    sharpe: float,
+    n_obs: int,
+    skew: float = 0.0,
+    kurtosis: float = 3.0,
+    benchmark: float = 0.0,
+) -> float:
+    """P(true Sharpe > ``benchmark``) given the observed Sharpe and its shape.
+
+    ``sharpe`` and ``benchmark`` are per-observation (i.e. *not* annualised).
+    """
+    if n_obs < 3:
+        return 0.5
+    denom = 1.0 - skew * sharpe + ((kurtosis - 1.0) / 4.0) * sharpe**2
+    if denom <= 0:
+        return 0.5
+    z = (sharpe - benchmark) * np.sqrt(n_obs - 1) / np.sqrt(denom)
+    return float(stats.norm.cdf(z))
+
+
+def expected_max_sharpe(n_trials: int, variance_of_trials: float) -> float:
+    """Expected maximum Sharpe across ``n_trials`` *skill-free* strategies.
+
+    This is the bar the winner has to clear to be interesting. It grows with
+    the number of things you tried — which is why "I found a strategy with
+    Sharpe 2" means nothing until you say how many you looked at.
+    """
+    n = max(int(n_trials), 1)
+    if n == 1 or variance_of_trials <= 0:
+        return 0.0
+    sd = np.sqrt(variance_of_trials)
+    # Bailey & López de Prado's Gumbel-based approximation.
+    q1 = stats.norm.ppf(1.0 - 1.0 / n)
+    q2 = stats.norm.ppf(1.0 - 1.0 / (n * np.e))
+    return float(sd * ((1.0 - _EULER) * q1 + _EULER * q2))
+
+
+def deflated_sharpe_ratio(
+    returns,
+    sharpe_annual: float,
+    periods_per_year: int,
+    n_trials: int,
+    trial_sharpes=None,
+    variance_of_trials: float | None = None,
+) -> dict:
+    """Deflate an annualised Sharpe for selection bias and non-normality.
+
+    Returns a dict with the PSR against a zero benchmark, the selection-bias
+    threshold, the deflated probability, and the inputs used, so the UI can
+    show its working rather than just a number.
+    """
+    arr = np.asarray(returns, dtype=float)
+    arr = arr[np.isfinite(arr)]
+    n_obs = arr.size
+    sr_per_period = sharpe_annual / np.sqrt(periods_per_year)
+    skew, kurt = _moments(arr)
+
+    if variance_of_trials is None:
+        if trial_sharpes is not None and len(trial_sharpes) > 1:
+            trials = np.asarray(trial_sharpes, dtype=float) / np.sqrt(periods_per_year)
+            trials = trials[np.isfinite(trials)]
+            variance_of_trials = float(np.var(trials, ddof=1)) if trials.size > 1 else 0.0
+        else:
+            # With no trial cloud to measure, fall back to the asymptotic
+            # variance of a skill-free Sharpe estimate.
+            variance_of_trials = 1.0 / max(n_obs - 1, 1)
+
+    threshold = expected_max_sharpe(n_trials, variance_of_trials)
+
+    psr = probabilistic_sharpe_ratio(sr_per_period, n_obs, skew, kurt, 0.0)
+    dsr = probabilistic_sharpe_ratio(sr_per_period, n_obs, skew, kurt, threshold)
+
+    return {
+        "psr": float(psr),
+        "dsr": float(dsr),
+        "sr_per_period": float(sr_per_period),
+        "threshold_sr_per_period": float(threshold),
+        "threshold_sr_annual": float(threshold * np.sqrt(periods_per_year)),
+        "n_obs": int(n_obs),
+        "n_trials": int(n_trials),
+        "skew": float(skew),
+        "kurtosis": float(kurt),
+        "variance_of_trials": float(variance_of_trials),
+    }
+
+
+def min_track_record_length(
+    sharpe: float,
+    n_obs: int,
+    skew: float = 0.0,
+    kurtosis: float = 3.0,
+    benchmark: float = 0.0,
+    confidence: float = 0.95,
+) -> float:
+    """Observations needed before the Sharpe is significant at ``confidence``.
+
+    Inputs are per-observation. Returns ``inf`` when the edge is too small to
+    ever clear the bar.
+    """
+    if sharpe <= benchmark:
+        return float("inf")
+    z = stats.norm.ppf(confidence)
+    denom = (sharpe - benchmark) ** 2
+    if denom <= 0:
+        return float("inf")
+    numer = 1.0 - skew * sharpe + ((kurtosis - 1.0) / 4.0) * sharpe**2
+    if numer <= 0:
+        return float("inf")
+    return float(1.0 + numer * (z / (sharpe - benchmark)) ** 2)

algotrader/validation/pbo.py

@@ -0,0 +1,120 @@
+"""Probability of Backtest Overfitting via Combinatorially Symmetric CV.
+
+Bailey, Borwein, López de Prado & Zhu (2016), "The Probability of Backtest
+Overfitting".
+
+Take the N parameter variants you tried, cut the timeline into S chunks, and
+for every way of splitting those chunks half-and-half: pick the variant that
+won in-sample, then look up where it ranked out-of-sample. If your selection
+process has skill, the in-sample winner should keep winning. If it is fitting
+noise, the winner lands in the bottom half about as often as not — and PBO
+approaches 0.5.
+"""
+
+from __future__ import annotations
+
+import itertools
+from typing import Dict, Sequence
+
+import numpy as np
+
+__all__ = ["probability_of_backtest_overfitting"]
+
+
+def _sharpe_columns(matrix: np.ndarray) -> np.ndarray:
+    """Per-column Sharpe (per-period, unannualised — ranks are all we need)."""
+    if matrix.shape[0] < 2:
+        return np.zeros(matrix.shape[1])
+    mean = matrix.mean(axis=0)
+    sd = matrix.std(axis=0, ddof=1)
+    # Absolute floor, not `> 0`: a flat column's std is float noise, and
+    # dividing by it would hand a do-nothing variant an enormous rank.
+    with np.errstate(divide="ignore", invalid="ignore"):
+        out = np.where(sd > 1e-12, mean / sd, 0.0)
+    return np.nan_to_num(out, nan=0.0, posinf=0.0, neginf=0.0)
+
+
+def probability_of_backtest_overfitting(
+    returns_matrix: np.ndarray,
+    n_splits: int = 8,
+    labels: Sequence[str] | None = None,
+    max_combinations: int = 200,
+) -> Dict[str, object]:
+    """Compute PBO for a ``T x N`` matrix of per-period strategy returns.
+
+    ``n_splits`` must be even. Returns PBO, the logit distribution, the
+    in-sample/out-of-sample Sharpe pairs for the selected variants, and the
+    rate at which the selected variant actually loses money out of sample.
+    """
+    matrix = np.asarray(returns_matrix, dtype=float)
+    if matrix.ndim != 2:
+        raise ValueError("returns_matrix must be 2-D (time x strategy)")
+    matrix = np.nan_to_num(matrix, nan=0.0, posinf=0.0, neginf=0.0)
+    t_obs, n_strats = matrix.shape
+
+    if n_strats < 2 or t_obs < 2 * n_splits:
+        return {
+            "pbo": float("nan"),
+            "n_strategies": int(n_strats),
+            "n_combinations": 0,
+            "logits": np.array([]),
+            "is_sharpes": np.array([]),
+            "oos_sharpes": np.array([]),
+            "prob_oos_loss": float("nan"),
+            "performance_degradation": float("nan"),
+            "note": "Not enough variants or observations to estimate PBO.",
+        }
+
+    if n_splits % 2:
+        n_splits += 1
+    chunks = np.array_split(np.arange(t_obs), n_splits)
+
+    combos = list(itertools.combinations(range(n_splits), n_splits // 2))
+    if len(combos) > max_combinations:
+        step = len(combos) / max_combinations
+        combos = [combos[int(i * step)] for i in range(max_combinations)]
+
+    logits, is_sr, oos_sr, chosen = [], [], [], []
+    for combo in combos:
+        is_idx = np.concatenate([chunks[c] for c in combo])
+        oos_idx = np.concatenate([chunks[c] for c in range(n_splits) if c not in combo])
+
+        is_perf = _sharpe_columns(matrix[is_idx])
+        oos_perf = _sharpe_columns(matrix[oos_idx])
+
+        best = int(np.argmax(is_perf))
+        chosen.append(best)
+        is_sr.append(float(is_perf[best]))
+        oos_sr.append(float(oos_perf[best]))
+
+        # Relative rank of the chosen variant in the OOS ranking, in (0, 1).
+        rank = float(np.sum(oos_perf <= oos_perf[best]))
+        omega = rank / (n_strats + 1.0)
+        omega = min(max(omega, 1e-6), 1.0 - 1e-6)
+        logits.append(float(np.log(omega / (1.0 - omega))))
+
+    logits_arr = np.asarray(logits)
+    is_arr, oos_arr = np.asarray(is_sr), np.asarray(oos_sr)
+
+    # Slope of OOS on IS: negative means better in-sample fits do *worse* live.
+    degradation = float("nan")
+    if is_arr.size > 2 and np.std(is_arr) > 1e-12:
+        degradation = float(np.polyfit(is_arr, oos_arr, 1)[0])
+
+    counts = np.bincount(chosen, minlength=n_strats)
+    most_selected = int(np.argmax(counts))
+
+    return {
+        "pbo": float(np.mean(logits_arr <= 0.0)),
+        "n_strategies": int(n_strats),
+        "n_combinations": int(len(combos)),
+        "logits": logits_arr,
+        "is_sharpes": is_arr,
+        "oos_sharpes": oos_arr,
+        "prob_oos_loss": float(np.mean(oos_arr <= 0.0)),
+        "performance_degradation": degradation,
+        "most_selected_index": most_selected,
+        "most_selected_label": (labels[most_selected] if labels is not None else str(most_selected)),
+        "selection_stability": float(counts[most_selected] / max(len(combos), 1)),
+        "note": "",
+    }

algotrader/validation/permutation.py

@@ -0,0 +1,187 @@
+"""Monte-Carlo permutation test for trading rules.
+
+The question this answers is not "did the strategy make money" but "would a
+rule of this shape have made this much money on a market with no exploitable
+structure?". We destroy the serial dependence in the price path while keeping
+its distribution of moves intact, re-run the *same* strategy on each shuffled
+market, and see where the real result lands in that null distribution.
+
+A strategy whose Sharpe sits comfortably inside the null is not a strategy —
+it is a lottery ticket that happened to win.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Callable, Optional
+
+import numpy as np
+import pandas as pd
+
+from ..engine import bars_to_returns, run_backtest
+from ..types import CostModel
+
+__all__ = ["permutation_test", "PermutationResult", "permute_bars"]
+
+
+@dataclass
+class PermutationResult:
+    observed: float
+    null: np.ndarray
+    p_value: float
+    method: str
+    n_permutations: int
+
+    @property
+    def null_mean(self) -> float:
+        return float(np.mean(self.null)) if self.null.size else 0.0
+
+    @property
+    def percentile(self) -> float:
+        """Where the observed Sharpe sits in the null distribution, 0-100."""
+        if not self.null.size:
+            return 50.0
+        return float((self.null < self.observed).mean() * 100.0)
+
+
+def _decompose(df: pd.DataFrame) -> tuple[np.ndarray, float]:
+    """Split bars into scale-free log moves that can be reshuffled safely."""
+    open_ = df["open"].to_numpy(dtype=float)
+    high = df["high"].to_numpy(dtype=float)
+    low = df["low"].to_numpy(dtype=float)
+    close = df["close"].to_numpy(dtype=float)
+    volume = df["volume"].to_numpy(dtype=float)
+
+    gap = np.log(open_[1:] / close[:-1])
+    hi = np.log(np.maximum(high[1:], open_[1:]) / open_[1:])
+    lo = np.log(np.minimum(low[1:], open_[1:]) / open_[1:])
+    body = np.log(close[1:] / open_[1:])
+    return np.column_stack([gap, hi, lo, body, volume[1:]]), float(close[0])
+
+
+def _rebuild(parts: np.ndarray, anchor: float, index: pd.Index, first_row: pd.Series) -> pd.DataFrame:
+    gap, hi, lo, body, volume = (parts[:, i] for i in range(5))
+    n = parts.shape[0] + 1
+
+    close = np.empty(n)
+    open_ = np.empty(n)
+    high = np.empty(n)
+    low = np.empty(n)
+    vol = np.empty(n)
+
+    close[0] = anchor
+    open_[0] = float(first_row["open"])
+    high[0] = float(first_row["high"])
+    low[0] = float(first_row["low"])
+    vol[0] = float(first_row["volume"])
+
+    # Cumulative product form: close[i] = close[0] * exp(cumsum(gap + body)).
+    close[1:] = anchor * np.exp(np.cumsum(gap + body))
+    open_[1:] = close[:-1] * np.exp(gap)
+    high[1:] = open_[1:] * np.exp(hi)
+    low[1:] = open_[1:] * np.exp(lo)
+    vol[1:] = volume
+
+    return pd.DataFrame(
+        {"open": open_, "high": high, "low": low, "close": close, "volume": vol}, index=index
+    )
+
+
+def permute_bars(
+    df: pd.DataFrame,
+    rng: np.random.Generator,
+    method: str = "permute",
+    block: int = 20,
+) -> pd.DataFrame:
+    """Return a shuffled market with the same index and bar anatomy.
+
+    ``permute`` reshuffles individual bars, destroying all serial structure.
+    ``block`` resamples contiguous blocks with replacement, which preserves
+    short-horizon autocorrelation and volatility clustering — a harder null
+    that trend strategies deserve to be tested against.
+    """
+    parts, anchor = _decompose(df)
+    m = parts.shape[0]
+    if m < 2:
+        return df.copy()
+
+    if method == "block":
+        size = max(2, min(int(block), m))
+        starts = rng.integers(0, m, size=int(np.ceil(m / size)))
+        order = np.concatenate([(np.arange(s, s + size) % m) for s in starts])[:m]
+    else:
+        order = rng.permutation(m)
+
+    return _rebuild(parts[order], anchor, df.index, df.iloc[0])
+
+
+def permutation_test(
+    df: pd.DataFrame,
+    signal_fn: Callable[[pd.DataFrame], pd.Series],
+    n_permutations: int = 300,
+    method: str = "permute",
+    block: int = 20,
+    costs: Optional[CostModel] = None,
+    lag: int = 1,
+    max_leverage: float = 1.0,
+    allow_short: bool = True,
+    seed: int = 0,
+    observed: Optional[float] = None,
+    progress: Optional[Callable[[float, str], None]] = None,
+) -> PermutationResult:
+    """Run ``signal_fn`` against ``n_permutations`` shuffled markets.
+
+    ``signal_fn`` must be the strategy's target-exposure generator; it is
+    re-evaluated on every synthetic market, which is the whole point — a rule
+    that only works because of the specific path it was tuned on will fall
+    apart here.
+    """
+    costs = costs or CostModel()
+    rng = np.random.default_rng(seed)
+
+    def sharpe_on(frame: pd.DataFrame) -> float:
+        target = signal_fn(frame)
+        result = run_backtest(
+            frame,
+            target,
+            costs=costs,
+            lag=lag,
+            max_leverage=max_leverage,
+            allow_short=allow_short,
+        )
+        return result.sharpe
+
+    if observed is None:
+        observed = sharpe_on(df)
+
+    null = np.empty(n_permutations, dtype=float)
+    for i in range(n_permutations):
+        null[i] = sharpe_on(permute_bars(df, rng, method, block))
+        if progress is not None and (i % 25 == 0 or i == n_permutations - 1):
+            progress((i + 1) / n_permutations, f"Permutation {i + 1}/{n_permutations}")
+
+    # +1 in both places: the observed result is itself one draw from the null
+    # under H0, which keeps the test from ever reporting an impossible p = 0.
+    p_value = float((1 + np.sum(null >= observed)) / (n_permutations + 1))
+
+    return PermutationResult(
+        observed=float(observed),
+        null=null,
+        p_value=p_value,
+        method=method,
+        n_permutations=n_permutations,
+    )
+
+
+def bootstrap_return_paths(returns: pd.Series, n: int = 500, seed: int = 0) -> np.ndarray:
+    """Bootstrap terminal-wealth outcomes from a realised return stream.
+
+    Useful for the "how wide is the cone of outcomes?" chart — the same edge
+    can produce wildly different equity curves.
+    """
+    arr = np.asarray(returns.dropna(), dtype=float)
+    if arr.size == 0:
+        return np.zeros((n, 1))
+    rng = np.random.default_rng(seed)
+    draws = rng.choice(arr, size=(n, arr.size), replace=True)
+    return np.cumprod(1.0 + draws, axis=1)

algotrader/validation/walkforward.py

@@ -0,0 +1,233 @@
+"""Walk-forward analysis.
+
+Re-tune on a training window, trade the next window blind, roll forward. The
+gap between in-sample and out-of-sample Sharpe is the honest estimate of how
+much of the backtest was curve-fitting.
+"""
+
+from __future__ import annotations
+
+from typing import Callable, Dict, List, Optional
+
+import numpy as np
+import pandas as pd
+
+from ..engine import run_backtest
+from ..strategies import Strategy
+from ..types import CostModel
+
+__all__ = ["walk_forward", "walk_forward_panel"]
+
+
+def walk_forward(
+    df: pd.DataFrame,
+    strategy: Strategy,
+    n_folds: int = 5,
+    train_ratio: float = 0.7,
+    costs: Optional[CostModel] = None,
+    lag: int = 1,
+    max_leverage: float = 1.0,
+    allow_short: bool = True,
+    grid_limit: int = 40,
+    progress: Optional[Callable[[float, str], None]] = None,
+) -> Dict[str, object]:
+    """Roll a train/test split forward ``n_folds`` times.
+
+    Each fold picks the best parameters by in-sample Sharpe and reports what
+    those parameters then did out of sample. The stitched OOS returns are the
+    closest thing to a paper-trading record this repo can produce offline.
+    """
+    costs = costs or CostModel()
+    grid = strategy.grid(limit=grid_limit)
+    n = len(df)
+
+    if n < 250 or n_folds < 2:
+        return {"folds": [], "note": "Not enough history for walk-forward analysis."}
+
+    # Each fold is a contiguous train+test block; blocks advance by test length.
+    block = int(n / (1 + (n_folds - 1) * (1 - train_ratio)))
+    block = min(block, n)
+    train_len = int(block * train_ratio)
+    test_len = block - train_len
+    if train_len < 100 or test_len < 20:
+        return {"folds": [], "note": "Not enough history for walk-forward analysis."}
+
+    folds: List[Dict[str, object]] = []
+    oos_returns: List[pd.Series] = []
+
+    for k in range(n_folds):
+        start = k * test_len
+        train = df.iloc[start : start + train_len]
+        test = df.iloc[start + train_len : start + train_len + test_len]
+        if len(test) < 20:
+            break
+
+        best_params, best_sharpe = None, -np.inf
+        for params in grid:
+            target = strategy.generate(train, params)
+            sr = run_backtest(
+                train, target, costs=costs, lag=lag,
+                max_leverage=max_leverage, allow_short=allow_short,
+            ).sharpe
+            if sr > best_sharpe:
+                best_params, best_sharpe = params, sr
+
+        # Generate signals on train+test so indicators are warm at the fold
+        # boundary, then evaluate only the test slice.
+        combined = df.iloc[start : start + train_len + len(test)]
+        target = strategy.generate(combined, best_params).loc[test.index]
+        oos = run_backtest(
+            test, target, costs=costs, lag=lag,
+            max_leverage=max_leverage, allow_short=allow_short,
+        )
+
+        folds.append(
+            {
+                "fold": k + 1,
+                "train_start": str(train.index[0].date()),
+                "train_end": str(train.index[-1].date()),
+                "test_start": str(test.index[0].date()),
+                "test_end": str(test.index[-1].date()),
+                "params": best_params,
+                "is_sharpe": float(best_sharpe),
+                "oos_sharpe": float(oos.sharpe),
+                "oos_return": float(oos.metrics.get("total_return", 0.0)),
+                "oos_max_dd": float(oos.metrics.get("max_drawdown", 0.0)),
+            }
+        )
+        oos_returns.append(oos.returns)
+
+        if progress is not None:
+            progress((k + 1) / n_folds, f"Walk-forward fold {k + 1}/{n_folds}")
+
+    if not folds:
+        return {"folds": [], "note": "Not enough history for walk-forward analysis."}
+
+    is_sharpes = np.array([f["is_sharpe"] for f in folds], dtype=float)
+    oos_sharpes = np.array([f["oos_sharpe"] for f in folds], dtype=float)
+    stitched = pd.concat(oos_returns) if oos_returns else pd.Series(dtype=float)
+    stitched = stitched[~stitched.index.duplicated(keep="first")].sort_index()
+
+    mean_is = float(np.mean(is_sharpes))
+    mean_oos = float(np.mean(oos_sharpes))
+
+    return {
+        "folds": folds,
+        "mean_is_sharpe": mean_is,
+        "mean_oos_sharpe": mean_oos,
+        # 1.0 = the edge fully survived; 0.0 = it evaporated out of sample.
+        "efficiency": float(mean_oos / mean_is) if mean_is > 1e-9 else 0.0,
+        "oos_win_rate": float(np.mean(oos_sharpes > 0)),
+        # 1.0 means every fold chose different parameters -- a tuning process
+        # that cannot make up its mind is fitting noise.
+        "param_instability": float(
+            len({str(f["params"]) for f in folds}) / max(len(folds), 1)
+        ),
+        "oos_returns": stitched,
+        "oos_equity": (1.0 + stitched).cumprod() if len(stitched) else stitched,
+        "note": "",
+    }
+
+
+def walk_forward_panel(
+    panel,
+    strategy,
+    n_folds: int = 4,
+    train_ratio: float = 0.7,
+    costs: Optional[CostModel] = None,
+    lag: int = 1,
+    gross_leverage: float = 1.0,
+    allow_short: bool = True,
+    rebalance: str = "M",
+    grid_limit: int = 16,
+    progress: Optional[Callable[[float, str], None]] = None,
+) -> Dict[str, object]:
+    """Walk-forward for cross-sectional strategies over a :class:`Panel`.
+
+    Same contract as :func:`walk_forward`: tune on the training window, trade
+    the next window blind, roll on. Signals are generated over train+test
+    together so the indicators are warm at the fold boundary, then evaluated
+    only on the test slice -- the panel equivalent of the single-asset path.
+    """
+    from ..portfolio import rebalance_schedule, run_portfolio_backtest
+
+    costs = costs or CostModel()
+    grid = strategy.grid(limit=grid_limit)
+    n = len(panel)
+
+    if n < 250 or n_folds < 2:
+        return {"folds": [], "note": "Not enough history for walk-forward analysis."}
+
+    block = min(int(n / (1 + (n_folds - 1) * (1 - train_ratio))), n)
+    train_len = int(block * train_ratio)
+    test_len = block - train_len
+    if train_len < 100 or test_len < 20:
+        return {"folds": [], "note": "Not enough history for walk-forward analysis."}
+
+    folds: List[Dict[str, object]] = []
+    oos_returns: List[pd.Series] = []
+
+    for k in range(n_folds):
+        start = k * test_len
+        train = panel.slice(panel.index[start], panel.index[min(start + train_len - 1, n - 1)])
+        test_start = start + train_len
+        if test_start + 20 > n:
+            break
+        test_end = min(test_start + test_len, n) - 1
+        test = panel.slice(panel.index[test_start], panel.index[test_end])
+        if len(test) < 20:
+            break
+
+        best_params, best_sharpe = None, -np.inf
+        for params in grid:
+            weights = strategy.generate(train, params)
+            sharpe = run_portfolio_backtest(
+                train, weights, costs=costs, lag=lag, gross_leverage=gross_leverage,
+                allow_short=allow_short, rebalance_on=rebalance_schedule(train.index, rebalance),
+            ).sharpe
+            if sharpe > best_sharpe:
+                best_params, best_sharpe = params, sharpe
+
+        combined = panel.slice(panel.index[start], panel.index[test_end])
+        weights = strategy.generate(combined, best_params).loc[test.index]
+        oos = run_portfolio_backtest(
+            test, weights, costs=costs, lag=lag, gross_leverage=gross_leverage,
+            allow_short=allow_short, rebalance_on=rebalance_schedule(test.index, rebalance),
+        )
+
+        folds.append({
+            "fold": k + 1,
+            "train_start": str(train.index[0].date()),
+            "train_end": str(train.index[-1].date()),
+            "test_start": str(test.index[0].date()),
+            "test_end": str(test.index[-1].date()),
+            "params": best_params,
+            "is_sharpe": float(best_sharpe),
+            "oos_sharpe": float(oos.sharpe),
+            "oos_return": float(oos.metrics.get("total_return", 0.0)),
+            "oos_max_dd": float(oos.metrics.get("max_drawdown", 0.0)),
+        })
+        oos_returns.append(oos.returns)
+        if progress is not None:
+            progress((k + 1) / n_folds, f"Walk-forward fold {k + 1}/{n_folds}")
+
+    if not folds:
+        return {"folds": [], "note": "Not enough history for walk-forward analysis."}
+
+    is_sharpes = np.array([f["is_sharpe"] for f in folds], dtype=float)
+    oos_sharpes = np.array([f["oos_sharpe"] for f in folds], dtype=float)
+    stitched = pd.concat(oos_returns) if oos_returns else pd.Series(dtype=float)
+    stitched = stitched[~stitched.index.duplicated(keep="first")].sort_index()
+    mean_is, mean_oos = float(np.mean(is_sharpes)), float(np.mean(oos_sharpes))
+
+    return {
+        "folds": folds,
+        "mean_is_sharpe": mean_is,
+        "mean_oos_sharpe": mean_oos,
+        "efficiency": float(mean_oos / mean_is) if mean_is > 1e-9 else 0.0,
+        "oos_win_rate": float(np.mean(oos_sharpes > 0)),
+        "param_instability": float(len({str(f["params"]) for f in folds}) / max(len(folds), 1)),
+        "oos_returns": stitched,
+        "oos_equity": (1.0 + stitched).cumprod() if len(stitched) else stitched,
+        "note": "",
+    }

algotrader/verdict.py

@@ -0,0 +1,211 @@
+"""Turn a pile of statistics into one number and one sentence.
+
+The Reality Score is deliberately harsh. Most published backtests would score
+below 40, and that is the point: the score exists to be screenshotted.
+"""
+
+from __future__ import annotations
+
+from typing import Dict, List, Optional
+
+import numpy as np
+
+__all__ = ["reality_score", "GRADES"]
+
+GRADES = [
+    (85, "A", "Survives everything we threw at it"),
+    (70, "B", "Probably a real edge, with caveats"),
+    (55, "C", "Ambiguous — could go either way"),
+    (40, "D", "Mostly luck"),
+    (0, "F", "Indistinguishable from randomness"),
+]
+
+WEIGHTS = {
+    "significance": 0.30,
+    "selection": 0.25,
+    "walk_forward": 0.20,
+    "overfitting": 0.15,
+    "robustness": 0.10,
+}
+
+
+def _ramp(value: float, good: float, bad: float) -> float:
+    """Linear 0-100 score where ``good`` maps to 100 and ``bad`` maps to 0."""
+    if not np.isfinite(value):
+        return 50.0
+    if good == bad:
+        return 50.0
+    scaled = (value - bad) / (good - bad)
+    return float(np.clip(scaled, 0.0, 1.0) * 100.0)
+
+
+def reality_score(
+    metrics: Dict[str, float],
+    benchmark_metrics: Dict[str, float],
+    p_value: Optional[float] = None,
+    dsr: Optional[float] = None,
+    pbo: Optional[float] = None,
+    wf_efficiency: Optional[float] = None,
+    wf_win_rate: Optional[float] = None,
+    cost_stress_ratio: Optional[float] = None,
+    benchmark_correlation: Optional[float] = None,
+    attribution: Optional[Dict[str, object]] = None,
+    survivorship: Optional[object] = None,
+    benchmark_name: str = "Buy & hold",
+    permutation_label: str = "shuffled, structure-free markets",
+) -> Dict[str, object]:
+    """Combine the validation panel into a 0-100 score, a grade and warnings.
+
+    ``attribution`` and ``survivorship`` are the portfolio-only inputs; both are
+    optional so the single-asset path is unaffected.
+    """
+    components: Dict[str, float] = {}
+
+    components["significance"] = _ramp(p_value, good=0.01, bad=0.50) if p_value is not None else 50.0
+    components["selection"] = float(np.clip(dsr, 0.0, 1.0) * 100.0) if dsr is not None else 50.0
+
+    if wf_efficiency is not None:
+        wf = _ramp(wf_efficiency, good=0.8, bad=-0.2)
+        if wf_win_rate is not None:
+            wf = 0.7 * wf + 0.3 * float(np.clip(wf_win_rate, 0.0, 1.0) * 100.0)
+        components["walk_forward"] = wf
+    else:
+        components["walk_forward"] = 50.0
+
+    components["overfitting"] = _ramp(pbo, good=0.05, bad=0.50) if pbo is not None and np.isfinite(pbo) else 50.0
+    components["robustness"] = (
+        _ramp(cost_stress_ratio, good=0.8, bad=0.0) if cost_stress_ratio is not None else 50.0
+    )
+
+    score = float(sum(components[k] * w for k, w in WEIGHTS.items()))
+
+    flags: List[str] = []
+    n_trades = int(metrics.get("n_trades", 0))
+    sharpe = float(metrics.get("sharpe", 0.0))
+    total_return = float(metrics.get("total_return", 0.0))
+    max_dd = float(metrics.get("max_drawdown", 0.0))
+    bench_sharpe = float(benchmark_metrics.get("sharpe", 0.0))
+
+    # The score asks "is the measured edge real?" — if the strategy lost money,
+    # there is no edge to validate, whatever the statistics say about it.
+    if total_return <= 0:
+        flags.append(
+            f"The strategy lost money over the test period ({total_return:.1%}). "
+            "There is no edge here to validate."
+        )
+        score = min(score, 50.0)
+    if total_return <= 0 < sharpe:
+        flags.append(
+            "Positive Sharpe with a negative total return: the average bar was profitable but "
+            "the compounding was not. Volatility drag ate the arithmetic edge."
+        )
+    if max_dd < -0.5:
+        flags.append(
+            f"Peak-to-trough drawdown of {max_dd:.0%} — an account running this would have been "
+            "closed long before the recovery arrived."
+        )
+        score = min(score, 60.0)
+
+    if n_trades < 20:
+        flags.append(
+            f"Only {n_trades} position changes — with this few decisions, "
+            "the result is a handful of coin flips, not a track record."
+        )
+        score = min(score, 55.0)
+    if p_value is not None and p_value > 0.10:
+        flags.append(
+            f"Permutation p-value is {p_value:.2f}: roughly {p_value * 100:.0f}% of "
+            f"{permutation_label} did this well or better."
+        )
+    if dsr is not None and dsr < 0.5:
+        flags.append(
+            f"Deflated Sharpe is {dsr:.2f} — once you account for how many variants were tried, "
+            "the edge does not clear the selection-bias bar."
+        )
+    if pbo is not None and np.isfinite(pbo) and pbo > 0.3:
+        flags.append(
+            f"Probability of backtest overfitting is {pbo:.0%}: the in-sample winner "
+            "usually lands in the bottom half out of sample."
+        )
+    if wf_efficiency is not None and wf_efficiency < 0.3:
+        flags.append(
+            f"Walk-forward efficiency is {wf_efficiency:.0%} — most of the in-sample Sharpe "
+            "does not survive re-tuning and trading forward."
+        )
+    if cost_stress_ratio is not None and cost_stress_ratio < 0.5:
+        flags.append(
+            "Tripling trading costs removes more than half the Sharpe. The edge is "
+            "smaller than the friction it has to pay."
+        )
+    if benchmark_correlation is not None and benchmark_correlation > 0.95:
+        flags.append(
+            f"Returns are {benchmark_correlation:.0%} correlated with {benchmark_name.lower()} — "
+            "this is mostly a repackaged long position."
+        )
+    if sharpe < bench_sharpe:
+        flags.append(
+            f"{benchmark_name} beat it on risk-adjusted return "
+            f"({bench_sharpe:.2f} vs {sharpe:.2f} Sharpe)."
+        )
+    if float(metrics.get("turnover_ann", 0.0)) > 100:
+        flags.append(
+            f"Annual turnover of {metrics.get('turnover_ann', 0):.0f}x is far beyond what "
+            "retail execution can absorb without moving the modelled fills."
+        )
+
+    # Portfolio-only checks. Style exposure you could buy in an ETF is not
+    # alpha, and a universe with no failures in it is not a universe.
+    if attribution and attribution.get("available"):
+        if not attribution.get("alpha_significant"):
+            flags.append(
+                f"Style regression leaves no significant alpha (t = "
+                f"{attribution.get('alpha_t_stat', 0):.1f}); the factors explain "
+                f"{attribution.get('r_squared', 0):.0%} of returns"
+                + (
+                    f", mostly {attribution['dominant_factor']} exposure."
+                    if attribution.get("dominant_factor")
+                    else "."
+                )
+            )
+            score = min(score, 65.0)
+        if float(attribution.get("r_squared", 0.0)) > 0.9:
+            flags.append(
+                "Over 90% of the return variation is explained by simple style factors — "
+                "this book is a repackaged index."
+            )
+    if survivorship is not None and getattr(survivorship, "biased", False):
+        flags.append(getattr(survivorship, "note", "Universe appears survivorship-biased."))
+        score = min(score, 60.0)
+
+    score = float(np.clip(score, 0.0, 100.0))
+    grade, headline = next((g, h) for threshold, g, h in GRADES if score >= threshold)
+
+    if score >= 70:
+        verdict = (
+            f"Grade {grade}. {headline}. The edge is still there after the permutation test, "
+            "after charging for every variant tried, and after walking it forward."
+        )
+    elif score >= 55:
+        verdict = (
+            f"Grade {grade}. {headline}. Parts of the panel hold up and parts do not — "
+            "this is the zone where more data, not more tuning, is what settles it."
+        )
+    elif score >= 40:
+        verdict = (
+            f"Grade {grade}. {headline}. The backtest looks better than the evidence supports; "
+            "the gap between the two is selection bias."
+        )
+    else:
+        verdict = (
+            f"Grade {grade}. {headline}. The null — {permutation_label} — produces results "
+            "like this often enough that there is nothing here to trade."
+        )
+
+    return {
+        "score": round(score, 1),
+        "grade": grade,
+        "headline": headline,
+        "verdict": verdict,
+        "components": {k: round(v, 1) for k, v in components.items()},
+        "flags": flags,
+    }

app.py

@@ -0,0 +1,849 @@
+"""Backtest Reality Check — the Hugging Face Space entrypoint.
+
+Point it at a ticker and a trading rule. It runs the backtest, then spends the
+rest of its time trying to prove the result was luck: shuffled markets,
+selection-bias deflation, walk-forward, and a cost stress test.
+
+Run locally with ``python app.py``.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+from typing import Dict, List
+
+import gradio as gr
+import pandas as pd
+
+from algotrader import __version__
+from algotrader.charts import (
+    arena_chart,
+    attribution_chart,
+    cross_permutation_chart,
+    drawdown_chart,
+    empty_figure,
+    equity_chart,
+    exposure_chart,
+    permutation_chart,
+    score_chart,
+    walkforward_chart,
+    weights_chart,
+)
+from algotrader.cross_sectional import XS_REGISTRY, get_xs_strategy
+from algotrader.data import DEFAULT_UNIVERSE
+from algotrader.lab import LabConfig, run_arena, run_lab
+from algotrader.portfolio_lab import DEFAULT_UNIVERSE as PORTFOLIO_UNIVERSE
+from algotrader.portfolio_lab import PortfolioLabConfig, run_portfolio_lab
+from algotrader.strategies import REGISTRY, get_strategy
+
+logging.basicConfig(level=logging.INFO, format="%(levelname)s %(name)s: %(message)s")
+logger = logging.getLogger("app")
+
+MAX_PARAMS = 3
+MAX_XS_PARAMS = 4
+STRATEGY_CHOICES = [(s.name, key) for key, s in REGISTRY.items()]
+XS_STRATEGY_CHOICES = [(s.name, key) for key, s in XS_REGISTRY.items()]
+
+GRADE_COLORS = {
+    "A": "#0ca30c",
+    "B": "#3987e5",
+    "C": "#fab219",
+    "D": "#ec835a",
+    "F": "#d03b3b",
+}
+
+CSS = """
+.gradio-container { max-width: 1280px !important; }
+#hero h1 { font-size: 2.4rem; line-height: 1.1; margin: 0 0 .4rem 0; letter-spacing: -.02em; }
+#hero p { color: #c3c2b7; margin: 0; font-size: 1.05rem; max-width: 60ch; }
+.score-card {
+  display: flex; gap: 24px; align-items: center; padding: 22px 24px;
+  border: 1px solid rgba(255,255,255,.10); border-radius: 14px; background: #1a1a19;
+}
+.score-badge {
+  min-width: 132px; text-align: center; padding: 14px 10px; border-radius: 12px;
+  background: #0d0d0d; border: 1px solid rgba(255,255,255,.10);
+}
+.score-badge .grade { font-size: 3.2rem; font-weight: 700; line-height: 1; }
+.score-badge .num { font-size: .95rem; color: #898781; margin-top: 6px; }
+.score-body h3 { margin: 0 0 6px 0; font-size: 1.15rem; color: #fff; }
+.score-body p { margin: 0; color: #c3c2b7; line-height: 1.55; }
+.tiles { display: grid; grid-template-columns: repeat(auto-fit, minmax(132px,1fr)); gap: 10px; margin-top: 14px; }
+.tile { padding: 12px 14px; border: 1px solid rgba(255,255,255,.10); border-radius: 10px; background: #1a1a19; }
+.tile .label { font-size: .72rem; text-transform: uppercase; letter-spacing: .06em; color: #898781; }
+.tile .value { font-size: 1.5rem; font-weight: 600; color: #fff; margin-top: 4px; }
+.tile .sub { font-size: .75rem; color: #898781; margin-top: 2px; }
+.flags { margin-top: 14px; padding: 0; list-style: none; }
+.flags li {
+  padding: 9px 12px; margin-bottom: 7px; border-radius: 8px; background: #1a1a19;
+  border-left: 3px solid #ec835a; color: #c3c2b7; font-size: .9rem; line-height: 1.5;
+}
+.provenance { font-size: .82rem; color: #898781; margin-top: 10px; }
+.provenance.sim { color: #fab219; }
+"""
+
+
+def _fmt_pct(value: float) -> str:
+    return f"{value * 100:,.1f}%"
+
+
+def param_controls(strategy_key: str):
+    """Re-label the shared sliders to match the selected strategy."""
+    strategy = get_strategy(strategy_key)
+    updates = []
+    for i in range(MAX_PARAMS):
+        if i < len(strategy.params):
+            spec = strategy.params[i]
+            lo = spec.minimum if spec.minimum is not None else min(spec.grid)
+            hi = spec.maximum if spec.maximum is not None else max(spec.grid)
+            updates.append(
+                gr.update(
+                    visible=True,
+                    label=spec.label,
+                    value=spec.cast(spec.default),
+                    minimum=lo,
+                    maximum=hi,
+                    step=spec.step or (1 if spec.kind == "int" else 0.1),
+                )
+            )
+        else:
+            updates.append(gr.update(visible=False))
+    return tuple(updates)
+
+
+def collect_params(strategy_key: str, *values) -> Dict[str, float]:
+    strategy = get_strategy(strategy_key)
+    return {spec.name: spec.cast(values[i]) for i, spec in enumerate(strategy.params[:MAX_PARAMS])}
+
+
+def _score_card(report) -> str:
+    v = report.verdict
+    color = GRADE_COLORS.get(v["grade"], "#898781")
+    market = report.market
+    provenance = (
+        f'<div class="provenance">Data: {market.source} · {market.symbol} · '
+        f"{market.start.date()} to {market.end.date()} · {len(market.df):,} bars</div>"
+        if market.is_real
+        else f'<div class="provenance sim">⚠ {market.note}</div>'
+    )
+    flags = "".join(f"<li>{f}</li>" for f in v["flags"])
+    flags_html = f'<ul class="flags">{flags}</ul>' if flags else ""
+
+    return f"""
+<div class="score-card">
+  <div class="score-badge">
+    <div class="grade" style="color:{color}">{v['grade']}</div>
+    <div class="num">{v['score']} / 100</div>
+  </div>
+  <div class="score-body">
+    <h3>{report.strategy.name} on {market.symbol}</h3>
+    <p>{v['verdict']}</p>
+  </div>
+</div>
+{flags_html}
+{provenance}
+"""
+
+
+def _tiles(report) -> str:
+    m = report.backtest.metrics
+    b = report.backtest.benchmark_metrics
+    perm = report.permutation
+    p_text = f"{perm.p_value:.3f}" if perm else "—"
+    p_sub = "vs shuffled markets" if perm else "test skipped"
+    pbo = report.pbo.get("pbo")
+    pbo_text = f"{pbo:.0%}" if pbo is not None and pbo == pbo else "n/a"
+
+    cells = [
+        ("Total return", _fmt_pct(m.get("total_return", 0)), f"buy & hold {_fmt_pct(b.get('total_return', 0))}"),
+        ("CAGR", _fmt_pct(m.get("cagr", 0)), f"over {m.get('years', 0):.1f} years"),
+        ("Sharpe", f"{m.get('sharpe', 0):.2f}", f"buy & hold {b.get('sharpe', 0):.2f}"),
+        ("Max drawdown", _fmt_pct(m.get("max_drawdown", 0)), f"{m.get('time_under_water_yrs', 0):.1f}y under water"),
+        ("p-value", p_text, p_sub),
+        ("Deflated Sharpe", f"{report.dsr.get('dsr', 0):.2f}", f"after {report.trials.get('n', 1)} variants"),
+        ("Overfit prob.", pbo_text, "in-sample winner fails OOS"),
+        ("Trades", f"{int(m.get('n_trades', 0)):,}", f"{m.get('turnover_ann', 0):.0f}x turnover/yr"),
+    ]
+    tiles = "".join(
+        f'<div class="tile"><div class="label">{label}</div>'
+        f'<div class="value">{value}</div><div class="sub">{sub}</div></div>'
+        for label, value, sub in cells
+    )
+    return f'<div class="tiles">{tiles}</div>'
+
+
+def xs_param_controls(strategy_key: str):
+    """Re-label the shared portfolio sliders for the selected strategy."""
+    strategy = get_xs_strategy(strategy_key)
+    updates = []
+    for i in range(MAX_XS_PARAMS):
+        if i < len(strategy.params):
+            spec = strategy.params[i]
+            updates.append(
+                gr.update(
+                    visible=True,
+                    label=spec.label,
+                    value=spec.cast(spec.default),
+                    minimum=spec.minimum if spec.minimum is not None else min(spec.grid),
+                    maximum=spec.maximum if spec.maximum is not None else max(spec.grid),
+                    step=spec.step or (1 if spec.kind == "int" else 0.05),
+                )
+            )
+        else:
+            updates.append(gr.update(visible=False))
+    return tuple(updates)
+
+
+def _portfolio_card(report) -> str:
+    v = report.verdict
+    color = GRADE_COLORS.get(v["grade"], "#898781")
+    panel = report.panel
+    survivorship = report.survivorship
+
+    provenance = (
+        f'<div class="provenance">{len(panel.symbols)} symbols · '
+        f"{panel.index[0].date()} to {panel.index[-1].date()} · {len(panel):,} bars · "
+        f"rebalance {report.config.rebalance}</div>"
+    )
+    if panel.note:
+        provenance += f'<div class="provenance sim">⚠ {panel.note}</div>'
+    # The survivorship note is already in the flag list when it is a problem;
+    # repeating it under the tiles just makes the card noisy.
+
+    flags = "".join(f"<li>{f}</li>" for f in v["flags"])
+    flags_html = f'<ul class="flags">{flags}</ul>' if flags else ""
+
+    return f"""
+<div class="score-card">
+  <div class="score-badge">
+    <div class="grade" style="color:{color}">{v['grade']}</div>
+    <div class="num">{v['score']} / 100</div>
+  </div>
+  <div class="score-body">
+    <h3>{report.strategy.name} across {len(panel.symbols)} names</h3>
+    <p>{v['verdict']}</p>
+  </div>
+</div>
+{flags_html}
+{provenance}
+"""
+
+
+def _portfolio_tiles(report) -> str:
+    m = report.backtest.metrics
+    b = report.backtest.benchmark_metrics
+    perm = report.permutation
+    attribution = report.attribution
+    pbo = report.pbo.get("pbo")
+
+    cells = [
+        ("Total return", _fmt_pct(m.get("total_return", 0)), f"equal weight {_fmt_pct(b.get('total_return', 0))}"),
+        ("Sharpe", f"{m.get('sharpe', 0):.2f}", f"equal weight {b.get('sharpe', 0):.2f}"),
+        ("Max drawdown", _fmt_pct(m.get("max_drawdown", 0)), f"{m.get('time_under_water_yrs', 0):.1f}y under water"),
+        ("Name-shuffle p", f"{perm.p_value:.3f}" if perm else "—", "vs same book, random names"),
+        ("Deflated Sharpe", f"{report.dsr.get('dsr', 0):.2f}", f"after {report.trials.get('n', 1)} variants"),
+        (
+            "Style alpha",
+            f"{attribution.get('alpha_annual', 0) * 100:,.1f}%" if attribution.get("available") else "n/a",
+            f"t = {attribution.get('alpha_t_stat', 0):.1f}" if attribution.get("available") else "not available",
+        ),
+        ("Overfit prob.", f"{pbo:.0%}" if pbo is not None and pbo == pbo else "n/a", "in-sample winner fails OOS"),
+        ("Gross / net", f"{m.get('gross_exposure', 0):.2f}", f"net {m.get('net_exposure', 0):+.2f}"),
+        ("Avg positions", f"{m.get('avg_positions', 0):.1f}", f"{m.get('turnover_ann', 0):.1f}x turnover/yr"),
+        (
+            "Survivorship",
+            f"{report.survivorship.survival_rate:.0%}",
+            f"{report.survivorship.n_delisted} of {report.survivorship.n_symbols} delisted",
+        ),
+    ]
+    tiles = "".join(
+        f'<div class="tile"><div class="label">{label}</div>'
+        f'<div class="value">{value}</div><div class="sub">{sub}</div></div>'
+        for label, value, sub in cells
+    )
+    return f'<div class="tiles">{tiles}</div>'
+
+
+def _portfolio_detail(report) -> str:
+    attribution, survivorship, wf = report.attribution, report.survivorship, report.walkforward
+    lines = ["### Reading the evidence", ""]
+
+    if report.permutation:
+        lines += [
+            f"**Did it pick the right names?** We rebuilt the book "
+            f"{report.permutation.n_permutations} times, keeping every date's gross exposure, net "
+            f"exposure and position count exactly as they were, and only randomising *which* asset "
+            f"got which weight. The real book's Sharpe of {report.permutation.observed:.2f} sits at "
+            f"the {report.permutation.percentile:.0f}th percentile of that null "
+            f"(p = {report.permutation.p_value:.3f}). This test deliberately ignores trading costs: "
+            "a shuffled book churns far more than a real one, and charging it for that would "
+            "flatter the strategy for reasons unrelated to skill.",
+            "",
+        ]
+    if attribution.get("available"):
+        lines += [f"**Is it alpha or is it beta?** {attribution['note']}", ""]
+    lines += [f"**Survivorship.** {survivorship.note}", ""]
+    if survivorship.delisted_symbols:
+        lines += [f"Stopped trading during the sample: `{'`, `'.join(survivorship.delisted_symbols)}`.", ""]
+    if wf.get("folds"):
+        lines += [
+            f"**Walk-forward.** Across {len(wf['folds'])} folds the tuned in-sample Sharpe averaged "
+            f"{wf.get('mean_is_sharpe', 0):.2f} against {wf.get('mean_oos_sharpe', 0):.2f} blind out "
+            f"of sample — {wf.get('efficiency', 0):.0%} efficiency, with "
+            f"{wf.get('oos_win_rate', 0):.0%} of folds profitable.",
+            "",
+        ]
+    lines += [
+        f"**Costs.** Sharpe is {report.cost_stress.get('sharpe_1x', 0):.2f} at the modelled friction "
+        f"and {report.cost_stress.get('sharpe_3x', 0):.2f} at triple it "
+        f"({report.cost_stress.get('ratio', 0):.0%} retained), on turnover of "
+        f"{report.backtest.metrics.get('turnover_ann', 0):.1f}x a year.",
+        "",
+        "> Past performance, simulated or otherwise, does not predict future returns. "
+        "This is research tooling, not investment advice.",
+    ]
+    return "\n".join(lines)
+
+
+def analyse_portfolio(
+    symbols: str,
+    start: str,
+    end: str,
+    strategy_key: str,
+    p1: float,
+    p2: float,
+    p3: float,
+    p4: float,
+    rebalance: str,
+    commission: float,
+    slippage: float,
+    allow_short: bool,
+    gross_leverage: float,
+    max_weight: float,
+    n_permutations: int,
+    progress=gr.Progress(),
+):
+    """Portfolio Lab handler. Like the single-asset one, it never raises into the UI."""
+    try:
+        universe = [s.strip().upper() for s in (symbols or "").replace("\n", ",").split(",") if s.strip()]
+        if len(universe) < 4:
+            raise ValueError(
+                "A cross-sectional strategy needs at least 4 symbols — it ranks names against "
+                "each other, and there is nothing to rank in a list this short."
+            )
+        strategy = get_xs_strategy(strategy_key)
+        values = (p1, p2, p3, p4)
+        params = {
+            spec.name: spec.cast(values[i])
+            for i, spec in enumerate(strategy.params[:MAX_XS_PARAMS])
+        }
+        cfg = PortfolioLabConfig(
+            symbols=universe,
+            start=start or "2015-01-01",
+            end=end or None,
+            source="yahoo",
+            strategy=strategy_key,
+            params=params,
+            commission_bps=float(commission),
+            slippage_bps=float(slippage),
+            allow_short=bool(allow_short),
+            gross_leverage=float(gross_leverage),
+            max_weight=float(max_weight) if max_weight else None,
+            rebalance=rebalance,
+            n_permutations=int(n_permutations),
+        )
+        report = run_portfolio_lab(cfg, progress=lambda f, m: progress(f, desc=m))
+    except Exception as exc:  # noqa: BLE001
+        logger.exception("Portfolio lab run failed")
+        message = (
+            f'<div class="score-card"><div class="score-body"><h3>Could not run that</h3>'
+            f"<p>{exc}</p></div></div>"
+        )
+        blank = empty_figure("No results.")
+        return message, "", blank, blank, blank, blank, blank, ""
+
+    return (
+        _portfolio_card(report),
+        _portfolio_tiles(report),
+        cross_permutation_chart(report),
+        equity_chart(report, benchmark_label="Equal weight"),
+        attribution_chart(report.attribution),
+        weights_chart(report),
+        score_chart(report.verdict, significance_label="Picks the right names"),
+        _portfolio_detail(report),
+    )
+
+
+def _detail_markdown(report) -> str:
+    dsr, wf, pbo, stress = report.dsr, report.walkforward, report.pbo, report.cost_stress
+    mtr = dsr.get("min_track_record_years", float("inf"))
+    mtr_text = f"{mtr:.1f} years" if mtr == mtr and mtr != float("inf") else "never, at this effect size"
+
+    lines = [
+        "### Reading the evidence",
+        "",
+        f"**Selection bias.** {report.trials.get('n', 1)} parameter variants of "
+        f"*{report.strategy.name}* were backtested. The luckiest skill-free variant of that many "
+        f"would be expected to show an annualised Sharpe of about "
+        f"**{dsr.get('threshold_sr_annual', 0):.2f}** on its own. Yours was "
+        f"**{report.backtest.sharpe:.2f}**, which puts the deflated probability of a real edge at "
+        f"**{dsr.get('dsr', 0):.0%}**.",
+        "",
+        f"**Track record needed.** To call this Sharpe significant at 95% confidence given its "
+        f"skew ({dsr.get('skew', 0):.2f}) and kurtosis ({dsr.get('kurtosis', 0):.1f}), you would need "
+        f"about **{mtr_text}** of live returns.",
+        "",
+        f"**Costs.** At the modelled friction the Sharpe is {stress.get('sharpe_1x', 0):.2f}. "
+        f"Triple the costs and it becomes {stress.get('sharpe_3x', 0):.2f} "
+        f"({stress.get('ratio', 0):.0%} retained).",
+        "",
+    ]
+
+    if wf.get("folds"):
+        lines += [
+            f"**Walk-forward.** Across {len(wf['folds'])} folds the tuned in-sample Sharpe averaged "
+            f"{wf.get('mean_is_sharpe', 0):.2f} and the blind out-of-sample Sharpe averaged "
+            f"{wf.get('mean_oos_sharpe', 0):.2f} — an efficiency of {wf.get('efficiency', 0):.0%}. "
+            f"{wf.get('oos_win_rate', 0):.0%} of folds were profitable out of sample, and the tuner "
+            f"picked a different parameter set in {wf.get('param_instability', 0):.0%} of them.",
+            "",
+        ]
+    if pbo.get("note"):
+        lines += [f"**Overfitting.** {pbo['note']}", ""]
+    elif pbo.get("pbo") == pbo.get("pbo"):
+        lines += [
+            f"**Overfitting (CSCV).** Over {pbo.get('n_combinations', 0)} in/out splits of "
+            f"{pbo.get('n_strategies', 0)} variants, the in-sample winner landed in the bottom half "
+            f"out of sample **{pbo.get('pbo', 0):.0%}** of the time, and lost money outright "
+            f"{pbo.get('prob_oos_loss', 0):.0%} of the time. The most frequently selected variant was "
+            f"`{pbo.get('most_selected_label', 'n/a')}` "
+            f"({pbo.get('selection_stability', 0):.0%} of splits).",
+            "",
+        ]
+
+    lines += [
+        "> Past performance, simulated or otherwise, does not predict future returns. "
+        "This is research tooling, not investment advice.",
+    ]
+    return "\n".join(lines)
+
+
+def analyse(
+    symbol: str,
+    start: str,
+    end: str,
+    strategy_key: str,
+    p1: float,
+    p2: float,
+    p3: float,
+    commission: float,
+    slippage: float,
+    allow_short: bool,
+    n_permutations: int,
+    perm_method: str,
+    wf_folds: int,
+    progress=gr.Progress(),
+):
+    """Main Lab handler. Never raises into the UI — it returns a readable message."""
+    try:
+        cfg = LabConfig(
+            symbol=symbol or "SPY",
+            start=start or "2015-01-01",
+            end=end or None,
+            source="yahoo",
+            strategy=strategy_key,
+            params=collect_params(strategy_key, p1, p2, p3),
+            commission_bps=float(commission),
+            slippage_bps=float(slippage),
+            allow_short=bool(allow_short),
+            n_permutations=int(n_permutations),
+            permutation_method="block" if perm_method.startswith("Block") else "permute",
+            wf_folds=int(wf_folds),
+        )
+        report = run_lab(cfg, progress=lambda f, m: progress(f, desc=m))
+    except Exception as exc:  # noqa: BLE001 - the UI must always say something useful
+        logger.exception("Lab run failed")
+        message = f'<div class="score-card"><div class="score-body"><h3>Could not run that</h3><p>{exc}</p></div></div>'
+        blank = empty_figure("No results.")
+        return message, "", blank, blank, blank, blank, blank, blank, ""
+
+    return (
+        _score_card(report),
+        _tiles(report),
+        permutation_chart(report),
+        equity_chart(report),
+        drawdown_chart(report),
+        exposure_chart(report),
+        walkforward_chart(report),
+        score_chart(report.verdict),
+        _detail_markdown(report),
+    )
+
+
+def race(symbol: str, start: str, allow_short: bool, n_permutations: int, progress=gr.Progress()):
+    try:
+        cfg = LabConfig(
+            symbol=symbol or "SPY",
+            start=start or "2015-01-01",
+            source="yahoo",
+            allow_short=bool(allow_short),
+        )
+        table, market, _ = run_arena(
+            cfg,
+            n_permutations=int(n_permutations),
+            progress=lambda f, m: progress(f, desc=m),
+        )
+    except Exception as exc:  # noqa: BLE001
+        logger.exception("Arena run failed")
+        return pd.DataFrame({"Error": [str(exc)]}), empty_figure("No results."), ""
+
+    display = table.copy()
+    for col in ("Return", "CAGR", "MaxDD"):
+        display[col] = display[col].map(lambda v: f"{v * 100:,.1f}%")
+    for col in ("Sharpe", "DSR", "Evidence"):
+        display[col] = display[col].map(lambda v: f"{v:.2f}")
+    display["p-value"] = display["p-value"].map(lambda v: "—" if v != v else f"{v:.3f}")
+    display = display.drop(columns=["key"])
+
+    provenance = (
+        f"Data: {market.source} · {market.symbol} · {market.start.date()} to {market.end.date()}"
+        if market.is_real
+        else f"⚠ {market.note}"
+    )
+    note = (
+        f"{provenance}\n\nRanked by **evidence** — `(1 − p) × deflated Sharpe` — not by return. "
+        "*Buy & Hold* and *Coin Flip* are in the field on purpose: a leaderboard without a "
+        "control group is marketing, not measurement."
+    )
+    return display, arena_chart(table), note
+
+
+HOW_IT_WORKS = """
+## Why most backtests are wrong
+
+A backtest is a measurement taken with a ruler you built after seeing the thing you
+are measuring. Four failure modes do almost all the damage, and this Space tests for
+each one.
+
+### 1. The market had no structure to find — permutation test
+
+We take the real price series and shuffle it: each bar's gap, high, low, body and
+volume are kept intact, but their **order** is destroyed. The result is a market with
+the same volatility and the same fat tails, and no exploitable structure whatsoever.
+Then we re-run *your exact rule* on hundreds of these shuffled markets.
+
+If your Sharpe sits inside that cloud of results, your rule found nothing that a
+coin-flip market would not also have handed it. The **p-value** is the share of
+shuffled markets that did as well or better.
+
+*Block mode* resamples contiguous chunks instead of single bars, preserving
+short-horizon momentum and volatility clustering. It is a harder null, and trend
+strategies should be held to it.
+
+### 2. You tried 200 things and reported the best — Deflated Sharpe Ratio
+
+If you test 200 worthless strategies, the best of them will show a Sharpe near 1.0
+purely by chance. The **Deflated Sharpe Ratio** (Bailey & López de Prado, 2014) works
+out what the luckiest of *N* skill-free variants would have scored, and asks whether
+yours beats that bar — with an extra penalty for negative skew and fat tails, the
+return shapes that flatter naive Sharpe ratios.
+
+This Space counts the whole parameter grid as trials, because that is what a
+researcher would really have run.
+
+### 3. The parameters were fitted to the past — PBO and walk-forward
+
+**Probability of Backtest Overfitting** (CSCV) cuts the timeline into chunks, and for
+every way of splitting them half in-sample and half out-of-sample, checks whether the
+in-sample winner stayed a winner. If the winner lands in the bottom half about half
+the time, PBO ≈ 50% and your selection process has no skill at all.
+
+**Walk-forward** re-tunes on a training window and trades the next window blind,
+rolling forward. Efficiency is out-of-sample Sharpe over in-sample Sharpe: 100% means
+the edge survived intact, 0% means it was entirely curve-fit.
+
+### 4. The edge is smaller than the costs — stress test
+
+Every result here is net of commission and slippage charged on exposure changes, plus
+a borrow fee on short positions. We then re-run at **triple** the friction. A real edge
+degrades; a fake one disappears.
+
+---
+
+## The Reality Score
+
+| Weight | Component | What it measures |
+|---:|---|---|
+| 30% | Significance | How far outside the shuffled-market null the result sits |
+| 25% | Selection | Deflated Sharpe — does it clear the best-of-N bar |
+| 20% | Walk-forward | How much of the tuned Sharpe survived trading forward |
+| 15% | Overfitting | 1 − PBO, from combinatorially symmetric cross-validation |
+| 10% | Robustness | Sharpe retained when costs triple |
+
+Grades: **A** ≥ 85 · **B** ≥ 70 · **C** ≥ 55 · **D** ≥ 40 · **F** below 40.
+
+The scale is deliberately harsh. Most strategies people post online score below 40,
+and the honest response to that is not to soften the scale.
+
+## No look-ahead, by construction
+
+A strategy emits a target exposure at each bar's close using only data up to that
+bar. The engine holds `position[t] = target[t - lag]` with `lag ≥ 1`, so a signal
+computed on Tuesday's close cannot earn Tuesday's move. That is the single line where
+look-ahead could enter, and the test suite asserts it directly.
+
+## Use it from Python
+
+```python
+from algotrader import LabConfig, run_lab
+
+report = run_lab(LabConfig(symbol="SPY", strategy="sma_cross", params={"fast": 20, "slow": 100}))
+print(report.verdict["grade"], report.verdict["score"])
+print(report.permutation.p_value, report.dsr["dsr"], report.pbo["pbo"])
+```
+
+Or from the command line:
+
+```bash
+python -m algotrader.cli lab --symbol SPY --strategy donchian_breakout --permutations 500
+python -m algotrader.cli arena --symbol BTC-USD
+```
+
+---
+
+*Research tooling, not investment advice. Nothing here is a recommendation to trade.*
+"""
+
+
+# Gradio 6 moved `css` and `theme` from the Blocks constructor to launch().
+# Spaces pin their own version, so pass them wherever the installed one wants.
+_GRADIO_MAJOR = int(gr.__version__.split(".")[0])
+_STYLE_KWARGS = {"css": CSS, "theme": gr.themes.Base()}
+_BLOCKS_KWARGS = {} if _GRADIO_MAJOR >= 6 else _STYLE_KWARGS
+# Gradio 6 also dropped launch(show_api=...).
+_LAUNCH_KWARGS = dict(_STYLE_KWARGS) if _GRADIO_MAJOR >= 6 else {"show_api": False}
+
+
+def build_app() -> gr.Blocks:
+    with gr.Blocks(title="Backtest Reality Check", **_BLOCKS_KWARGS) as demo:
+        with gr.Column(elem_id="hero"):
+            gr.HTML(
+                "<h1>Backtest Reality Check</h1>"
+                "<p>Your backtest is probably lying to you. Pick a market and a trading rule — "
+                "this runs it, then spends the rest of its effort trying to prove the result "
+                "was luck.</p>"
+            )
+
+        with gr.Tabs():
+            with gr.Tab("The Lab"):
+                with gr.Row():
+                    with gr.Column(scale=1):
+                        symbol = gr.Dropdown(
+                            choices=DEFAULT_UNIVERSE, value="SPY", label="Ticker",
+                            allow_custom_value=True,
+                            info="Any Yahoo Finance symbol. Falls back to a simulated market if offline.",
+                        )
+                        with gr.Row():
+                            start = gr.Textbox(value="2015-01-01", label="Start", scale=1)
+                            end = gr.Textbox(value="", label="End (blank = today)", scale=1)
+
+                        strategy = gr.Dropdown(
+                            choices=STRATEGY_CHOICES, value="sma_cross", label="Strategy"
+                        )
+                        strategy_note = gr.Markdown(get_strategy("sma_cross").description)
+
+                        param_sliders = [
+                            gr.Slider(label=f"Parameter {i + 1}", visible=False, minimum=0, maximum=100)
+                            for i in range(MAX_PARAMS)
+                        ]
+
+                        with gr.Accordion("Costs and testing", open=False):
+                            commission = gr.Slider(0, 20, value=1, step=0.5, label="Commission (bps per trade)")
+                            slippage = gr.Slider(0, 50, value=2, step=0.5, label="Slippage (bps per trade)")
+                            allow_short = gr.Checkbox(value=True, label="Allow short positions")
+                            n_perms = gr.Slider(
+                                0, 1000, value=250, step=50, label="Shuffled markets to test against",
+                                info="More is stricter and slower. 250 is plenty for a first look.",
+                            )
+                            perm_method = gr.Radio(
+                                ["Shuffle bars (standard)", "Block bootstrap (harder)"],
+                                value="Shuffle bars (standard)", label="Null market",
+                            )
+                            wf_folds = gr.Slider(2, 8, value=5, step=1, label="Walk-forward folds")
+
+                        run_button = gr.Button("Run reality check", variant="primary", size="lg")
+
+                        gr.Examples(
+                            label="Or try one of these",
+                            examples=[
+                                ["SPY", "sma_cross"],
+                                ["BTC-USD", "donchian_breakout"],
+                                ["NVDA", "rsi_reversion"],
+                                ["QQQ", "momentum"],
+                                ["SPY", "coin_flip"],
+                            ],
+                            inputs=[symbol, strategy],
+                        )
+
+                    with gr.Column(scale=2):
+                        verdict_html = gr.HTML(
+                            '<div class="score-card"><div class="score-body">'
+                            "<h3>Nothing tested yet</h3><p>Pick a market and a rule, then hit "
+                            "<b>Run reality check</b>. A full run is a few seconds.</p>"
+                            "</div></div>"
+                        )
+                        tiles_html = gr.HTML("")
+
+                # The headline test gets the full width — it is the whole point.
+                perm_plot = gr.Plot(value=empty_figure("The headline test appears here.", height=320))
+                equity_plot = gr.Plot(value=empty_figure())
+                with gr.Row():
+                    dd_plot = gr.Plot(value=empty_figure(height=240))
+                    exposure_plot = gr.Plot(value=empty_figure(height=200))
+                with gr.Row():
+                    wf_plot = gr.Plot(value=empty_figure(height=300))
+                    components_plot = gr.Plot(value=empty_figure(height=260))
+                detail_md = gr.Markdown("")
+
+                strategy.change(
+                    fn=param_controls, inputs=strategy, outputs=param_sliders
+                ).then(
+                    fn=lambda k: get_strategy(k).description, inputs=strategy, outputs=strategy_note
+                )
+
+                run_button.click(
+                    fn=analyse,
+                    inputs=[
+                        symbol, start, end, strategy, *param_sliders,
+                        commission, slippage, allow_short, n_perms, perm_method, wf_folds,
+                    ],
+                    outputs=[
+                        verdict_html, tiles_html, perm_plot, equity_plot,
+                        dd_plot, exposure_plot, wf_plot, components_plot, detail_md,
+                    ],
+                )
+
+            with gr.Tab("Portfolio"):
+                gr.Markdown(
+                    "Cross-sectional strategies rank names against each other, so they get a "
+                    "harder null: we keep every date's gross exposure, net exposure and position "
+                    "count exactly as they were and randomise only **which name got which "
+                    "weight**. A book that beats that is picking names. One that doesn't was "
+                    "being paid for style exposure you can buy in an ETF — which the factor "
+                    "regression below measures directly."
+                )
+                with gr.Row():
+                    with gr.Column(scale=1):
+                        xs_symbols = gr.Textbox(
+                            value=", ".join(PORTFOLIO_UNIVERSE),
+                            label="Universe",
+                            lines=3,
+                            info="Comma-separated. At least 4 names — fewer cannot be ranked.",
+                        )
+                        with gr.Row():
+                            xs_start = gr.Textbox(value="2015-01-01", label="Start", scale=1)
+                            xs_end = gr.Textbox(value="", label="End", scale=1)
+
+                        xs_strategy = gr.Dropdown(
+                            choices=XS_STRATEGY_CHOICES, value="xs_momentum", label="Strategy"
+                        )
+                        xs_note = gr.Markdown(get_xs_strategy("xs_momentum").description)
+                        xs_sliders = [
+                            gr.Slider(label=f"Parameter {i + 1}", visible=False, minimum=0, maximum=100)
+                            for i in range(MAX_XS_PARAMS)
+                        ]
+                        xs_rebalance = gr.Radio(
+                            ["D", "W", "M", "Q"], value="M", label="Rebalance",
+                            info="Daily rebalancing of a real book is rarely affordable.",
+                        )
+
+                        with gr.Accordion("Costs, limits and testing", open=False):
+                            xs_commission = gr.Slider(0, 20, value=1, step=0.5, label="Commission (bps)")
+                            xs_slippage = gr.Slider(0, 50, value=2, step=0.5, label="Slippage (bps)")
+                            xs_short = gr.Checkbox(value=True, label="Allow short positions")
+                            xs_leverage = gr.Slider(0.1, 3.0, value=1.0, step=0.1, label="Gross leverage cap")
+                            xs_maxw = gr.Slider(0.0, 1.0, value=0.25, step=0.05, label="Max weight per name")
+                            xs_perms = gr.Slider(
+                                0, 500, value=150, step=25, label="Name shuffles",
+                            )
+
+                        xs_button = gr.Button("Run portfolio check", variant="primary", size="lg")
+
+                    with gr.Column(scale=2):
+                        xs_verdict = gr.HTML(
+                            '<div class="score-card"><div class="score-body">'
+                            "<h3>Nothing tested yet</h3><p>Pick a universe and a ranking rule, then "
+                            "hit <b>Run portfolio check</b>.</p></div></div>"
+                        )
+                        xs_tiles = gr.HTML("")
+
+                xs_perm_plot = gr.Plot(value=empty_figure("The name-shuffle test appears here.", height=320))
+                xs_equity_plot = gr.Plot(value=empty_figure())
+                with gr.Row():
+                    xs_attr_plot = gr.Plot(value=empty_figure(height=280))
+                    xs_weights_plot = gr.Plot(value=empty_figure(height=240))
+                xs_components_plot = gr.Plot(value=empty_figure(height=260))
+                xs_detail = gr.Markdown("")
+
+                xs_strategy.change(
+                    fn=xs_param_controls, inputs=xs_strategy, outputs=xs_sliders
+                ).then(
+                    fn=lambda k: get_xs_strategy(k).description, inputs=xs_strategy, outputs=xs_note
+                )
+                xs_button.click(
+                    fn=analyse_portfolio,
+                    inputs=[
+                        xs_symbols, xs_start, xs_end, xs_strategy, *xs_sliders,
+                        xs_rebalance, xs_commission, xs_slippage, xs_short,
+                        xs_leverage, xs_maxw, xs_perms,
+                    ],
+                    outputs=[
+                        xs_verdict, xs_tiles, xs_perm_plot, xs_equity_plot,
+                        xs_attr_plot, xs_weights_plot, xs_components_plot, xs_detail,
+                    ],
+                )
+
+            with gr.Tab("Arena"):
+                gr.Markdown(
+                    "Race every strategy on the same market, ranked by **evidence** rather than "
+                    "return. Buy & hold and a coin flip stay in the field as controls."
+                )
+                with gr.Row():
+                    arena_symbol = gr.Dropdown(
+                        choices=DEFAULT_UNIVERSE, value="SPY", label="Ticker", allow_custom_value=True
+                    )
+                    arena_start = gr.Textbox(value="2015-01-01", label="Start")
+                    arena_short = gr.Checkbox(value=True, label="Allow shorts")
+                    arena_perms = gr.Slider(0, 400, value=120, step=20, label="Shuffled markets per strategy")
+                arena_button = gr.Button("Run the arena", variant="primary")
+                arena_note = gr.Markdown("")
+                arena_table = gr.Dataframe(interactive=False, wrap=True)
+                arena_plot = gr.Plot(value=empty_figure(height=380))
+
+                arena_button.click(
+                    fn=race,
+                    inputs=[arena_symbol, arena_start, arena_short, arena_perms],
+                    outputs=[arena_table, arena_plot, arena_note],
+                )
+
+            with gr.Tab("How it works"):
+                gr.Markdown(HOW_IT_WORKS)
+
+        gr.Markdown(
+            f"<sub>algotrader {__version__} · Apache-2.0 · "
+            "Research tooling, not investment advice.</sub>"
+        )
+
+        demo.load(fn=param_controls, inputs=strategy, outputs=param_sliders)
+        demo.load(fn=xs_param_controls, inputs=xs_strategy, outputs=xs_sliders)
+
+    return demo
+
+
+if __name__ == "__main__":
+    build_app().queue(max_size=24).launch(
+        server_name="0.0.0.0",
+        server_port=int(os.environ.get("PORT", 7860)),
+        **_LAUNCH_KWARGS,
+    )

config.yaml

@@ -1,11 +1,11 @@
 # Configuration file for the agentic AI trading system
 data_source:
-  type: 'csv'
+  type: 'yahoo'
   path: 'data/market_data.csv'
 
 trading:
   symbol: 'AAPL'
-  timeframe: '1m'
+  timeframe: '1d'
   capital: 100000
 
 risk:
@@ -29,11 +29,17 @@ alpaca:
   websocket_url: 'wss://stream.data.alpaca.markets/v2/iex'  # WebSocket URL
   account_type: 'paper'  # 'paper' or 'live'
 
-# Yahoo Finance (optional). Set data_source.type: 'yahoo' to use.
+# Yahoo Finance (default ingest). Unofficial API, typically delayed; 1m lookback is ~7 days.
 # Unofficial API, typically delayed; 1m lookback is ~7 days.
 yahoo:
   poll_interval_seconds: 60
-  auto_adjust: false
+  # Keep true. With auto_adjust off, Yahoo returns raw Close and every stock
+  # split reads as a crash (NVDA's June 2024 10:1 becomes a -90% bar).
+  auto_adjust: true
+  # Yahoo returns the still-forming period as an ordinary row. Emitting it would
+  # trade on a close that has not happened yet.
+  emit_incomplete_bars: false
+  max_backoff_seconds: 900
   start_date: '2024-01-01'
   end_date: '2026-12-31'
 

docs/AGENTIC_SYSTEM_V1.md

@@ -0,0 +1,130 @@
+# Algorithmic Trading
+
+FinRL reinforcement-learning trading with Alpaca execution, plus Yahoo Finance OHLCV as the default public tape. Parallel LLC.
+
+This is **research and paper-trading infrastructure**. Live capital requires a separate evaluation contract, feature-parity tests, and a rewritten execution path. Do not treat `paper_trading: false` as a promotion gate.
+
+---
+
+## 1. Title and Summary
+
+**Algorithmic Trading**  
+Ingest OHLCV, compute indicators or train a FinRL policy, size orders under position and drawdown caps, route to paper or live Alpaca.
+
+GitHub `main` is the FinRL / Docker / Streamlit tree plus algotrader 2.0. `dev` is the integration branch. Yahoo is the default `data_source.type`.
+
+**Design themes**
+
+* Four ingest paths: CSV replay, synthetic GBM, Alpaca REST, Yahoo (`yfinance>=1.0`)
+* FinRL policies (PPO, A2C, DDPG, TD3) on a Gymnasium-style environment
+* Alpaca for authenticated market data and order routing (paper by default)
+* Yahoo for delayed public bars when no broker key is available
+* Secrets from environment (`ALPACA_API_KEY`, `ALPACA_SECRET_KEY`), never committed
+* Tests and Docker/CI as already present on this tree
+
+---
+
+## 2. Concepts and Methods
+
+### Market data
+
+| Source | When to use | Failure modes |
+| ------ | ----------- | ------------- |
+| **CSV** | Offline replay; default in `config.yaml` | Missing path or OHLCV columns → `None` |
+| **Synthetic** | Unit tests and demos | GBM is not tradable edge |
+| **Alpaca** | Authenticated bars and live/paper orders | Auth, feed, and rate-limit failures |
+| **Yahoo** | Real Close without a broker account | Unofficial API, ~15 min delay, interval lookback caps (1m ≈ 7 days). Pin `yfinance>=1.0`; 0.2.x fails against the current chart API |
+
+`load_data` dispatches on `data_source.type`. Existing `alpaca` / `csv` / `synthetic` branches are unchanged.
+
+### Strategy and FinRL
+
+* `StrategyAgent`: SMA, RSI, Bollinger, MACD on Close; teaching rule, not an alpha claim
+* `FinRLAgent`: PPO / A2C / DDPG / TD3 via Stable-Baselines3; persist under `models/`
+* `ExecutionAgent` / `AlpacaBroker`: paper simulation or Alpaca market/limit orders
+
+Backtests in this repo are in-sample passes unless you add a purged walk-forward yourself. Leakage is the null hypothesis.
+
+---
+
+## 3. Stack
+
+| Layer | Tools |
+| ----- | ----- |
+| Language | Python 3.11 (CI); 3.8+ stated for local |
+| RL | FinRL / Stable-Baselines3, Gym/Gymnasium, PyTorch |
+| Broker | alpaca-py |
+| Market data | Alpaca REST; yfinance ≥ 1.0 (Yahoo) |
+| Tabular | pandas, NumPy, scikit-learn |
+| UI | Streamlit, Dash, Jupyter widgets |
+| Deploy | Docker Compose, GitHub Actions |
+| Tests | pytest, pytest-cov |
+
+---
+
+## 4. Structure
+
+```
+algorithmic_trading/
+├── agentic_ai_system/     # ingest, strategy, FinRL, Alpaca, Yahoo
+├── ui/                    # Streamlit, Dash, Jupyter, WebSocket
+├── tests/
+├── models/                # trained artifacts (gitignored bodies)
+├── data/                  # generated CSV (gitignored)
+├── scripts/               # Docker / deploy helpers
+├── .github/workflows/     # CI/CD, release, backtesting
+├── config.yaml
+├── requirements.txt
+├── Dockerfile
+└── docker-compose*.yml
+```
+
+Branch policy: **`main`** (protected) and **`dev`** only. Do not re-enable Dependabot or the Monday `dependency-updates` workflow; those created extra branches.
+
+---
+
+## 5. Quick start
+
+```bash
+git clone https://github.com/ParallelLLC/algorithmic_trading.git
+cd algorithmic_trading
+python -m venv .venv && source .venv/bin/activate
+pip install -r requirements.txt
+cp .env.example .env   # Alpaca keys if using alpaca ingest or orders
+```
+
+Default ingest is CSV. For Yahoo daily bars without a broker:
+
+```yaml
+data_source:
+  type: 'yahoo'
+trading:
+  symbol: 'AAPL'
+  timeframe: '1d'
+```
+
+```bash
+python demo.py
+python -m agentic_ai_system.main --mode backtest --start-date 2024-01-01 --end-date 2024-12-31
+pytest tests/ -q
+```
+
+UI launchers and Docker are documented in `UI_SETUP.md` and `DOCKER_HUB_SETUP.md`. Paper-trade before live. Yahoo is not a SIP tape.
+
+---
+
+## 6. Configuration (additive Yahoo keys)
+
+| Key | Meaning |
+| --- | ------- |
+| `data_source.type` | `csv` \| `synthetic` \| `alpaca` \| `yahoo` |
+| `yahoo.start_date` / `end_date` | Historical window; clamped per Yahoo interval limits |
+| `yahoo.auto_adjust` | Passed to `yfinance` |
+| `execution.broker_api` | `paper` \| `alpaca_paper` \| `alpaca_live` |
+| `finrl.algorithm` | PPO, A2C, DDPG, TD3 |
+
+---
+
+**License:** Apache License 2.0  
+**Organization:** [Parallel LLC](https://github.com/ParallelLLC)  
+**Repository:** <https://github.com/ParallelLLC/algorithmic_trading>

requirements-space.txt

@@ -0,0 +1,12 @@
+# Dependencies for the Hugging Face Space (app.py + the algotrader package).
+#
+# Deliberately minimal: a Space that takes ten minutes to build is a Space
+# nobody waits for. The v1 agentic system's heavier stack (torch,
+# stable-baselines3, dash, alpaca-py) lives in requirements.txt and is not
+# needed to run the Lab.
+gradio>=4.44,<7
+numpy>=1.24
+pandas>=2.0
+scipy>=1.10
+plotly>=5.18
+yfinance>=0.2.40

scripts/deploy_hf_space.sh

@@ -0,0 +1,65 @@
+#!/usr/bin/env bash
+#
+# Assemble and push the Hugging Face Space.
+#
+# The Space gets only what it needs to run: app.py, the algotrader package, the
+# Space card, and the minimal requirements file. The v1 agentic system and its
+# heavy ML stack stay in this repo, so the Space builds in under a minute.
+#
+# Usage:
+#   HF_TOKEN=hf_xxx ./scripts/deploy_hf_space.sh <hf-username>/<space-name>
+#
+# Create the Space first at https://huggingface.co/new-space (SDK: Gradio).
+
+set -euo pipefail
+
+SPACE_ID="${1:-}"
+if [[ -z "$SPACE_ID" ]]; then
+  echo "usage: HF_TOKEN=hf_xxx $0 <hf-username>/<space-name>" >&2
+  exit 2
+fi
+if [[ -z "${HF_TOKEN:-}" ]]; then
+  echo "error: HF_TOKEN is not set. Create a write token at https://huggingface.co/settings/tokens" >&2
+  exit 2
+fi
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+STAGING="$(mktemp -d)"
+trap 'rm -rf "$STAGING"' EXIT
+
+echo "==> Staging Space contents in $STAGING"
+git clone --quiet "https://user:${HF_TOKEN}@huggingface.co/spaces/${SPACE_ID}" "$STAGING/space"
+cd "$STAGING/space"
+
+# Replace tracked content wholesale so deletions propagate, but keep .git.
+find . -mindepth 1 -maxdepth 1 ! -name .git -exec rm -rf {} +
+
+cp -r "$REPO_ROOT/algotrader" ./algotrader
+cp "$REPO_ROOT/app.py" ./app.py
+cp "$REPO_ROOT/LICENSE" ./LICENSE
+cp "$REPO_ROOT/SPACE_README.md" ./README.md
+cp "$REPO_ROOT/requirements-space.txt" ./requirements.txt
+find ./algotrader -name '__pycache__' -type d -exec rm -rf {} + 2>/dev/null || true
+
+# A small test suite ships too: reviewers who check whether the statistics are
+# real are exactly the audience worth convincing.
+mkdir -p tests
+cp "$REPO_ROOT/tests/test_v2_engine.py" \
+   "$REPO_ROOT/tests/test_v2_validation.py" \
+   "$REPO_ROOT/tests/test_v2_strategies.py" \
+   "$REPO_ROOT/tests/test_v2_portfolio.py" tests/
+
+echo "==> Files staged:"
+find . -path ./.git -prune -o -type f -print | sed 's|^\./|  |'
+
+git add -A
+if git diff --cached --quiet; then
+  echo "==> Space is already up to date; nothing to push."
+  exit 0
+fi
+
+git -c user.email="deploy@localhost" -c user.name="space-deploy" \
+    commit --quiet -m "Deploy algotrader $(python3 -c 'import sys; sys.path.insert(0, "'"$REPO_ROOT"'"); import algotrader; print(algotrader.__version__)')"
+git push --quiet origin HEAD:main
+
+echo "==> Pushed. Live at https://huggingface.co/spaces/${SPACE_ID}"

tests/test_data_ingestion.py

@@ -132,7 +132,7 @@ class TestDataIngestion:
                 
                 assert isinstance(result, pd.DataFrame)
                 assert len(result) == len(sample_csv_data)
-                assert result['timestamp'].dtype == 'datetime64[ns]'
+                assert pd.api.types.is_datetime64_any_dtype(result['timestamp'])
                 
             finally:
                 os.unlink(tmp_file.name)
@@ -289,7 +289,7 @@ class TestDataIngestion:
                 result = _load_csv_data(config)
                 
                 # Check that timestamp is converted to datetime
-                assert result['timestamp'].dtype == 'datetime64[ns]'
+                assert pd.api.types.is_datetime64_any_dtype(result['timestamp'])
                 
             finally:
                 os.unlink(tmp_file.name)

tests/test_synthetic_data_generator.py

@@ -56,8 +56,8 @@ class TestSyntheticDataGenerator:
             assert col in df.columns
         
         # Check data types
-        assert df['timestamp'].dtype == 'datetime64[ns]'
-        assert df['symbol'].dtype == 'object'
+        assert pd.api.types.is_datetime64_any_dtype(df['timestamp'])
+        assert pd.api.types.is_string_dtype(df['symbol'])
         assert df['open'].dtype in ['float64', 'float32']
         assert df['high'].dtype in ['float64', 'float32']
         assert df['low'].dtype in ['float64', 'float32']

tests/test_v2_cli.py

@@ -0,0 +1,89 @@
+"""CLI surface — argument handling and that each command actually completes."""
+
+from __future__ import annotations
+
+import json
+
+import pytest
+
+from algotrader.cli import _parse_params, main
+
+
+class TestArgumentParsing:
+    def test_params_are_parsed_into_floats(self):
+        assert _parse_params(["fast=10", "slow=50.5"]) == {"fast": 10.0, "slow": 50.5}
+
+    def test_params_without_an_equals_sign_are_rejected(self):
+        with pytest.raises(SystemExit, match="name=value"):
+            _parse_params(["fast10"])
+
+    def test_no_params_is_an_empty_dict(self):
+        assert _parse_params(None) == {}
+
+    def test_an_unknown_strategy_is_rejected_at_parse_time(self):
+        with pytest.raises(SystemExit):
+            main(["lab", "--strategy", "nope"])
+
+    def test_a_missing_subcommand_is_rejected(self):
+        with pytest.raises(SystemExit):
+            main([])
+
+
+class TestCommands:
+    """All runs force the simulator so the suite never touches the network."""
+
+    def test_lab_prints_a_report(self, capsys):
+        code = main([
+            "lab", "--symbol", "SPY", "--source", "synthetic", "--strategy", "sma_cross",
+            "--start", "2018-01-01", "--end", "2023-01-01",
+            "--permutations", "10", "--folds", "2", "--quiet",
+        ])
+        out = capsys.readouterr().out
+        assert code == 0
+        assert "REALITY SCORE" in out
+        assert "Deflated Sharpe" in out
+
+    def test_lab_json_output_is_machine_readable(self, capsys):
+        code = main([
+            "lab", "--source", "synthetic", "--start", "2018-01-01", "--end", "2023-01-01",
+            "--permutations", "10", "--folds", "2", "--quiet", "--json",
+        ])
+        payload = json.loads(capsys.readouterr().out)
+        assert code == 0
+        assert 0.0 <= payload["verdict"]["score"] <= 100.0
+        assert payload["verdict"]["grade"] in {"A", "B", "C", "D", "F"}
+        assert payload["source"] == "synthetic"
+
+    def test_lab_honours_param_overrides(self, capsys):
+        main([
+            "lab", "--source", "synthetic", "--start", "2018-01-01", "--end", "2023-01-01",
+            "--strategy", "sma_cross", "--param", "fast=5", "--param", "slow=60",
+            "--permutations", "0", "--folds", "2", "--quiet", "--json",
+        ])
+        payload = json.loads(capsys.readouterr().out)
+        assert payload["params"] == {"fast": 5, "slow": 60}
+        assert payload["p_value"] is None  # permutations disabled
+
+    def test_arena_ranks_the_zoo(self, capsys):
+        code = main([
+            "arena", "--source", "synthetic", "--start", "2018-01-01", "--end", "2023-01-01",
+            "--permutations", "0", "--quiet",
+        ])
+        out = capsys.readouterr().out
+        assert code == 0
+        assert "Buy & Hold" in out and "Coin Flip" in out
+        assert "Ranked by evidence" in out
+
+    def test_strategies_lists_the_registry(self, capsys):
+        assert main(["strategies"]) == 0
+        out = capsys.readouterr().out
+        assert "sma_cross" in out and "donchian_breakout" in out
+
+    def test_errors_are_reported_not_raised(self, capsys):
+        code = main([
+            "lab", "--source", "synthetic",
+            "--start", "2022-01-01", "--end", "2022-02-01",  # far too short
+            "--permutations", "0", "--quiet",
+        ])
+        assert code == 1
+        assert "error:" in capsys.readouterr().err

tests/test_v2_engine.py

@@ -0,0 +1,158 @@
+"""Engine correctness: alignment, costs, and the no-look-ahead guarantee."""
+
+from __future__ import annotations
+
+import numpy as np
+import pandas as pd
+import pytest
+
+from algotrader.engine import bars_to_returns, run_backtest
+from algotrader.metrics import compute_metrics, infer_periods_per_year, max_drawdown, sharpe_ratio
+from algotrader.types import CostModel
+
+
+def make_bars(n: int = 400, seed: int = 0) -> pd.DataFrame:
+    rng = np.random.default_rng(seed)
+    close = 100 * np.exp(np.cumsum(rng.normal(0.0003, 0.01, n)))
+    index = pd.date_range("2020-01-01", periods=n, freq="B")
+    return pd.DataFrame(
+        {
+            "open": close,
+            "high": close * 1.005,
+            "low": close * 0.995,
+            "close": close,
+            "volume": 1e6,
+        },
+        index=index,
+    )
+
+
+class TestNoLookAhead:
+    """The one property the whole project rests on."""
+
+    def test_signal_earns_the_following_bar_not_its_own(self):
+        df = make_bars()
+        asset_ret = bars_to_returns(df)
+
+        # A target that knows the *next* bar's direction must be perfect...
+        clairvoyant = np.sign(asset_ret.shift(-1)).fillna(0.0)
+        good = run_backtest(df, clairvoyant, costs=CostModel(0, 0, 0))
+        assert (good.returns.iloc[1:-1] >= -1e-12).all(), "perfect foresight should never lose"
+
+        # ...and a target that only knows the *current* bar must not be.
+        hindsight = np.sign(asset_ret).fillna(0.0)
+        meh = run_backtest(df, hindsight, costs=CostModel(0, 0, 0))
+        assert (meh.returns < 0).any(), "same-bar signal must not be risk-free"
+        assert good.sharpe > meh.sharpe
+
+    def test_position_is_target_shifted_by_lag(self):
+        df = make_bars(200)
+        target = pd.Series(np.linspace(-1, 1, len(df)), index=df.index)
+        for lag in (1, 2, 5):
+            result = run_backtest(df, target, lag=lag)
+            expected = target.shift(lag).fillna(0.0)
+            pd.testing.assert_series_equal(result.position, expected, check_names=False)
+
+    def test_lag_zero_is_rejected(self):
+        df = make_bars(120)
+        with pytest.raises(ValueError, match="lag"):
+            run_backtest(df, pd.Series(1.0, index=df.index), lag=0)
+
+    def test_future_bars_cannot_change_past_equity(self):
+        """Truncating the data must not alter the equity curve before the cut."""
+        df = make_bars(400)
+        target = pd.Series(np.tile([1.0, -1.0], len(df) // 2), index=df.index)
+
+        full = run_backtest(df, target)
+        cut = run_backtest(df.iloc[:250], target.iloc[:250])
+        np.testing.assert_allclose(
+            full.equity.iloc[:250].to_numpy(), cut.equity.to_numpy(), rtol=1e-12
+        )
+
+
+class TestCosts:
+    def test_buy_and_hold_pays_once(self):
+        df = make_bars(300)
+        target = pd.Series(1.0, index=df.index)
+        result = run_backtest(df, target, costs=CostModel(commission_bps=5, slippage_bps=5, short_borrow_bps=0))
+        # One entry at 10bps one-way, and no further turnover.
+        assert result.costs.sum() == pytest.approx(10 / 1e4, rel=1e-9)
+        assert int(result.metrics["n_trades"]) == 1
+
+    def test_flipping_every_bar_costs_more_than_holding(self):
+        df = make_bars(300)
+        costs = CostModel(commission_bps=5, slippage_bps=5, short_borrow_bps=0)
+        hold = run_backtest(df, pd.Series(1.0, index=df.index), costs=costs)
+        flip = run_backtest(df, pd.Series(np.tile([1.0, -1.0], 150), index=df.index), costs=costs)
+        assert flip.costs.sum() > 100 * hold.costs.sum()
+
+    def test_zero_costs_means_gross_equals_net(self):
+        df = make_bars(200)
+        target = pd.Series(np.tile([1.0, 0.0], 100), index=df.index)
+        result = run_backtest(df, target, costs=CostModel(0, 0, 0))
+        pd.testing.assert_series_equal(result.returns, result.gross_returns, check_names=False)
+
+    def test_short_borrow_is_charged_only_on_shorts(self):
+        df = make_bars(260)
+        costs = CostModel(commission_bps=0, slippage_bps=0, short_borrow_bps=365)
+        long_only = run_backtest(df, pd.Series(1.0, index=df.index), costs=costs)
+        short_only = run_backtest(df, pd.Series(-1.0, index=df.index), costs=costs)
+        assert long_only.costs.sum() == pytest.approx(0.0, abs=1e-12)
+        assert short_only.costs.sum() > 0
+
+    def test_higher_costs_never_improve_returns(self):
+        df = make_bars(300)
+        target = pd.Series(np.tile([1.0, -1.0], 150), index=df.index)
+        cheap = run_backtest(df, target, costs=CostModel(1, 1, 0))
+        dear = run_backtest(df, target, costs=CostModel(20, 20, 0))
+        assert dear.equity.iloc[-1] < cheap.equity.iloc[-1]
+
+
+class TestConstraints:
+    def test_shorts_are_clipped_when_disallowed(self):
+        df = make_bars(150)
+        target = pd.Series(-1.0, index=df.index)
+        result = run_backtest(df, target, allow_short=False)
+        assert (result.target >= 0).all()
+        assert (result.position >= 0).all()
+
+    def test_leverage_is_clipped(self):
+        df = make_bars(150)
+        result = run_backtest(df, pd.Series(5.0, index=df.index), max_leverage=1.5)
+        assert result.target.max() == pytest.approx(1.5)
+
+    def test_empty_frame_is_rejected(self):
+        with pytest.raises(ValueError):
+            run_backtest(pd.DataFrame(columns=["open", "high", "low", "close", "volume"]), pd.Series(dtype=float))
+
+
+class TestMetrics:
+    def test_sharpe_of_constant_returns_is_zero_not_infinite(self):
+        flat = pd.Series([0.001] * 100)
+        assert sharpe_ratio(flat, 252) == 0.0
+
+    def test_sharpe_scales_with_annualisation(self):
+        rng = np.random.default_rng(1)
+        returns = pd.Series(rng.normal(0.001, 0.01, 5000))
+        assert sharpe_ratio(returns, 252) == pytest.approx(sharpe_ratio(returns, 1) * np.sqrt(252))
+
+    def test_max_drawdown_matches_a_hand_worked_example(self):
+        equity = pd.Series([100.0, 120.0, 60.0, 90.0])
+        assert max_drawdown(equity) == pytest.approx(-0.5)
+
+    def test_buy_and_hold_metrics_match_the_price_series(self):
+        df = make_bars(500)
+        result = run_backtest(df, pd.Series(1.0, index=df.index), costs=CostModel(0, 0, 0))
+        expected = df["close"].iloc[-1] / df["close"].iloc[0] - 1.0
+        assert result.metrics["total_return"] == pytest.approx(expected, rel=1e-9)
+
+    def test_periodicity_inference(self):
+        daily = pd.date_range("2020-01-01", periods=300, freq="B")
+        assert 200 <= infer_periods_per_year(daily) <= 300
+        hourly = pd.date_range("2020-01-01", periods=300, freq="h")
+        assert infer_periods_per_year(hourly) > 1000
+
+    def test_metrics_survive_a_degenerate_series(self):
+        empty = pd.Series(dtype=float)
+        out = compute_metrics(empty, pd.Series(dtype=float))
+        assert "periods_per_year" in out

tests/test_v2_portfolio.py

@@ -0,0 +1,377 @@
+"""Multi-asset engine, panel, cross-sectional strategies and their null."""
+
+from __future__ import annotations
+
+import numpy as np
+import pandas as pd
+import pytest
+
+from algotrader.attribution import build_style_factors, factor_attribution
+from algotrader.cross_sectional import XS_REGISTRY, get_xs_strategy, scores_to_weights
+from algotrader.data import simulate_ohlcv
+from algotrader.engine import run_backtest
+from algotrader.panel import Panel, load_panel
+from algotrader.portfolio import (
+    normalise_weights,
+    rebalance_schedule,
+    run_portfolio_backtest,
+)
+from algotrader.types import CostModel
+from algotrader.validation.cross_permutation import (
+    cross_sectional_permutation_test,
+    permute_within_dates,
+)
+
+XS_KEYS = sorted(XS_REGISTRY)
+
+
+def make_panel(n_assets=8, n=1200, drift_sd=0.0, seed=1, common_vol=0.008, idio=0.012) -> Panel:
+    """A synthetic universe. ``drift_sd`` controls genuine cross-sectional structure."""
+    rng = np.random.default_rng(seed)
+    index = pd.date_range("2016-01-01", periods=n, freq="B")
+    drift = rng.normal(0, drift_sd, n_assets)
+    returns = (
+        rng.normal(0.0002, common_vol, n)[:, None]
+        + rng.normal(0, idio, (n, n_assets))
+        + drift[None, :]
+    )
+    close = 100 * np.exp(np.cumsum(returns, axis=0))
+    columns = [f"A{i}" for i in range(n_assets)]
+
+    def frame(values):
+        return pd.DataFrame(values, index=index, columns=columns)
+
+    return Panel(
+        fields={
+            "open": frame(close),
+            "high": frame(close * 1.004),
+            "low": frame(close * 0.996),
+            "close": frame(close),
+            "volume": frame(np.full_like(close, 1e6)),
+        },
+        sources={c: "synthetic" for c in columns},
+    )
+
+
+@pytest.fixture(scope="module")
+def panel() -> Panel:
+    return make_panel()
+
+
+class TestPanel:
+    def test_fields_are_aligned(self, panel):
+        assert panel.shape == (1200, 8)
+        for name in ("open", "high", "low", "close", "volume"):
+            assert panel.fields[name].index.equals(panel.index)
+            assert list(panel.fields[name].columns) == panel.symbols
+
+    def test_misaligned_fields_are_rejected(self, panel):
+        broken = dict(panel.fields)
+        broken["high"] = broken["high"].iloc[:-5]
+        with pytest.raises(ValueError, match="not aligned"):
+            Panel(fields=broken)
+
+    def test_missing_field_is_rejected(self, panel):
+        with pytest.raises(ValueError, match="missing field"):
+            Panel(fields={"close": panel.close})
+
+    def test_missing_data_is_not_forward_filled(self):
+        """Filling a gap invents liquidity that never existed."""
+        df = simulate_ohlcv("AAPL", "2018-01-01", "2022-01-01")
+        gapped = df.drop(df.index[100:140])
+        built = Panel.from_frames({"AAPL": gapped, "MSFT": df})
+        assert built.close["AAPL"].isna().sum() == 40
+
+    def test_tradable_requires_two_consecutive_prices(self, panel):
+        assert not panel.tradable().iloc[0].any()
+        assert panel.tradable().iloc[1:].all().all()
+
+    def test_returns_are_masked_where_untradable(self, panel):
+        assert panel.returns().iloc[0].isna().all()
+
+    def test_delisting_is_detected(self):
+        df = simulate_ohlcv("AAPL", "2016-01-01", "2022-01-01")
+        dead = df.iloc[: len(df) // 2]
+        built = Panel.from_frames({"ALIVE": df, "DEAD": dead})
+        report = built.survivorship()
+        assert report.n_delisted == 1
+        assert "DEAD" in report.delisted_symbols
+        assert not report.biased
+
+    def test_all_survivors_is_flagged_as_biased(self, panel):
+        report = panel.survivorship()
+        assert report.survival_rate == 1.0
+        assert report.biased
+        assert "upper bound" in report.note
+
+    def test_load_panel_skips_symbols_without_history(self):
+        built = load_panel(["SPY", "AAPL"], "2018-01-01", "2022-01-01", source="synthetic")
+        assert set(built.symbols) == {"SPY", "AAPL"}
+        assert all(s == "synthetic" for s in built.sources.values())
+
+    def test_select_and_slice(self, panel):
+        subset = panel.select(["A0", "A1"])
+        assert subset.symbols == ["A0", "A1"]
+        sliced = panel.slice(panel.index[10], panel.index[50])
+        assert len(sliced) == 41
+
+
+class TestPortfolioEngine:
+    def test_matches_single_asset_engine_exactly(self):
+        """One column through the matrix engine must equal the fast path."""
+        df = simulate_ohlcv("SPY", "2018-01-01", "2023-01-01")
+        single = Panel.from_frames({"SPY": df})
+        costs = CostModel(1, 2, 50)
+        n = len(df)
+        cases = {
+            "buy_and_hold": np.ones(n),
+            "flip": np.tile([1.0, -1.0], n // 2 + 1)[:n],
+            "long_flat": np.tile([1.0, 0.0], n // 2 + 1)[:n],
+            "fractional": np.full(n, 0.5),
+        }
+        for name, target in cases.items():
+            portfolio = run_portfolio_backtest(
+                single, pd.DataFrame({"SPY": target}, index=df.index), costs=costs
+            )
+            direct = run_backtest(df, pd.Series(target, index=df.index), costs=costs)
+            np.testing.assert_allclose(
+                portfolio.returns.to_numpy(), direct.returns.to_numpy(),
+                atol=1e-12, err_msg=f"mismatch for {name}",
+            )
+
+    def test_holding_still_costs_nothing_but_drift_does(self, panel):
+        """A full-notional long needs no rebalancing; a short does."""
+        costs = CostModel(10, 10, 0)
+        long_only = pd.DataFrame(1.0 / len(panel.symbols), index=panel.index, columns=panel.symbols)
+        result = run_portfolio_backtest(panel, long_only, costs=costs, gross_leverage=1.0)
+        # Equal-weight across many names still drifts apart, so turnover > 0...
+        assert result.metrics["turnover_ann"] > 0
+        # ...but a single full-notional position does not.
+        one = pd.DataFrame(0.0, index=panel.index, columns=panel.symbols)
+        one["A0"] = 1.0
+        assert run_portfolio_backtest(panel, one, costs=costs).costs.sum() == pytest.approx(
+            20 / 1e4, rel=1e-6
+        )
+
+    def test_rebalancing_less_often_lowers_turnover(self, panel):
+        weights = get_xs_strategy("equal_weight").generate(panel)
+        turnovers = []
+        for frequency in ("D", "M", "Q"):
+            result = run_portfolio_backtest(
+                panel, weights, costs=CostModel(1, 2, 0),
+                rebalance_on=rebalance_schedule(panel.index, frequency),
+            )
+            turnovers.append(result.metrics["turnover_ann"])
+        assert turnovers[0] > turnovers[1] > turnovers[2]
+
+    def test_gross_leverage_is_capped(self, panel):
+        weights = pd.DataFrame(1.0, index=panel.index, columns=panel.symbols)
+        result = run_portfolio_backtest(panel, weights, gross_leverage=1.0)
+        assert result.weights.abs().sum(axis=1).max() <= 1.0 + 1e-9
+
+    def test_leverage_is_scaled_down_never_up(self, panel):
+        small = pd.DataFrame(0.01, index=panel.index, columns=panel.symbols)
+        result = run_portfolio_backtest(panel, small, gross_leverage=1.0)
+        assert result.weights.abs().sum(axis=1).max() < 0.2
+
+    def test_per_name_cap_is_applied(self, panel):
+        weights = pd.DataFrame(0.0, index=panel.index, columns=panel.symbols)
+        weights["A0"] = 1.0
+        result = run_portfolio_backtest(panel, weights, max_weight=0.1)
+        assert result.weights.abs().max().max() <= 0.1 + 1e-9
+
+    def test_shorts_are_blocked_when_disallowed(self, panel):
+        weights = pd.DataFrame(-0.1, index=panel.index, columns=panel.symbols)
+        result = run_portfolio_backtest(panel, weights, allow_short=False)
+        assert (result.weights >= 0).all().all()
+
+    def test_delisted_names_cannot_be_held(self):
+        df = simulate_ohlcv("AAPL", "2016-01-01", "2022-01-01")
+        built = Panel.from_frames({"ALIVE": df, "DEAD": df.iloc[: len(df) // 2]})
+        weights = pd.DataFrame(0.5, index=built.index, columns=built.symbols)
+        result = run_portfolio_backtest(built, weights)
+        after_death = built.close["DEAD"].last_valid_index()
+        assert result.held.loc[result.held.index > after_death, "DEAD"].abs().max() == 0.0
+
+    def test_lag_zero_is_rejected(self, panel):
+        with pytest.raises(ValueError, match="lag"):
+            run_portfolio_backtest(panel, pd.DataFrame(0.1, index=panel.index, columns=panel.symbols), lag=0)
+
+    def test_future_bars_cannot_change_past_equity(self, panel):
+        weights = get_xs_strategy("xs_momentum").generate(panel)
+        full = run_portfolio_backtest(panel, weights)
+        cut = run_portfolio_backtest(
+            panel.slice(panel.index[0], panel.index[799]), weights.iloc[:800]
+        )
+        np.testing.assert_allclose(
+            full.equity.iloc[:800].to_numpy(), cut.equity.to_numpy(), rtol=1e-10
+        )
+
+    def test_attribution_sums_to_gross_return(self, panel):
+        weights = get_xs_strategy("equal_weight").generate(panel)
+        result = run_portfolio_backtest(panel, weights, costs=CostModel(0, 0, 0))
+        assert result.attribution().sum() == pytest.approx(result.gross_returns.sum(), rel=1e-9)
+
+    def test_portfolio_metrics_are_reported(self, panel):
+        result = run_portfolio_backtest(panel, get_xs_strategy("xs_momentum").generate(panel))
+        for key in ("gross_exposure", "net_exposure", "avg_positions", "concentration_hhi"):
+            assert key in result.metrics
+
+
+class TestCrossSectionalStrategies:
+    @pytest.mark.parametrize("key", XS_KEYS)
+    def test_weights_are_well_formed(self, panel, key):
+        weights = get_xs_strategy(key).generate(panel)
+        assert weights.shape == panel.shape
+        assert weights.notna().all().all()
+        assert weights.abs().sum(axis=1).max() <= 1.0 + 1e-9
+
+    @pytest.mark.parametrize("key", XS_KEYS)
+    def test_weights_are_causal(self, panel, key):
+        cut = 700
+        full = get_xs_strategy(key).generate(panel).iloc[:cut]
+        partial = get_xs_strategy(key).generate(panel.slice(panel.index[0], panel.index[cut - 1]))
+        pd.testing.assert_frame_equal(full, partial, rtol=1e-9)
+
+    def test_long_short_books_are_dollar_neutral(self, panel):
+        weights = get_xs_strategy("xs_momentum").generate(panel)
+        active = weights[weights.abs().sum(axis=1) > 1e-9]
+        assert active.sum(axis=1).abs().max() < 1e-9
+
+    def test_long_only_uses_the_whole_book(self):
+        scores = pd.DataFrame(np.arange(40).reshape(4, 10).astype(float))
+        weights = scores_to_weights(scores, long_frac=0.3, long_only=True)
+        assert weights.sum(axis=1).round(9).eq(1.0).all()
+        assert (weights >= 0).all().all()
+
+    def test_thin_cross_sections_are_skipped(self):
+        scores = pd.DataFrame(np.random.default_rng(0).normal(size=(10, 3)))
+        assert scores_to_weights(scores).abs().sum(axis=1).max() == 0.0
+
+    def test_equal_weight_holds_everything(self, panel):
+        weights = get_xs_strategy("equal_weight").generate(panel)
+        assert (weights.iloc[-1] > 0).all()
+
+    def test_unknown_strategy_names_alternatives(self):
+        with pytest.raises(KeyError, match="Available"):
+            get_xs_strategy("nope")
+
+
+class TestCrossSectionalNull:
+    def test_permutation_preserves_the_shape_of_the_book(self, panel):
+        weights = get_xs_strategy("xs_momentum").generate(panel)
+        investable = panel.close.notna()
+        shuffled = permute_within_dates(weights, investable, np.random.default_rng(0))
+
+        np.testing.assert_allclose(
+            np.sort(weights.to_numpy(), axis=1), np.sort(shuffled.to_numpy(), axis=1), atol=1e-12
+        )
+        np.testing.assert_allclose(
+            weights.abs().sum(axis=1).to_numpy(), shuffled.abs().sum(axis=1).to_numpy(), atol=1e-12
+        )
+        np.testing.assert_allclose(
+            weights.sum(axis=1).to_numpy(), shuffled.sum(axis=1).to_numpy(), atol=1e-12
+        )
+
+    def test_permutation_actually_reassigns(self, panel):
+        weights = get_xs_strategy("xs_momentum").generate(panel)
+        shuffled = permute_within_dates(weights, panel.close.notna(), np.random.default_rng(0))
+        assert not np.allclose(weights.to_numpy(), shuffled.to_numpy())
+
+    def test_non_investable_names_stay_empty(self):
+        df = simulate_ohlcv("AAPL", "2016-01-01", "2022-01-01")
+        built = Panel.from_frames({"ALIVE": df, "DEAD": df.iloc[: len(df) // 2]})
+        weights = pd.DataFrame(0.5, index=built.index, columns=built.symbols)
+        shuffled = permute_within_dates(weights, built.close.notna(), np.random.default_rng(1))
+        after_death = built.close["DEAD"].last_valid_index()
+        assert shuffled.loc[shuffled.index > after_death, "DEAD"].abs().max() == 0.0
+
+    def test_real_cross_sectional_skill_is_detected(self):
+        strong = make_panel(drift_sd=0.003, seed=11)
+        weights = get_xs_strategy("xs_momentum").generate(strong)
+        result = cross_sectional_permutation_test(
+            strong, weights, n_permutations=100,
+            rebalance_on=rebalance_schedule(strong.index, "M"), seed=2,
+        )
+        assert result.observed > result.null_mean
+        assert result.p_value < 0.05
+
+    def test_no_structure_is_not_significant(self):
+        flat = make_panel(drift_sd=0.0, seed=12)
+        weights = get_xs_strategy("xs_momentum").generate(flat)
+        result = cross_sectional_permutation_test(
+            flat, weights, n_permutations=100,
+            rebalance_on=rebalance_schedule(flat.index, "M"), seed=4,
+        )
+        assert result.p_value > 0.05
+
+    def test_p_value_can_never_be_zero(self, panel):
+        weights = get_xs_strategy("xs_momentum").generate(panel)
+        result = cross_sectional_permutation_test(panel, weights, n_permutations=20, seed=0)
+        assert result.p_value >= 1 / 21
+
+
+class TestAttribution:
+    def test_equal_weight_is_explained_by_the_market(self, panel):
+        factors = build_style_factors(panel)
+        weights = get_xs_strategy("equal_weight").generate(panel)
+        result = run_portfolio_backtest(panel, weights)
+        report = factor_attribution(result.returns, factors)
+
+        assert report["available"]
+        assert report["r_squared"] > 0.85
+        assert report["dominant_factor"] == "market"
+        assert not report["alpha_significant"]
+        assert "more cheaply" in report["note"]
+
+    def test_a_factor_regressed_on_itself_has_no_alpha(self, panel):
+        factors = build_style_factors(panel)
+        report = factor_attribution(factors["market"], factors)
+        assert abs(report["alpha_annual"]) < 0.05
+        assert report["r_squared"] > 0.99
+
+    def test_pure_noise_has_no_alpha_and_no_fit(self, panel):
+        rng = np.random.default_rng(5)
+        noise = pd.Series(rng.normal(0, 0.01, len(panel)), index=panel.index)
+        report = factor_attribution(noise, build_style_factors(panel))
+        assert not report["alpha_significant"]
+        assert report["r_squared"] < 0.2
+
+    def test_short_series_degrade_gracefully(self, panel):
+        factors = build_style_factors(panel)
+        report = factor_attribution(factors["market"].iloc[:20], factors)
+        assert not report["available"]
+        assert report["note"]
+
+
+class TestPortfolioLab:
+    def test_full_pipeline(self):
+        from algotrader.portfolio_lab import PortfolioLabConfig, run_portfolio_lab
+
+        report = run_portfolio_lab(
+            PortfolioLabConfig(
+                symbols=["SPY", "QQQ", "AAPL", "MSFT", "NVDA", "GLD"],
+                start="2017-01-01", end="2023-01-01", source="synthetic",
+                strategy="xs_momentum", n_permutations=20, wf_folds=2, grid_limit=6,
+            )
+        )
+        assert 0.0 <= report.verdict["score"] <= 100.0
+        assert report.verdict["grade"] in {"A", "B", "C", "D", "F"}
+        assert 0 < report.permutation.p_value <= 1
+        assert report.attribution["available"]
+        assert report.survivorship.n_symbols == 6
+        assert report.cost_stress["sharpe_3x"] <= report.cost_stress["sharpe_1x"] + 1e-9
+
+    def test_survivorship_bias_reaches_the_verdict(self):
+        from algotrader.portfolio_lab import PortfolioLabConfig, run_portfolio_lab
+
+        report = run_portfolio_lab(
+            PortfolioLabConfig(
+                symbols=["SPY", "QQQ", "AAPL", "MSFT", "NVDA", "GLD"],
+                start="2015-01-01", end="2023-01-01", source="synthetic",
+                strategy="equal_weight", n_permutations=0, wf_folds=2, grid_limit=3,
+            )
+        )
+        assert report.survivorship.biased
+        assert any("still trading" in f for f in report.verdict["flags"])
+        assert report.verdict["score"] <= 60

tests/test_v2_strategies.py

@@ -0,0 +1,234 @@
+"""Strategy zoo, data loading, and the end-to-end Lab pipeline."""
+
+from __future__ import annotations
+
+import numpy as np
+import pandas as pd
+import pytest
+
+from algotrader import LabConfig, run_lab
+from algotrader.data import _normalise, load_ohlcv, simulate_ohlcv
+from algotrader.indicators import atr, bollinger, donchian, ema, macd, rsi, sma
+from algotrader.lab import run_arena
+from algotrader.strategies import REGISTRY, get_strategy, list_strategies
+from algotrader.types import MarketData
+from algotrader.verdict import reality_score
+
+ALL_KEYS = sorted(REGISTRY)
+
+
+@pytest.fixture(scope="module")
+def bars() -> pd.DataFrame:
+    return simulate_ohlcv("AAPL", "2016-01-01", "2023-01-01")
+
+
+class TestIndicatorsAreCausal:
+    """An indicator at bar t must not move when bars after t arrive."""
+
+    @pytest.mark.parametrize(
+        "fn",
+        [
+            lambda d: sma(d["close"], 20),
+            lambda d: ema(d["close"], 12),
+            lambda d: rsi(d["close"], 14),
+            lambda d: macd(d["close"])[0],
+            lambda d: bollinger(d["close"])[2],
+            lambda d: atr(d, 14),
+            lambda d: donchian(d, 20)[1],
+        ],
+    )
+    def test_prefix_is_stable(self, bars, fn):
+        cut = 800
+        full = fn(bars).iloc[:cut]
+        partial = fn(bars.iloc[:cut])
+        pd.testing.assert_series_equal(full, partial, check_names=False, rtol=1e-9)
+
+    def test_rsi_stays_in_range(self, bars):
+        values = rsi(bars["close"], 14).dropna()
+        assert values.between(0, 100).all()
+
+    def test_donchian_excludes_the_current_bar(self, bars):
+        _, upper = donchian(bars, 20)
+        # A breakout must be possible: the channel cannot already contain today.
+        assert (bars["high"] > upper).any()
+
+
+class TestStrategies:
+    @pytest.mark.parametrize("key", ALL_KEYS)
+    def test_output_is_well_formed(self, bars, key):
+        target = get_strategy(key).generate(bars)
+        assert target.index.equals(bars.index)
+        assert target.notna().all()
+        assert target.between(-1.0, 1.0).all()
+
+    @pytest.mark.parametrize("key", ALL_KEYS)
+    def test_signals_are_causal(self, bars, key):
+        cut = 900
+        full = get_strategy(key).generate(bars).iloc[:cut]
+        partial = get_strategy(key).generate(bars.iloc[:cut])
+        pd.testing.assert_series_equal(full, partial, check_names=False, rtol=1e-9)
+
+    @pytest.mark.parametrize("key", ALL_KEYS)
+    def test_grid_is_non_empty_and_valid(self, key):
+        strategy = get_strategy(key)
+        grid = strategy.grid(limit=40)
+        assert 1 <= len(grid) <= 40
+        for combo in grid:
+            assert set(combo) <= {p.name for p in strategy.params}
+            if "fast" in combo and "slow" in combo:
+                assert combo["fast"] < combo["slow"]
+
+    def test_unknown_strategy_names_the_alternatives(self):
+        with pytest.raises(KeyError, match="Available"):
+            get_strategy("does_not_exist")
+
+    def test_clean_ignores_unknown_params_and_casts_types(self):
+        strategy = get_strategy("sma_cross")
+        cleaned = strategy.clean({"fast": 15.7, "nonsense": 1})
+        assert cleaned == {"fast": 15, "slow": 100}
+
+    def test_buy_and_hold_is_always_fully_invested(self, bars):
+        assert (get_strategy("buy_and_hold").generate(bars) == 1.0).all()
+
+    def test_list_strategies_honours_exclusions(self):
+        keys = {s.key for s in list_strategies(exclude=["coin_flip"])}
+        assert "coin_flip" not in keys and "sma_cross" in keys
+
+
+class TestData:
+    def test_simulation_is_deterministic_per_symbol(self):
+        a = simulate_ohlcv("NVDA", "2018-01-01", "2022-01-01")
+        b = simulate_ohlcv("NVDA", "2018-01-01", "2022-01-01")
+        pd.testing.assert_frame_equal(a, b)
+
+    def test_different_symbols_simulate_differently(self):
+        a = simulate_ohlcv("NVDA", "2018-01-01", "2022-01-01")
+        b = simulate_ohlcv("TSLA", "2018-01-01", "2022-01-01")
+        assert not np.allclose(a["close"].to_numpy(), b["close"].to_numpy())
+
+    def test_simulated_bars_are_internally_consistent(self):
+        df = simulate_ohlcv("SPY", "2015-01-01", "2023-01-01")
+        assert (df["high"] >= df[["open", "close"]].max(axis=1) - 1e-9).all()
+        assert (df["low"] <= df[["open", "close"]].min(axis=1) + 1e-9).all()
+        assert (df["close"] > 0).all()
+        assert df.index.is_monotonic_increasing
+
+    def test_simulation_has_fat_tails_and_vol_clustering(self):
+        """Naive GBM flatters strategies; the simulator must be harder than that."""
+        returns = simulate_ohlcv("SPY", "2005-01-01", "2023-01-01")["close"].pct_change().dropna()
+        assert returns.kurtosis() > 1.0
+        assert returns.abs().autocorr(1) > 0.05
+
+    def test_yahoo_source_does_not_silently_simulate(self, monkeypatch):
+        monkeypatch.setattr("algotrader.data._download", lambda *a, **k: None)
+        with pytest.raises(RuntimeError, match="Yahoo returned no usable bars"):
+            load_ohlcv("SPY", "2018-01-01", "2022-01-01", source="yahoo")
+
+    def test_offline_load_falls_back_and_says_so(self, monkeypatch):
+        monkeypatch.setattr("algotrader.data._download", lambda *a, **k: None)
+        monkeypatch.setattr("algotrader.data._read_cache", lambda *a, **k: None)
+        market = load_ohlcv("SPY", "2018-01-01", "2022-01-01", source="auto")
+        assert market.source == "synthetic"
+        assert not market.is_real
+        assert "unavailable" in market.note
+
+    def test_normalise_handles_yahoo_style_frames(self):
+        index = pd.date_range("2020-01-01", periods=5, tz="UTC")
+        raw = pd.DataFrame(
+            {"Open": 1.0, "High": 2.0, "Low": 0.5, "Adj Close": 1.5, "Volume": 10},
+            index=index,
+        )
+        out = _normalise(raw)
+        assert list(out.columns) == ["open", "high", "low", "close", "volume"]
+        assert out.index.tz is None
+
+    def test_normalise_rejects_frames_with_no_price(self):
+        with pytest.raises(ValueError, match="missing required column"):
+            _normalise(pd.DataFrame({"volume": [1, 2, 3]}))
+
+    def test_too_short_a_range_is_rejected(self):
+        with pytest.raises(ValueError, match="too short"):
+            simulate_ohlcv("SPY", "2020-01-01", "2020-01-10")
+
+
+class TestVerdict:
+    def test_strong_evidence_outranks_weak_evidence(self):
+        metrics = {"n_trades": 300, "sharpe": 1.2, "total_return": 0.8, "max_drawdown": -0.2}
+        benchmark = {"sharpe": 0.4}
+        strong = reality_score(metrics, benchmark, p_value=0.001, dsr=0.99, pbo=0.02,
+                               wf_efficiency=0.9, wf_win_rate=1.0, cost_stress_ratio=0.9)
+        weak = reality_score(metrics, benchmark, p_value=0.45, dsr=0.10, pbo=0.55,
+                             wf_efficiency=-0.2, wf_win_rate=0.2, cost_stress_ratio=0.1)
+        assert strong["score"] > 85 > weak["score"]
+        assert strong["grade"] == "A" and weak["grade"] == "F"
+
+    def test_score_is_always_inside_the_scale(self):
+        for p in (0.0, 0.5, 1.0):
+            for dsr in (0.0, 1.0):
+                out = reality_score(
+                    {"n_trades": 100, "sharpe": 0.5, "total_return": 0.2, "max_drawdown": -0.1},
+                    {"sharpe": 0.1}, p_value=p, dsr=dsr, pbo=0.2,
+                    wf_efficiency=0.5, cost_stress_ratio=0.5,
+                )
+                assert 0.0 <= out["score"] <= 100.0
+
+    def test_losing_money_caps_the_score(self):
+        out = reality_score(
+            {"n_trades": 200, "sharpe": 0.3, "total_return": -0.4, "max_drawdown": -0.6},
+            {"sharpe": 0.5}, p_value=0.001, dsr=0.99, pbo=0.01,
+            wf_efficiency=1.0, cost_stress_ratio=1.0,
+        )
+        assert out["score"] <= 50
+        assert any("lost money" in f for f in out["flags"])
+
+    def test_too_few_trades_is_flagged_and_capped(self):
+        out = reality_score(
+            {"n_trades": 3, "sharpe": 2.5, "total_return": 1.0, "max_drawdown": -0.1},
+            {"sharpe": 0.3}, p_value=0.001, dsr=0.99, pbo=0.01,
+            wf_efficiency=1.0, cost_stress_ratio=1.0,
+        )
+        assert out["score"] <= 55
+        assert any("coin flips" in f for f in out["flags"])
+
+    def test_closet_indexing_is_called_out(self):
+        out = reality_score(
+            {"n_trades": 50, "sharpe": 0.6, "total_return": 0.5, "max_drawdown": -0.2},
+            {"sharpe": 0.6}, benchmark_correlation=0.99,
+        )
+        assert any("repackaged long position" in f for f in out["flags"])
+
+
+class TestLabEndToEnd:
+    def test_full_pipeline_produces_a_complete_report(self, monkeypatch):
+        monkeypatch.setattr("algotrader.lab.load_ohlcv", lambda *a, **k: MarketData(
+            "SIM", simulate_ohlcv("SPY", "2016-01-01", "2023-01-01"), "synthetic", "1d", "test"
+        ))
+        report = run_lab(LabConfig(strategy="sma_cross", n_permutations=25, wf_folds=3, grid_limit=12))
+
+        assert 0.0 <= report.verdict["score"] <= 100.0
+        assert report.verdict["grade"] in {"A", "B", "C", "D", "F"}
+        assert 0 < report.permutation.p_value <= 1
+        assert 0.0 <= report.dsr["dsr"] <= 1.0
+        assert report.trials["n"] > 1
+        assert len(report.backtest.equity) == len(report.market.df)
+        assert report.cost_stress["sharpe_3x"] <= report.cost_stress["sharpe_1x"] + 1e-9
+
+    def test_too_little_history_gives_a_readable_error(self, monkeypatch):
+        short = simulate_ohlcv("SPY", "2020-01-01", "2020-06-01")
+        monkeypatch.setattr(
+            "algotrader.lab.load_ohlcv",
+            lambda *a, **k: MarketData("SIM", short, "synthetic", "1d", "test"),
+        )
+        with pytest.raises(ValueError, match="Widen the date range"):
+            run_lab(LabConfig(n_permutations=0, wf_folds=2))
+
+    def test_arena_ranks_every_strategy_and_keeps_the_controls(self, monkeypatch):
+        monkeypatch.setattr("algotrader.lab.load_ohlcv", lambda *a, **k: MarketData(
+            "SIM", simulate_ohlcv("SPY", "2017-01-01", "2022-01-01"), "synthetic", "1d", "test"
+        ))
+        table, market, curves = run_arena(LabConfig(), n_permutations=0)
+
+        assert len(table) == len(REGISTRY)
+        assert {"buy_and_hold", "coin_flip"} <= set(table["key"])
+        assert table["Evidence"].is_monotonic_decreasing
+        assert set(curves) == set(REGISTRY)

tests/test_v2_validation.py

@@ -0,0 +1,179 @@
+"""Statistical machinery.
+
+These tests matter more than the engine's, because a validation suite that
+always says "no edge" is as useless as one that always says "great edge". Each
+class below checks both directions: it must reject noise *and* detect signal.
+"""
+
+from __future__ import annotations
+
+import numpy as np
+import pandas as pd
+import pytest
+
+from algotrader.data import simulate_ohlcv
+from algotrader.strategies import get_strategy
+from algotrader.validation.deflated_sharpe import (
+    deflated_sharpe_ratio,
+    expected_max_sharpe,
+    min_track_record_length,
+    probabilistic_sharpe_ratio,
+)
+from algotrader.validation.pbo import probability_of_backtest_overfitting
+from algotrader.validation.permutation import permutation_test, permute_bars
+from algotrader.validation.walkforward import walk_forward
+
+
+def trending_market(n: int = 2200, phi: float = 0.35, seed: int = 5) -> pd.DataFrame:
+    """A market with genuine, exploitable serial correlation."""
+    rng = np.random.default_rng(seed)
+    returns = np.zeros(n)
+    noise = rng.normal(0, 0.01, n)
+    for i in range(1, n):
+        returns[i] = phi * returns[i - 1] + noise[i]
+    close = 100 * np.exp(np.cumsum(returns))
+    index = pd.date_range("2012-01-01", periods=n, freq="B")
+    return pd.DataFrame(
+        {"open": close, "high": close * 1.004, "low": close * 0.996, "close": close, "volume": 1e6},
+        index=index,
+    )
+
+
+class TestPermutationMechanics:
+    def test_shuffling_preserves_the_distribution_of_moves(self):
+        df = simulate_ohlcv("SPY", "2018-01-01", "2023-01-01")
+        shuffled = permute_bars(df, np.random.default_rng(0))
+
+        assert len(shuffled) == len(df)
+        assert shuffled.index.equals(df.index)
+        original = np.sort(np.log(df["close"] / df["open"]).to_numpy()[1:])
+        permuted = np.sort(np.log(shuffled["close"] / shuffled["open"]).to_numpy()[1:])
+        np.testing.assert_allclose(original, permuted, rtol=1e-9)
+
+    def test_shuffling_keeps_bars_internally_valid(self):
+        df = simulate_ohlcv("AAPL", "2019-01-01", "2023-01-01")
+        shuffled = permute_bars(df, np.random.default_rng(3))
+        assert (shuffled["high"] >= shuffled["low"]).all()
+        assert (shuffled["high"] >= shuffled["close"]).all()
+        assert (shuffled["low"] <= shuffled["close"]).all()
+        assert (shuffled["close"] > 0).all()
+
+    def test_shuffling_destroys_serial_correlation(self):
+        df = trending_market()
+        real = df["close"].pct_change().dropna().autocorr(1)
+        shuffled = permute_bars(df, np.random.default_rng(1))["close"].pct_change().dropna().autocorr(1)
+        assert real > 0.2
+        assert abs(shuffled) < 0.1
+
+    def test_block_mode_retains_some_structure(self):
+        df = trending_market()
+        blocked = permute_bars(df, np.random.default_rng(2), method="block", block=40)
+        assert blocked["close"].pct_change().dropna().autocorr(1) > 0.1
+
+    def test_p_value_can_never_be_zero(self):
+        """+1 correction: the observed run is itself a draw from the null."""
+        df = trending_market()
+        strategy = get_strategy("momentum")
+        result = permutation_test(
+            df, lambda f: strategy.generate(f, {"lookback": 5}), n_permutations=30, seed=0
+        )
+        assert result.p_value >= 1 / 31
+        assert 0 < result.p_value <= 1
+
+
+class TestPermutationPower:
+    def test_real_edge_is_detected(self):
+        df = trending_market()
+        strategy = get_strategy("momentum")
+        result = permutation_test(
+            df, lambda f: strategy.generate(f, {"lookback": 5}), n_permutations=200, seed=1
+        )
+        assert result.observed > result.null.mean()
+        assert result.p_value < 0.05
+
+    def test_random_strategy_on_a_structureless_market_is_not_significant(self):
+        df = simulate_ohlcv("SIM", "2010-01-01", "2023-01-01")
+        strategy = get_strategy("coin_flip")
+        result = permutation_test(
+            df, lambda f: strategy.generate(f, {"hold": 5, "seed": 7}), n_permutations=200, seed=2
+        )
+        assert result.p_value > 0.05
+
+
+class TestDeflatedSharpe:
+    def test_selection_bar_rises_with_the_number_of_trials(self):
+        low = expected_max_sharpe(10, 0.01)
+        high = expected_max_sharpe(1000, 0.01)
+        assert 0 < low < high
+
+    def test_a_single_trial_has_no_selection_bar(self):
+        assert expected_max_sharpe(1, 0.01) == 0.0
+
+    def test_more_trials_lowers_the_deflated_sharpe(self):
+        rng = np.random.default_rng(7)
+        returns = rng.normal(0.0006, 0.01, 2000)
+        few = deflated_sharpe_ratio(returns, 1.0, 252, n_trials=2, variance_of_trials=0.01)
+        many = deflated_sharpe_ratio(returns, 1.0, 252, n_trials=500, variance_of_trials=0.01)
+        assert many["dsr"] < few["dsr"]
+        assert many["psr"] == pytest.approx(few["psr"])  # PSR ignores selection
+
+    def test_psr_rises_with_track_record_length(self):
+        short = probabilistic_sharpe_ratio(0.05, 100)
+        long = probabilistic_sharpe_ratio(0.05, 5000)
+        assert 0.5 < short < long < 1.0
+
+    def test_negative_skew_and_fat_tails_are_penalised(self):
+        clean = probabilistic_sharpe_ratio(0.06, 1000, skew=0.0, kurtosis=3.0)
+        nasty = probabilistic_sharpe_ratio(0.06, 1000, skew=-1.5, kurtosis=12.0)
+        assert nasty < clean
+
+    def test_track_record_requirement_is_infinite_below_the_bar(self):
+        assert min_track_record_length(0.01, 500, benchmark=0.05) == float("inf")
+        assert np.isfinite(min_track_record_length(0.10, 500, benchmark=0.02))
+
+
+class TestPBO:
+    def test_pure_noise_scores_near_one_half(self):
+        rng = np.random.default_rng(11)
+        matrix = rng.normal(0, 0.01, size=(1200, 30))  # 30 skill-free variants
+        result = probability_of_backtest_overfitting(matrix, n_splits=8)
+        assert 0.3 < result["pbo"] < 0.7
+
+    def test_a_genuinely_better_variant_is_not_flagged(self):
+        rng = np.random.default_rng(12)
+        matrix = rng.normal(0, 0.01, size=(1200, 20))
+        matrix[:, 3] += 0.004  # column 3 has a persistent, real edge
+        result = probability_of_backtest_overfitting(matrix, n_splits=8)
+        assert result["pbo"] < 0.15
+        assert result["most_selected_index"] == 3
+        assert result["selection_stability"] > 0.9
+
+    def test_too_few_variants_returns_nan_not_a_crash(self):
+        rng = np.random.default_rng(13)
+        result = probability_of_backtest_overfitting(rng.normal(0, 0.01, size=(500, 1)))
+        assert np.isnan(result["pbo"])
+        assert result["note"]
+
+    def test_odd_split_counts_are_made_even(self):
+        rng = np.random.default_rng(14)
+        result = probability_of_backtest_overfitting(rng.normal(0, 0.01, (800, 10)), n_splits=7)
+        assert result["n_combinations"] > 0
+
+
+class TestWalkForward:
+    def test_a_real_edge_survives_out_of_sample(self):
+        result = walk_forward(trending_market(), get_strategy("momentum"), n_folds=4)
+        assert result["folds"]
+        assert result["mean_oos_sharpe"] > 0
+        assert result["efficiency"] > 0.3
+
+    def test_folds_do_not_overlap_train_and_test(self):
+        result = walk_forward(trending_market(), get_strategy("sma_cross"), n_folds=4)
+        for fold in result["folds"]:
+            assert fold["train_end"] <= fold["test_start"]
+
+    def test_short_history_degrades_gracefully(self):
+        df = trending_market(n=150)
+        result = walk_forward(df, get_strategy("momentum"), n_folds=5)
+        assert result["folds"] == []
+        assert result["note"]

tests/test_yahoo_data_stream.py

@@ -1,3 +1,5 @@
+import logging
+
 import pandas as pd
 import pytest
 from unittest.mock import patch
@@ -32,6 +34,39 @@ def _sample_yahoo_frame():
     )
 
 
+def _split_frame():
+    """An unadjusted 10:1 split, as Yahoo returns it with auto_adjust=False.
+
+    Modelled on NVDA, 10 June 2024: the raw Close drops from ~1200 to ~120 and
+    a backtest reads it as a -90% day.
+    """
+    idx = pd.date_range('2024-06-06', periods=4, freq='D', tz='America/New_York')
+    close = [1200.0, 1208.0, 120.5, 121.0]
+    return pd.DataFrame(
+        {
+            'Open': close,
+            'High': [c * 1.01 for c in close],
+            'Low': [c * 0.99 for c in close],
+            'Close': close,
+            'Volume': [1_000_000] * 4,
+        },
+        index=idx,
+    )
+
+
+def _dated_frame(timestamps, close=100.0):
+    return pd.DataFrame(
+        {
+            'timestamp': [pd.Timestamp(t) for t in timestamps],
+            'open': close,
+            'high': close,
+            'low': close,
+            'close': close,
+            'volume': 1_000.0,
+        }
+    )
+
+
 class TestYahooDataStream:
     def test_initialization_from_symbol(self, yahoo_config):
         stream = YahooDataStream(yahoo_config)
@@ -58,3 +93,137 @@ class TestYahooDataStream:
             df = stream.get_historical_data('AAPL', '2024-01-01', '2024-12-31')
         assert len(df) == 3
         assert 'open' in df.columns
+
+
+class TestPriceAdjustment:
+    """Unadjusted prices turn every split into a phantom crash."""
+
+    def test_adjustment_is_on_by_default(self):
+        stream = YahooDataStream({'trading': {'symbol': 'AAPL', 'timeframe': '1d'}})
+        assert stream.auto_adjust is True
+
+    def test_auto_adjust_is_passed_through_to_yfinance(self):
+        stream = YahooDataStream({'trading': {'symbol': 'AAPL', 'timeframe': '1d'}})
+        with patch('yfinance.download', return_value=_sample_yahoo_frame()) as download:
+            stream._download('AAPL', period='5d', interval='1d')
+        assert download.call_args.kwargs['auto_adjust'] is True
+
+    def test_opting_out_of_adjustment_warns(self, caplog):
+        with caplog.at_level(logging.WARNING):
+            YahooDataStream({
+                'trading': {'symbol': 'AAPL', 'timeframe': '1d'},
+                'yahoo': {'auto_adjust': False},
+            })
+        assert any('not split' in r.message.lower() for r in caplog.records)
+
+    def test_split_sized_move_is_flagged(self, yahoo_config, caplog):
+        stream = YahooDataStream(yahoo_config)
+        df = stream._normalize_ohlcv(_split_frame())
+        with caplog.at_level(logging.WARNING):
+            found = stream._warn_if_unadjusted('NVDA', df)
+        assert found == 1
+        assert any('auto_adjust' in r.message for r in caplog.records)
+
+    def test_ordinary_moves_are_not_flagged(self, yahoo_config, caplog):
+        stream = YahooDataStream(yahoo_config)
+        df = stream._normalize_ohlcv(_sample_yahoo_frame())
+        with caplog.at_level(logging.WARNING):
+            assert stream._warn_if_unadjusted('AAPL', df) == 0
+
+    def test_intraday_bars_are_not_split_checked(self, yahoo_config):
+        """A 40% move in one minute is a halt or a fat finger, not a split."""
+        yahoo_config['trading']['timeframe'] = '1m'
+        stream = YahooDataStream(yahoo_config)
+        df = stream._normalize_ohlcv(_split_frame())
+        assert stream._warn_if_unadjusted('NVDA', df) == 0
+
+
+class TestIncompleteBars:
+    """The bar Yahoo is still building must not be reported as final."""
+
+    def test_forming_bar_is_dropped(self, yahoo_config):
+        stream = YahooDataStream(yahoo_config)
+        now = pd.Timestamp.now(tz='UTC').tz_convert(None).normalize()
+        df = _dated_frame([now - pd.Timedelta(days=2), now - pd.Timedelta(days=1), now])
+        kept = stream._drop_incomplete(df)
+        assert len(kept) == 2
+        assert kept['timestamp'].max() < now
+
+    def test_finished_bars_all_survive(self, yahoo_config):
+        stream = YahooDataStream(yahoo_config)
+        now = pd.Timestamp.now(tz='UTC').tz_convert(None).normalize()
+        df = _dated_frame([now - pd.Timedelta(days=5), now - pd.Timedelta(days=4)])
+        assert len(stream._drop_incomplete(df)) == 2
+
+    def test_opting_in_keeps_the_forming_bar(self, yahoo_config):
+        yahoo_config['yahoo']['emit_incomplete_bars'] = True
+        stream = YahooDataStream(yahoo_config)
+        now = pd.Timestamp.now(tz='UTC').tz_convert(None).normalize()
+        df = _dated_frame([now - pd.Timedelta(days=1), now])
+        assert len(stream._drop_incomplete(df)) == 2
+
+    def test_partial_bar_is_never_emitted_then_stranded(self, yahoo_config):
+        """The bug this guards: emitting the forming bar advanced the watermark,
+        so the finished version of that same bar never reached a callback."""
+        stream = YahooDataStream(yahoo_config)
+        received = []
+        stream.add_data_callback(lambda kind, bar: received.append(bar))
+
+        now = pd.Timestamp.now(tz='UTC').tz_convert(None).normalize()
+        yesterday, today = now - pd.Timedelta(days=1), now
+
+        stream._ingest_new_bars('AAPL', _dated_frame([yesterday, today], close=100.0))
+        assert len(received) == 1  # only yesterday's completed bar
+
+        # Next day: what was the forming bar is now final and must arrive.
+        with patch.object(stream, '_drop_incomplete', side_effect=lambda d: d):
+            stream._ingest_new_bars('AAPL', _dated_frame([yesterday, today], close=105.0))
+        assert len(received) == 2
+        assert received[-1]['close'] == 105.0
+
+
+class TestPollBackoff:
+    """Yahoo rate-limits hard, and a fixed interval keeps you throttled."""
+
+    def test_success_polls_at_the_configured_interval(self, yahoo_config):
+        yahoo_config['yahoo']['poll_interval_seconds'] = 60
+        stream = YahooDataStream(yahoo_config)
+        stream._consecutive_failures = 0
+        assert 48 <= stream._next_delay() <= 72  # 60s +/- jitter
+
+    def test_delay_grows_with_consecutive_failures(self, yahoo_config):
+        yahoo_config['yahoo']['poll_interval_seconds'] = 60
+        stream = YahooDataStream(yahoo_config)
+        delays = []
+        for failures in (1, 2, 3):
+            stream._consecutive_failures = failures
+            delays.append(stream._next_delay())
+        assert delays[0] < delays[1] < delays[2]
+
+    def test_backoff_is_capped(self, yahoo_config):
+        yahoo_config['yahoo']['poll_interval_seconds'] = 60
+        yahoo_config['yahoo']['max_backoff_seconds'] = 300
+        stream = YahooDataStream(yahoo_config)
+        stream._consecutive_failures = 20
+        assert stream._next_delay() <= 300 * 1.2
+
+    def test_jitter_desynchronises_retries(self, yahoo_config):
+        stream = YahooDataStream(yahoo_config)
+        stream._consecutive_failures = 3
+        assert len({stream._next_delay() for _ in range(20)}) > 1
+
+    def test_poll_reports_failure_when_every_symbol_fails(self, yahoo_config):
+        stream = YahooDataStream(yahoo_config)
+        with patch.object(stream, '_download', side_effect=RuntimeError('429 Too Many Requests')):
+            assert stream._poll_once() is False
+
+    def test_poll_reports_success_when_a_symbol_returns_bars(self, yahoo_config):
+        stream = YahooDataStream(yahoo_config)
+        with patch.object(stream, '_download', return_value=_sample_yahoo_frame()):
+            assert stream._poll_once() is True
+
+    def test_empty_response_counts_as_failure(self, yahoo_config):
+        """A rate-limited yfinance returns an empty frame rather than raising."""
+        stream = YahooDataStream(yahoo_config)
+        with patch.object(stream, '_download', return_value=pd.DataFrame()):
+            assert stream._poll_once() is False

ui/dash_app.py

@@ -158,11 +158,12 @@ class TradingDashApp:
                                     dbc.Select(
                                         id="data-source-select",
                                         options=[
+                                            {"label": "Yahoo Finance", "value": "yahoo"},
                                             {"label": "CSV File", "value": "csv"},
                                             {"label": "Alpaca API", "value": "alpaca"},
                                             {"label": "Synthetic Data", "value": "synthetic"}
                                         ],
-                                        value="csv"
+                                        value="yahoo"
                                     )
                                 ], width=4),
                                 dbc.Col([

ui/jupyter_widgets.py

@@ -62,8 +62,8 @@ class TradingJupyterUI:
         
         # Data widgets
         self.data_source = widgets.Dropdown(
-            options=['csv', 'alpaca', 'synthetic'],
-            value='csv',
+            options=['yahoo', 'csv', 'alpaca', 'synthetic'],
+            value='yahoo',
             description='Data Source:',
             style={'description_width': '120px'}
         )
@@ -76,7 +76,7 @@ class TradingJupyterUI:
         
         self.timeframe_input = widgets.Dropdown(
             options=['1m', '5m', '15m', '1h', '1d'],
-            value='1m',
+            value='1d',
             description='Timeframe:',
             style={'description_width': '120px'}
         )