README.md

---
license: cc-by-nc-4.0
library_name: pytorch
pipeline_tag: time-series-forecasting
tags:
  - time-series
  - forecasting
  - foundation-model
  - zero-shot
  - probabilistic-forecasting
  - gift-eval
---

# Modular-TS-2.4M

**A 2.4M-parameter time-series foundation model, pretrained from scratch by [Modular Computing](https://huggingface.co/Modularcomputing).**

Modular-TS-2.4M is a small, fast, zero-shot probabilistic forecaster: give it any univariate or multivariate series and it returns 21 quantiles for every future step, with no fine-tuning. It is built for the GPU-poor: it runs comfortably on a CPU or a phone-class device.

## Results (GIFT-Eval, full 97 configurations, zero-shot)

Geometric mean relative to Seasonal Naive (lower is better).

| Model | Params | MASE | CRPS |
|---|---|---|---|
| **Modular-TS-2.4M (ours)** | **2.41M** | **0.745** | **0.511** |
| IBM TTM-R3 (PT) | ~1–5M | 0.724 | 0.520 |
| TimeSSM-2.5M | 2.5M | 0.754 | 0.513 |
| Seasonal Naive | – | 1.000 | 1.000 |

- Best CRPS (probabilistic accuracy) of any open model at 2.5M parameters or fewer that we evaluated.
- Beats TimeSSM-2.5M on both metrics and TTM-R3 on CRPS; TTM-R3 is still ahead on MASE.
- Our numbers are strictly zero-shot: the model never sees a test dataset's own training split at inference time. (TTM-R3-PT uses each dataset's train split for backtest ensembling.)
- Evaluated with the official GIFT-Eval harness at leaderboard settings; the same harness reproduces published leaderboard scores for reference models exactly.

## Model

| | |
|---|---|
| Architecture | Decoder-style patch transformer (Timestral-style), d_model 128, 2 heads, MLP 320, 8 layers alternating time and variate attention, patch 32 |
| Parameters | 2,407,680 (embed 107,392 / trunk 1,804,672 / head 495,616) |
| Context | up to 8,192 steps |
| Output | 21 monotone quantiles per step (plus mean) |

The architecture code adapts layers from Mistral AI's Apache-2.0 `timestral` inference package. **No Timestral weights were loaded, fine-tuned, or distilled at any point.** Every weight in this repo was trained from random initialisation.

## Training data

- **Synthetic (majority):** a GPU-vectorised generator with Gaussian-process kernels, regime-switching Ornstein–Uhlenbeck processes, waveform and load-curve templates, steps, growth curves, intermittent counts, multivariate lead-lag mixing, covariates and missingness, followed by an augmentation cascade.
- **Real (~40% in the final stage):** GiftEvalPretrain, resolution-sampled with TSMixup, after a contamination filter that removes any dataset overlapping the GIFT-Eval test sets.

## Usage

```python
from huggingface_hub import snapshot_download
path = snapshot_download("Modularcomputing/Modular-TS-2.4M")
# path/export/model.safetensors + path/export/config.json
# Load with the bundled modular_ts package (see modular_ts/README.md),
# which wraps the Apache-2.0 timestral forecaster around these weights.
```

Files:
- `export/model.safetensors`, `export/config.json`: inference weights and config
- `modular_ts/`: training, synthetic data generation and evaluation code
- `results/all_results.csv`: per-configuration GIFT-Eval scores

## Licence

- **Weights:** CC BY-NC 4.0 (research and non-commercial use), because the real-data portion of training mixes datasets under several licences.
- **Code (`modular_ts/`):** Apache-2.0, including the adapted `timestral` components (see `modular_ts/NOTICE`).

## Limitations

- Trails TTM-R3 on point accuracy (MASE); weakest on some long-horizon, high-frequency energy and weather series.
- A small model: it captures common seasonal and trend structure well but can miss rare regime changes.

## Citation

```bibtex
@misc{modularts2026,
  title  = {Modular-TS-2.4M: a small time-series foundation model trained from scratch},
  author = {Akuegbo, Chima Emmanuel and Modular Computing},
  year   = {2026},
  url    = {https://huggingface.co/Modularcomputing/Modular-TS-2.4M}
}
```