README.md

---
license: mit
library_name: transformers
pipeline_tag: time-series-forecasting
tags:
- time-series
- mixture-of-experts
- forecasting
- pytorch
- fft
model-index:
- name: SuperLinear
  results: []
---

# Super-Linear: A Mixture of Experts Time Series Forecasting Model

Super-Linear is a lightweight and scalable mixture-of-experts (MoE) model for general time series forecasting. It replaces deep architectures with simple frequency-specialized linear experts, trained on resampled data across multiple frequency regimes. A lightweight spectral gating mechanism dynamically selects relevant experts, enabling efficient, accurate forecasting.

The model was introduced in the paper [Super-Linear: A Lightweight Pretrained Mixture of Linear Experts for Time Series Forecasting](https://huggingface.co/papers/2509.15105).

## Model Architecture

The Super-Linear model consists of:

- **Sparse Mixture of Experts (MoE)**: Routes inputs to the top-k most relevant experts.
- **FFT-based Gating Network**: Uses frequency domain analysis to determine expert routing.
- **Frequency-specific Experts**: Pre-trained experts specialized for different temporal patterns.

## Key Features

- **Adaptive Expert Selection**: Dynamic routing based on input characteristics.
- **Frequency-aware Processing**: Leverages FFT analysis for intelligent expert selection.
- **Auto-regressive Capabilities**: Supports long-horizon forecasting.
- **Multi-scale Processing**: Handles various sequence lengths through resampling.

## Updates
- On 26/01/2026 a slight implementation modification was introduced, now inference is more than 10x faster! Making Super-Linear the fastest pre-trained forecaster!
  
## Usage

You can use the model via the `transformers` library. Ensure you have `trust_remote_code=True` set.

```python
import torch
import numpy as np
from transformers import AutoModelForCausalLM

model_path = "SequentialLearning/SuperLinear" 
model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True)

seq_len = 512
pred_len = 96

# Create sample data
freq = 1/24
amp = 1
ph = 0

t = torch.arange(0, seq_len + pred_len)
s = amp * torch.sin(2 * np.pi * freq * t + ph)
x = s[:seq_len].unsqueeze(0)  # Add batch dim

with torch.no_grad():
    # takes shapes (B, V, L) or (B, L)
    output = model(x, pred_len=pred_len, get_prob=True)
    preds = output.logits # Predicted values
    probs = output.attentions  # Expert probabilities stored here

expert_names = model.backbone.experts.keys()
```

## Configuration

Key parameters in `config.json`:

- `train_seq_len`: Training sequence length (default: 512)
- `train_pred_len`: Training prediction length (default: 96)
- `top_k_experts`: Number of experts to use (default: 12)
- `use_fft`: Whether to use FFT-based gating (default: True)
- `freq_experts`: Frequency-specific expert configuration
- `moe_temp`: Temperature for expert selection during inference (default: 1)

## Links

- **GitHub Repository**: [https://github.com/azencot-group/SuperLinear](https://github.com/azencot-group/SuperLinear)
- **Paper**: [https://arxiv.org/abs/2509.15105](https://arxiv.org/abs/2509.15105)

## Citation

If you use Super-Linear in your research, please cite:

```bibtex
@article{nochumsohn2025super,
  title={Super-Linear: A Lightweight Pretrained Mixture of Linear Experts for Time Series Forecasting},
  author={Nochumsohn, Liran and Marshanski, Raz and Zisling, Hedi and Azencot, Omri},
  journal={arXiv preprint arXiv:2509.15105},
  year={2025}
}
```

## License

This model is released under the MIT License.