OptimizerStudy
/

mars_520m_1

Model card Files Files and versions

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

Model Card

Source: https://arxiv.org/abs/2509.02046
Optimizer: mars
Model size: 520m
Data size: 10B

Best configuration

Hyperparameter	Value
beta1	`0.95`
beta2	`0.95`
epsilon	`1e-25`
gamma	`0.025`
learning_rate	`0.008`
max_grad_norm	`1`
min_lr_ratio	`0`
train_batch_size	`256`
warmup	`2000`
weight_decay	`0.1`

Downloads last month: 3

Safetensors

Model size

0.8B params

Tensor type

F32

·

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for OptimizerStudy/mars_520m_1

Fantastic Pretraining Optimizers and Where to Find Them

Paper • 2509.02046 • Published Sep 2, 2025 • 14