metadata
library_name: mlx
pipeline_tag: text-generation
tags:
- music
- piano-roll
- causal-lm
- qwen3
- safetensors
Musicroll-50M
Musicroll-50M is a small Qwen3-style causal language model over a 259-token byte-level music vocabulary. It was used as the MusicRoll negative-control model in the replication artifact for "Many Next-Token Predictors are In-Context Learners".
The checkpoint is provided as model.safetensors with a compact tokenizer:
- vocabulary size: 259
- hidden size: 512
- layers: 16
- attention heads: 4
- key/value heads: 2
- context length: 4096
- BOS/EOS/PAD token ids: 256/257/258
The bitstring replication harness encodes tasks as piano-roll-like text and loads this checkpoint through the local MLX model loader.
Intended Use
This upload is primarily for reproducing the MusicRoll negative-control runs in the ICLManyReplication artifact. General music-generation behavior has not been characterized here.