| library_name: mlx | |
| pipeline_tag: text-generation | |
| tags: | |
| - music | |
| - piano-roll | |
| - causal-lm | |
| - qwen3 | |
| - safetensors | |
| # Musicroll-50M | |
| Musicroll-50M is a small Qwen3-style causal language model over a 259-token | |
| byte-level music vocabulary. It was used as the MusicRoll negative-control model | |
| in the replication artifact for "Many Next-Token Predictors are In-Context | |
| Learners". | |
| The checkpoint is provided as `model.safetensors` with a compact tokenizer: | |
| - vocabulary size: 259 | |
| - hidden size: 512 | |
| - layers: 16 | |
| - attention heads: 4 | |
| - key/value heads: 2 | |
| - context length: 4096 | |
| - BOS/EOS/PAD token ids: 256/257/258 | |
| The bitstring replication harness encodes tasks as piano-roll-like text and | |
| loads this checkpoint through the local MLX model loader. | |
| ## Intended Use | |
| This upload is primarily for reproducing the MusicRoll negative-control runs in | |
| the ICLManyReplication artifact. General music-generation behavior has not been | |
| characterized here. | |