Hoglet-33 commited on
Commit
eebf5da
·
verified ·
1 Parent(s): 0db4edf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -15,10 +15,10 @@ tags:
15
  - hybrid
16
  ---
17
 
18
- # Pebble-10M-Base
19
  ![Banner](banner.png)
20
 
21
- Pebble-10M-Base is a compact, hybrid autoregressive language model. It combines the efficiency of state-space models with the proven performance of attention layers, optimized using a custom Muon + AdamW optimizer split.
22
 
23
  ## Model Details
24
 
@@ -48,7 +48,7 @@ The model was trained on a 25B token subset of the following datasets:
48
 
49
  ## Benchmarks
50
 
51
- Pebble-10M-Base performs above random chance on several commonsense and arithmetic benchmarks.
52
 
53
  | Benchmark | Accuracy | Random Baseline |
54
  |-----------|----------|------------------|
@@ -84,7 +84,7 @@ Here is a simple Python script to load the model and generate text interactively
84
  import torch
85
  from transformers import AutoModelForCausalLM, AutoTokenizer
86
 
87
- MODEL_ID = "basically-ai/Pebble-10M-Base"
88
 
89
  def main():
90
  print("Loading Pebble 10M...")
 
15
  - hybrid
16
  ---
17
 
18
+ # Pebble-10M
19
  ![Banner](banner.png)
20
 
21
+ Pebble-10M is a compact, hybrid autoregressive language model. It combines the efficiency of state-space models with the proven performance of attention layers, optimized using a custom Muon + AdamW optimizer split.
22
 
23
  ## Model Details
24
 
 
48
 
49
  ## Benchmarks
50
 
51
+ Pebble-10M performs above random chance on several commonsense and arithmetic benchmarks.
52
 
53
  | Benchmark | Accuracy | Random Baseline |
54
  |-----------|----------|------------------|
 
84
  import torch
85
  from transformers import AutoModelForCausalLM, AutoTokenizer
86
 
87
+ MODEL_ID = "basically-ai/Pebble-10M"
88
 
89
  def main():
90
  print("Loading Pebble 10M...")