README
🚀
AI & ML interests
None defined yet.
Recent Activity
Hoglet-33
updated a
Space about 7 hours ago
Post
2293
Pebble 10M and Pebble 10M Chat are now released!
Both models use our Mamba/Transformer 3:1 hybrid architecture and were pretrained on 25 billion tokens.
Pebble 10M Chat was additionally fine-tuned on 250 million tokens of Smol-SmolTalk to improve its conversational capabilities.
You can find them here:
- basically-ai/Pebble-10M
- basically-ai/Pebble-10M-Chat
We hope you enjoy using them. The rest of the Pebble family will be released soon.
Follow for more:
@Hoglet-33
basically-ai
Both models use our Mamba/Transformer 3:1 hybrid architecture and were pretrained on 25 billion tokens.
Pebble 10M Chat was additionally fine-tuned on 250 million tokens of Smol-SmolTalk to improve its conversational capabilities.
You can find them here:
- basically-ai/Pebble-10M
- basically-ai/Pebble-10M-Chat
We hope you enjoy using them. The rest of the Pebble family will be released soon.
Follow for more:
@Hoglet-33
- 1 reply
Post
2998
We are announcing the first generation of the Pebble model family!
These are the models we are releasing:
- Pebble 10M
- Pebble 25M
- Pebble 50M
Each model will use a Mamba-Transformer 3:1 hybrid architecture and will be pretrained on 25 billion tokens before IFT and SFT.
Depending on development time and resources, we may also release:
- Pebble 5M
- Pebble 75M
- Pebble 1M (possibly)
We hope you're excited and enjoy the models!
Follow for more:
@Hoglet-33
basically-ai
These are the models we are releasing:
- Pebble 10M
- Pebble 25M
- Pebble 50M
Each model will use a Mamba-Transformer 3:1 hybrid architecture and will be pretrained on 25 billion tokens before IFT and SFT.
Depending on development time and resources, we may also release:
- Pebble 5M
- Pebble 75M
- Pebble 1M (possibly)
We hope you're excited and enjoy the models!
Follow for more:
@Hoglet-33
- 7 replies