view post Post 2913 We are announcing the first generation of the Pebble model family!These are the models we are releasing: - Pebble 10M - Pebble 25M - Pebble 50MEach model will use a Mamba-Transformer 3:1 hybrid architecture and will be pretrained on 25 billion tokens before IFT and SFT.Depending on development time and resources, we may also release: - Pebble 5M - Pebble 75M - Pebble 1M (possibly)We hope you're excited and enjoy the models!Follow for more:@Hoglet-33 basically-ai See translation 7 replies · 🔥 17 17 👍 4 4 ❤️ 2 2 + Reply