DevOfKia

I didn't know it was based off of ornith, that's pretty neat!
You can also make some more datasets. Every dataset you've made seems to be perfect quality somehow, and all of them do well.

While I'm here, how do you generate your datasets?
I thought it'd be cool to do something similar to what you do with grug for a custom finetune, and I have no idea how to generate a dataset anywhere near your 1m for grug v1.1.

Do you open source them (or the big one eventually?)

You should grug-ify ornith 1.5 35b a3b.
Ornith is cranking out bangers, there would be huge demand for a MOE v1.1 with Ornith level performance, and maybe a working MTP head.
You could also do a smaller model for the GPU weak, like ling or lfm2.5, that doesn't move at a snail's pace (they both are MOE, and overthink)