int4 convrot text encoder?
Will there be int4 convrot text encoder?
Second this - nvfp4 is for blackwell only, isn't it? Would it be possible to make 4-bit for those with Ada?
yes please! a pruned int4 would get it to a more usable point for those in Ada
also, I wonder how much can text enconder and video VAE be quantized without messing the quality too much
The nvfp4 is not for Blackwell only, you can just use it.
Kijai, Are you willing to build a Lightx lora for the Minimax H3 model?
Kijai, Are you willing to build a Lightx lora for the Minimax H3 model?
Everybody think about distillation of this BigDaddy)
Everything in its own time
Stay tuned and F5 huggingface
P.S. I think distillation may take 3~5 days
Will there be int4 convrot text encoder?
side note for you guys, my repo contains some gguf encoders, they arent much smaller than the nvfp4 encoder in this repo but it should help A LITTLE i guess. also 2 quants of the model and a mixed precision gguf of a q2 with q6 layering i havent tested yet
https://huggingface.co/realrebelai/MiniMax-H3_GGUFs/tree/main
The nvfp4 is not for Blackwell only, you can just use it.
"NVFP4 is specifically tied to NVIDIA's hardware-accelerated 4-bit floating-point format, requiring dedicated 5th-generation Tensor Cores found in Blackwell and newer architectures for native execution. While older or non-Blackwell GPUs can theoretically parse or upcast these weights through software emulation, they lack native silicon support, leading to severe performance and efficiency penalties." According to Google.
The nvfp4 is not for Blackwell only, you can just use it.
"NVFP4 is specifically tied to NVIDIA's hardware-accelerated 4-bit floating-point format, requiring dedicated 5th-generation Tensor Cores found in Blackwell and newer architectures for native execution. While older or non-Blackwell GPUs can theoretically parse or upcast these weights through software emulation, they lack native silicon support, leading to severe performance and efficiency penalties." According to Google.
It's more nuanced than that, in this case it's not really different than using something like GGUF, which also has to be upcast to be used, it's just used as storage format here, there's nothing hardware specific about that.