Timon

At 4 bit or lower use IQ-type quant. The math is more advanced and quantization error is lower. You can double the context with -ctk q8_0 -ctv q8_0for virtually no loss of quality and speed.

New activity in unsloth/gemma-4-26B-A4B-it-GGUF 6 months ago

Gemma 4 seems to work best with high temperature for coding

👍 1

#21 opened 6 months ago by

Reverger

New activity in mistralai/Mistral-Small-4-119B-2603 6 months ago

Recommended sampler?

#4 opened 6 months ago by

mratsim

New activity in bartowski/Qwen_Qwen3.5-397B-A17B-GGUF 7 months ago

Older quants get in the way

#1 opened 7 months ago by

KeyboardMasher

New activity in ggml-org/gpt-oss-120b-GGUF about 1 year ago

Error with built-in Web UI

#3 opened about 1 year ago by

KeyboardMasher

New activity in legraphista/Llama-3.2-3B-Instruct-IMat-GGUF about 1 year ago

Thanks for IQ4_NL

❤️ 1

#1 opened about 1 year ago by

KeyboardMasher

New activity in bartowski/Qwen_Qwen3-30B-A3B-GGUF over 1 year ago

128k Context GGUF, please?

#2 opened over 1 year ago by

MikeNate

Timon

AI & ML interests

Recent Activity

Organizations

KeyboardMasher's activity

KeyboardMasher/Step-3.5-Flash-MTP-GGUF-backup

KeyboardMasher/Step-3.5-Flash-MTP-GGUF-backup

Any idea when is the PR for llama.cpp comming?

Update metadata (tokenizer.ggml.model)

Poor performance

Temperature is last for Llama.cpp

Correct Samplers Order?

mmproj files are missing

version for 16 GB VRAM

Feedback

token_embd.weight is quantized too much

Gemma 4 seems to work best with high temperature for coding

Recommended sampler?

Older quants get in the way

Error with built-in Web UI

Thanks for IQ4_NL

128k Context GGUF, please?