Quickly tested with vLLM. Not fully compatible yet.
AI & ML interests
None defined yet.
Recent Activity
View all activity
Verified models.
Verified models.
Verified models. All compatible with vLLM for very fast inference. Use the 3.1 models as they are more recent.
-
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-32g-4096-gptq
73B • Updated -
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-128g-4096-gptq
73B • Updated -
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-128g-2048-gptq
73B • Updated -
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-64g-4096-gptq
73B • Updated
-
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-8bit
Text Generation • 4B • Updated • 3 -
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-3bit
Text Generation • 4B • Updated • 3 -
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-4bit
Text Generation • 4B • Updated • 15 -
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-2bit
Text Generation • 4B • Updated • 3
-
kaitchup/DeepSeek-R1-Distill-Llama-8B-AutoRound-GPTQ-4bit
Text Generation • 8B • Updated • 391 • 1 -
kaitchup/DeepSeek-R1-Distill-Qwen-14B-AutoRound-GPTQ-4bit
Text Generation • 15B • Updated • 43 • 7 -
kaitchup/DeepSeek-R1-Distill-Qwen-7B-AutoRound-GPTQ-4bit
Text Generation • 8B • Updated • 2
-
kaitchup/Falcon3-10B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 10B • Updated • 3 -
kaitchup/Falcon3-10B-Base-AutoRound-GPTQ-4bit
Text Generation • 10B • Updated -
kaitchup/Falcon3-7B-Base-AutoRound-GPTQ-4bit
Text Generation • 7B • Updated -
kaitchup/Falcon3-7B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 7B • Updated • 1
-
kaitchup/Qwen2.5-1.5B-AutoRound-GPTQ-asym-4bit
Text Generation • 2B • Updated • 1 -
kaitchup/Qwen2.5-7B-AutoRound-GPTQ-asym-4bit
Text Generation • 8B • Updated • 1 -
kaitchup/Qwen2.5-1.5B-Instruct-AutoRound-GPTQ-asym-4bit
Text Generation • 2B • Updated • 4 -
kaitchup/Qwen2.5-7B-Instruct-AutoRound-GPTQ-asym-4bit
Text Generation • 8B • Updated • 4
Machine translation adapters for Llama 2 7B.
Quantized and fine-tuned versions of the Yi models
A collection of 7B models made with mergekit.
-
kaitchup/Phi-3-mini-4k-instruct-gptq-4bit
Text Generation • 4B • Updated • 877k • 2 -
kaitchup/Phi-3-medium-128k-instruct-awq-4bit
Text Generation • 14B • Updated -
kaitchup/Phi-3-mini-4k-instruct-bnb-4bit
Text Generation • 4B • Updated • 1 -
kaitchup/Phi-3-medium-4k-instruct-awq-4bit
Text Generation • 14B • Updated
-
kaitchup/Meta-Llama-3.1-8B-Instruct-autoround-gptq-4bit-sym
Text Generation • 8B • Updated • 1 • 1 -
kaitchup/Meta-Llama-3.1-8B-Instruct-awq-4bit
Text Generation • 8B • Updated • 26 • 1 -
kaitchup/Meta-Llama-3.1-8B-awq-4bit
Text Generation • 8B • Updated -
kaitchup/Meta-Llama-3.1-8B-Instruct-gptq-4bit
Text Generation • 8B • Updated • 2
-
kaitchup/Mistral-NeMo-Minitron-8B-Base-Minivoc-32k-v0.1a
Text Generation • 8B • Updated -
kaitchup/Llama-3.1-8B-Minivoc-32k-v0.1a
Text Generation • 7B • Updated -
kaitchup/Qwen2-1.5B-Minivoc-32k-v0.1a
Text Generation • 1B • Updated • 1 • 2 -
kaitchup/Qwen2.5-1.5B-Minivoc-32k-v0.1a-AutoRound-GPTQ-asym-4bit
Text Generation • 1B • Updated • 1
Verified models. Compatible with vLLM.
Verified models.
Verified models. Tested with vLLM.
LLMs quantized with LLM Compressor to NVFP4. Check the JSON files in the model directories for more information.
-
kaitchup/QwQ-32B-bnb-4bit
Text Generation • 34B • Updated • 3 • 1 -
kaitchup/QwQ-32B-AutoRoundGPTQ-8bit
Text Generation • 33B • Updated • 103 • 2 -
kaitchup/QwQ-32B-AutoRoundGPTQ-4bit
Text Generation • 33B • Updated • 1 • 1 -
kaitchup/QwQ-32B-AutoRoundGPTQ-3bit
Text Generation • 31B • Updated • 3
-
kaitchup/OLMo-2-1124-7B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 7B • Updated • 1 -
kaitchup/Llama-3.1-Tulu-3-70B-AutoRound-GPTQ-4bit
Text Generation • 71B • Updated -
kaitchup/Llama-3.1-Tulu-3-8B-AutoRound-GPTQ-4bit
Text Generation • 8B • Updated -
kaitchup/OLMo-2-1124-13B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 14B • Updated
Some language pairs of OPUS formatted to have source and target sentences as single sequences. Intended to facilitate fine-tuning of causal LLMs.
Llama 2 7B, 13B, Llama 3 8B, and Mistral 7B quantized with GPTQ in 2-bit, 3-bit, 4-bit and 8-bit with GPTQ.
-
kaitchup/Llama-2-7b-hf-gptq-8bit
Text Generation • 7B • Updated • 4 -
kaitchup/Llama-2-7b-hf-gptq-3bit
Text Generation • 6B • Updated • 858 • 1 -
kaitchup/Llama-2-7b-hf-gptq-2bit
Text Generation • 7B • Updated • 862 -
kaitchup/Llama-2-7b-hf-gptq-4bit
Text Generation • 7B • Updated • 1
Contaminated Mistral 7B and TinyLlama adapters, and the datasets used for contamination.
-
kaitchup/Mistral-NeMo-Minitron-8B-Base-AutoRound-GPTQ-sym-4bit
Text Generation • 8B • Updated • 1 -
kaitchup/Mistral-NeMo-Minitron-8B-Base-AutoRound-GPTQ-asym-4bit
Text Generation • 8B • Updated -
kaitchup/Mistral-Nemo-Base-2407-AutoRound-GPTQ-asym-4bit
Text Generation • 12B • Updated -
kaitchup/Llama-3.1-Minitron-4B-Width-Base-AutoRound-GPTQ-asym-4bit
Text Generation • 5B • Updated
Quickly tested with vLLM. Not fully compatible yet.
Verified models. Compatible with vLLM.
Verified models.
Verified models.
Verified models.
Verified models. Tested with vLLM.
Verified models. All compatible with vLLM for very fast inference. Use the 3.1 models as they are more recent.
LLMs quantized with LLM Compressor to NVFP4. Check the JSON files in the model directories for more information.
-
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-32g-4096-gptq
73B • Updated -
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-128g-4096-gptq
73B • Updated -
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-128g-2048-gptq
73B • Updated -
kaitchup/Qwen2.5-72B-Instruct-autoround-2bit-64g-4096-gptq
73B • Updated
-
kaitchup/QwQ-32B-bnb-4bit
Text Generation • 34B • Updated • 3 • 1 -
kaitchup/QwQ-32B-AutoRoundGPTQ-8bit
Text Generation • 33B • Updated • 103 • 2 -
kaitchup/QwQ-32B-AutoRoundGPTQ-4bit
Text Generation • 33B • Updated • 1 • 1 -
kaitchup/QwQ-32B-AutoRoundGPTQ-3bit
Text Generation • 31B • Updated • 3
-
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-8bit
Text Generation • 4B • Updated • 3 -
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-3bit
Text Generation • 4B • Updated • 3 -
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-4bit
Text Generation • 4B • Updated • 15 -
kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-2bit
Text Generation • 4B • Updated • 3
-
kaitchup/DeepSeek-R1-Distill-Llama-8B-AutoRound-GPTQ-4bit
Text Generation • 8B • Updated • 391 • 1 -
kaitchup/DeepSeek-R1-Distill-Qwen-14B-AutoRound-GPTQ-4bit
Text Generation • 15B • Updated • 43 • 7 -
kaitchup/DeepSeek-R1-Distill-Qwen-7B-AutoRound-GPTQ-4bit
Text Generation • 8B • Updated • 2
-
kaitchup/Falcon3-10B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 10B • Updated • 3 -
kaitchup/Falcon3-10B-Base-AutoRound-GPTQ-4bit
Text Generation • 10B • Updated -
kaitchup/Falcon3-7B-Base-AutoRound-GPTQ-4bit
Text Generation • 7B • Updated -
kaitchup/Falcon3-7B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 7B • Updated • 1
-
kaitchup/OLMo-2-1124-7B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 7B • Updated • 1 -
kaitchup/Llama-3.1-Tulu-3-70B-AutoRound-GPTQ-4bit
Text Generation • 71B • Updated -
kaitchup/Llama-3.1-Tulu-3-8B-AutoRound-GPTQ-4bit
Text Generation • 8B • Updated -
kaitchup/OLMo-2-1124-13B-Instruct-AutoRound-GPTQ-4bit
Text Generation • 14B • Updated
-
kaitchup/Qwen2.5-1.5B-AutoRound-GPTQ-asym-4bit
Text Generation • 2B • Updated • 1 -
kaitchup/Qwen2.5-7B-AutoRound-GPTQ-asym-4bit
Text Generation • 8B • Updated • 1 -
kaitchup/Qwen2.5-1.5B-Instruct-AutoRound-GPTQ-asym-4bit
Text Generation • 2B • Updated • 4 -
kaitchup/Qwen2.5-7B-Instruct-AutoRound-GPTQ-asym-4bit
Text Generation • 8B • Updated • 4
Some language pairs of OPUS formatted to have source and target sentences as single sequences. Intended to facilitate fine-tuning of causal LLMs.
Machine translation adapters for Llama 2 7B.
Llama 2 7B, 13B, Llama 3 8B, and Mistral 7B quantized with GPTQ in 2-bit, 3-bit, 4-bit and 8-bit with GPTQ.
-
kaitchup/Llama-2-7b-hf-gptq-8bit
Text Generation • 7B • Updated • 4 -
kaitchup/Llama-2-7b-hf-gptq-3bit
Text Generation • 6B • Updated • 858 • 1 -
kaitchup/Llama-2-7b-hf-gptq-2bit
Text Generation • 7B • Updated • 862 -
kaitchup/Llama-2-7b-hf-gptq-4bit
Text Generation • 7B • Updated • 1
Quantized and fine-tuned versions of the Yi models
Contaminated Mistral 7B and TinyLlama adapters, and the datasets used for contamination.
A collection of 7B models made with mergekit.
-
kaitchup/Phi-3-mini-4k-instruct-gptq-4bit
Text Generation • 4B • Updated • 877k • 2 -
kaitchup/Phi-3-medium-128k-instruct-awq-4bit
Text Generation • 14B • Updated -
kaitchup/Phi-3-mini-4k-instruct-bnb-4bit
Text Generation • 4B • Updated • 1 -
kaitchup/Phi-3-medium-4k-instruct-awq-4bit
Text Generation • 14B • Updated
-
kaitchup/Meta-Llama-3.1-8B-Instruct-autoround-gptq-4bit-sym
Text Generation • 8B • Updated • 1 • 1 -
kaitchup/Meta-Llama-3.1-8B-Instruct-awq-4bit
Text Generation • 8B • Updated • 26 • 1 -
kaitchup/Meta-Llama-3.1-8B-awq-4bit
Text Generation • 8B • Updated -
kaitchup/Meta-Llama-3.1-8B-Instruct-gptq-4bit
Text Generation • 8B • Updated • 2
-
kaitchup/Mistral-NeMo-Minitron-8B-Base-AutoRound-GPTQ-sym-4bit
Text Generation • 8B • Updated • 1 -
kaitchup/Mistral-NeMo-Minitron-8B-Base-AutoRound-GPTQ-asym-4bit
Text Generation • 8B • Updated -
kaitchup/Mistral-Nemo-Base-2407-AutoRound-GPTQ-asym-4bit
Text Generation • 12B • Updated -
kaitchup/Llama-3.1-Minitron-4B-Width-Base-AutoRound-GPTQ-asym-4bit
Text Generation • 5B • Updated
-
kaitchup/Mistral-NeMo-Minitron-8B-Base-Minivoc-32k-v0.1a
Text Generation • 8B • Updated -
kaitchup/Llama-3.1-8B-Minivoc-32k-v0.1a
Text Generation • 7B • Updated -
kaitchup/Qwen2-1.5B-Minivoc-32k-v0.1a
Text Generation • 1B • Updated • 1 • 2 -
kaitchup/Qwen2.5-1.5B-Minivoc-32k-v0.1a-AutoRound-GPTQ-asym-4bit
Text Generation • 1B • Updated • 1