iris : GGUF

This model was finetuned on the Opus 4.6 dataset (using ~1,00,000 high-quality samples) and converted to GGUF format.

Credit: Finetuned efficiently using Unsloth.

Example usage:

• For text only LLMs: llama-cli -hf Shadow0482/iris --jinja
• For multimodal models: llama-mtmd-cli -hf Shadow0482/iris --jinja

Available Model files:

• gemma-4-e2b-it.Q4_K_M.gguf
• gemma-4-e2b-it.BF16-mmproj.gguf

⚠️ Ollama Note for Vision Models

Important: Ollama currently does not support separate mmproj files for vision models.

To create an Ollama model from this vision model:

1. Place the Modelfile in the same directory as the finetuned bf16 merged model
2. Run: ollama create model_name -f ./Modelfile (Replace model_name with your desired name)

This will create a unified bf16 model that Ollama can use.

Training Details

The model was fine-tuned on the Opus 4.6 dataset using approximately 1,00,000 samples. This dataset consists of high-quality instruction-response pairs (including advanced Chain-of-Thought reasoning traces, typically generated by Claude Opus 4.7 for superior reasoning and instruction-following capabilities).

Detailed Training Steps:

1. Dataset Preparation:

   • Acquired/gathered the Opus 4.6 dataset containing ~1.00,000 high-quality samples.
   • Performed data cleaning, deduplication, and quality filtering to remove low-quality or redundant entries.
   • Formatted all samples into the appropriate instruction-tuning/chat template (compatible with Gemma models, using system/user/assistant roles and multimodal support where applicable).
   • Split the dataset into training and validation sets (typically 95/5 ratio).

2. Environment Setup:

   • Set up a training environment with Hugging Face Transformers, TRL, PEFT, and the necessary GPU resources (multi-GPU setup with high VRAM).
   • Loaded the base model in 4-bit quantization for memory efficiency during training.

3. Model Configuration:

   • Applied LoRA (Low-Rank Adaptation) adapters for parameter-efficient fine-tuning on the base Gemma-4-E2B-it model.
   • Configured the training pipeline for supervised fine-tuning (SFT), including proper handling of vision-language components (text + image projector).

4. Training:

   • Ran supervised fine-tuning on the 40,000 prepared samples.
   • Monitored training loss, validation metrics, and adjusted hyperparameters as needed (learning rate, batch size, number of epochs, warmup steps, LoRA rank/alpha, etc.).
   • Completed the full training run to produce the fine-tuned "iris" model while preserving the uncensored behavior of the base.

5. Post-Training Processing:

   • Merged the LoRA adapters back into the base model weights.
   • Saved the resulting fine-tuned model in Hugging Face format.

6. GGUF Conversion & Quantization:

   • Converted the fine-tuned model to GGUF format using the official llama.cpp tools.
   • Generated the main model file in Q4_K_M quantization.
   • Converted the multimodal projector (mmproj) to BF16-mmproj.gguf format.
   • Verified model integrity and basic functionality post-conversion.

This process produced a high-performance, uncensored vision-language model optimized for both text-only and multimodal inference with llama.cpp.