Safetensors
GGUF
English
Spanish
code
qwen2
code-generation
android
kotlin
java
jetpack-compose
qwen2.5
unsloth
ollama
3b
conversational
Instructions to use LaboAI/LaboAI-0.3.3-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use LaboAI/LaboAI-0.3.3-3B with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf LaboAI/LaboAI-0.3.3-3B:Q4_K_M # Run inference directly in the terminal: llama cli -hf LaboAI/LaboAI-0.3.3-3B:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf LaboAI/LaboAI-0.3.3-3B:Q4_K_M # Run inference directly in the terminal: llama cli -hf LaboAI/LaboAI-0.3.3-3B:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf LaboAI/LaboAI-0.3.3-3B:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf LaboAI/LaboAI-0.3.3-3B:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf LaboAI/LaboAI-0.3.3-3B:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf LaboAI/LaboAI-0.3.3-3B:Q4_K_M
Use Docker
docker model run hf.co/LaboAI/LaboAI-0.3.3-3B:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use LaboAI/LaboAI-0.3.3-3B with Ollama:
ollama run hf.co/LaboAI/LaboAI-0.3.3-3B:Q4_K_M
- Unsloth Desktop
- Pi
How to use LaboAI/LaboAI-0.3.3-3B with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf LaboAI/LaboAI-0.3.3-3B:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "LaboAI/LaboAI-0.3.3-3B:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use LaboAI/LaboAI-0.3.3-3B with Docker Model Runner:
docker model run hf.co/LaboAI/LaboAI-0.3.3-3B:Q4_K_M
- Lemonade
How to use LaboAI/LaboAI-0.3.3-3B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull LaboAI/LaboAI-0.3.3-3B:Q4_K_M
Run and chat with the model
lemonade run user.LaboAI-0.3.3-3B-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use LaboAI/LaboAI-0.3.3-3B with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf LaboAI/LaboAI-0.3.3-3B:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default LaboAI/LaboAI-0.3.3-3B:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use LaboAI/LaboAI-0.3.3-3B with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf LaboAI/LaboAI-0.3.3-3B:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "LaboAI/LaboAI-0.3.3-3B:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Update README.md
Browse files
README.md
CHANGED
|
@@ -14,7 +14,6 @@ tags:
|
|
| 14 |
- gguf
|
| 15 |
- ollama
|
| 16 |
- 3b
|
| 17 |
-
- function-calling
|
| 18 |
license: apache-2.0
|
| 19 |
base_model: Qwen/Qwen2.5-3B-Instruct
|
| 20 |
datasets:
|
|
@@ -26,43 +25,98 @@ datasets:
|
|
| 26 |
|
| 27 |
# 🤖 LaboAI-0.3.3-3B
|
| 28 |
|
| 29 |
-
This is a
|
| 30 |
|
| 31 |
-
|
| 32 |
|
| 33 |
## 📋 Model Details
|
| 34 |
|
| 35 |
- **Developed by:** Mmxa
|
| 36 |
- **Organization:** LaboAI
|
| 37 |
-
- **Model type:** Causal Language Model (Code Generation
|
| 38 |
-
- **Languages:** Kotlin, Java, English, Spanish
|
| 39 |
- **License:** Apache 2.0 (inherited from Qwen2.5)
|
| 40 |
- **Base model:** [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct)
|
| 41 |
|
| 42 |
## 🚀 Uses
|
| 43 |
|
| 44 |
### Direct Use
|
| 45 |
-
-
|
| 46 |
-
-
|
| 47 |
-
-
|
|
|
|
|
|
|
| 48 |
|
| 49 |
### Ecosystem Use (Recommended)
|
| 50 |
-
This model
|
| 51 |
|
| 52 |
### Out-of-Scope Uses
|
| 53 |
-
- It
|
| 54 |
-
-
|
|
|
|
| 55 |
|
| 56 |
## ⚠️ Limitations and Risks
|
| 57 |
-
- **
|
| 58 |
-
- **
|
| 59 |
-
- **
|
| 60 |
|
| 61 |
-
##
|
| 62 |
|
| 63 |
-
This repository includes both the original format (`safetensors`) and the quantized format (`GGUF` Q4_K_M).
|
| 64 |
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
``
|
| 68 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
- gguf
|
| 15 |
- ollama
|
| 16 |
- 3b
|
|
|
|
| 17 |
license: apache-2.0
|
| 18 |
base_model: Qwen/Qwen2.5-3B-Instruct
|
| 19 |
datasets:
|
|
|
|
| 25 |
|
| 26 |
# 🤖 LaboAI-0.3.3-3B
|
| 27 |
|
| 28 |
+
This is a mid-sized language model (3B parameters) fine-tuned specifically for generating, understanding, and debugging **Kotlin** code and **Android** development (with a strong emphasis on Jetpack Compose, Coroutines, and modern architectures).
|
| 29 |
|
| 30 |
+
This version (0.3.3) shares the same training recipe as the 1.5B variant but leverages the increased capacity of the 3B base model for superior reasoning, better handling of complex code structures, and improved generalization across diverse Android development scenarios. It is optimized using **QLoRA (4-bit)** to run efficiently on consumer GPUs with 6-8GB VRAM.
|
| 31 |
|
| 32 |
## 📋 Model Details
|
| 33 |
|
| 34 |
- **Developed by:** Mmxa
|
| 35 |
- **Organization:** LaboAI
|
| 36 |
+
- **Model type:** Causal Language Model (Code Generation)
|
| 37 |
+
- **Languages:** Kotlin, Java, English, Spanish (instructions)
|
| 38 |
- **License:** Apache 2.0 (inherited from Qwen2.5)
|
| 39 |
- **Base model:** [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct)
|
| 40 |
|
| 41 |
## 🚀 Uses
|
| 42 |
|
| 43 |
### Direct Use
|
| 44 |
+
- Generating boilerplate for Activities, Fragments, ViewModels, and Repositories in Kotlin.
|
| 45 |
+
- Creating modern UI components with **Jetpack Compose**.
|
| 46 |
+
- Debugging compilation errors or logic flaws in Android code snippets.
|
| 47 |
+
- Translating legacy Java logic into modern, idiomatic Kotlin.
|
| 48 |
+
- Understanding and explaining complex Android architecture patterns (MVVM, MVI, Clean Architecture).
|
| 49 |
|
| 50 |
### Ecosystem Use (Recommended)
|
| 51 |
+
This model shines when used as a local coding assistant via **Ollama** and the **Continue** extension in VS Code. This guarantees complete privacy (your code never leaves your machine) and low latency.
|
| 52 |
|
| 53 |
### Out-of-Scope Uses
|
| 54 |
+
- It is not optimized for general chat, creative writing, or complex mathematical reasoning.
|
| 55 |
+
- It should not be used to generate malicious code or exploits.
|
| 56 |
+
- All generated code must be reviewed by a human developer before being merged into a main branch.
|
| 57 |
|
| 58 |
## ⚠️ Limitations and Risks
|
| 59 |
+
- **API Hallucinations:** In rare cases, it might suggest deprecated Android APIs instead of modern alternatives.
|
| 60 |
+
- **Context Window:** Optimized for 2048 tokens. It is not suitable for analyzing massive, multi-thousand-line codebase files all at once.
|
| 61 |
+
- **Dependencies:** It does not have real-time knowledge of the latest Android library updates.
|
| 62 |
|
| 63 |
+
## 💻 How to Get Started (Local Setup)
|
| 64 |
|
| 65 |
+
This repository includes both the original format (`safetensors`) and the quantized format (`GGUF` Q4_K_M). To use it on your PC with a 6-8GB VRAM GPU:
|
| 66 |
|
| 67 |
+
1. Install [Ollama](https://ollama.com/).
|
| 68 |
+
2. Download the `.gguf` file from this repository (e.g., `LaboAI-0.3.3-3B-Q4_K_M.gguf`).
|
| 69 |
+
3. Create a file named `Modelfile` in the same folder with the following content:
|
| 70 |
+
```text
|
| 71 |
+
FROM ./LaboAI-0.3.3-3B-Q4_K_M.gguf
|
| 72 |
+
TEMPLATE """{{- if .Messages }}
|
| 73 |
+
{{- if or .System .Tools }}<|im_start|>system
|
| 74 |
+
{{- if .System }}
|
| 75 |
+
{{ .System }}
|
| 76 |
+
{{- end }}
|
| 77 |
+
{{- if .Tools }}
|
| 78 |
+
|
| 79 |
+
# Tools
|
| 80 |
+
|
| 81 |
+
You may call one or more functions to assist with the user query.
|
| 82 |
+
|
| 83 |
+
You are provided with function signatures within <tools></tools> XML tags:
|
| 84 |
+
<tools>
|
| 85 |
+
{{- range .Tools }}
|
| 86 |
+
{"type": "function", "function": {{ .Function }}}
|
| 87 |
+
{{- end }}
|
| 88 |
+
</tools>
|
| 89 |
+
|
| 90 |
+
For each function call, return a json object with function name and arguments within XML tags:
|
| 91 |
+
|
| 92 |
+
{{- end }}<|im_end|>
|
| 93 |
+
{{ end }}
|
| 94 |
+
{{- range $i, $_ := .Messages }}
|
| 95 |
+
{{- $last := eq (len (slice $.Messages $i)) 1 -}}
|
| 96 |
+
{{- if eq .Role "user" }}<|im_start|>user
|
| 97 |
+
{{ .Content }}<|im_end|>
|
| 98 |
+
{{ else if eq .Role "assistant" }}<|im_start|>assistant
|
| 99 |
+
{{ if .Content }}{{ .Content }}
|
| 100 |
+
{{- else if .ToolCalls }}
|
| 101 |
+
{{- end }}{{ if not $last }}<|im_end|>
|
| 102 |
+
{{ end }}
|
| 103 |
+
{{- else if eq .Role "tool" }}<|im_start|>user
|
| 104 |
+
<tool_response>
|
| 105 |
+
{{ .Content }}
|
| 106 |
+
</tool_response><|im_end|>
|
| 107 |
+
{{ end }}
|
| 108 |
+
{{- if and (ne .Role "assistant") $last }}<|im_start|>assistant
|
| 109 |
+
{{ end }}
|
| 110 |
+
{{- end }}
|
| 111 |
+
{{- else }}
|
| 112 |
+
{{- if .System }}<|im_start|>system
|
| 113 |
+
{{ .System }}<|im_end|>
|
| 114 |
+
{{ end }}{{ if .Prompt }}<|im_start|>user
|
| 115 |
+
{{ .Prompt }}<|im_end|>
|
| 116 |
+
{{ end }}<|im_start|>assistant
|
| 117 |
+
{{ end }}{{ .Response }}{{ if .Response }}<|im_end|>{{ end }}"""
|
| 118 |
+
PARAMETER stop "<|im_end|>"
|
| 119 |
+
PARAMETER stop "<|endoftext|>"
|
| 120 |
+
PARAMETER temperature 0.7
|
| 121 |
+
PARAMETER min_p 0.1
|
| 122 |
+
SYSTEM """You are LaboAI, a helpful assistant specialized in Kotlin and Android development."""
|