Instructions to use shibatch/tiny1m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use shibatch/tiny1m with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("shibatch/tiny1m", dtype="auto") - llama-cpp-python
How to use shibatch/tiny1m with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="shibatch/tiny1m", filename="tiny1m.BF16.gguf", )
output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use shibatch/tiny1m with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf shibatch/tiny1m:Q4_K_M # Run inference directly in the terminal: llama-cli -hf shibatch/tiny1m:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf shibatch/tiny1m:Q4_K_M # Run inference directly in the terminal: llama-cli -hf shibatch/tiny1m:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf shibatch/tiny1m:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf shibatch/tiny1m:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf shibatch/tiny1m:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf shibatch/tiny1m:Q4_K_M
Use Docker
docker model run hf.co/shibatch/tiny1m:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use shibatch/tiny1m with Ollama:
ollama run hf.co/shibatch/tiny1m:Q4_K_M
- Unsloth Studio
How to use shibatch/tiny1m with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for shibatch/tiny1m to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for shibatch/tiny1m to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for shibatch/tiny1m to start chatting
- Docker Model Runner
How to use shibatch/tiny1m with Docker Model Runner:
docker model run hf.co/shibatch/tiny1m:Q4_K_M
- Lemonade
How to use shibatch/tiny1m with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull shibatch/tiny1m:Q4_K_M
Run and chat with the model
lemonade run user.tiny1m-Q4_K_M
List all available models
lemonade list
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -16,7 +16,7 @@ tags:
|
|
| 16 |
This repository provides ultra-lightweight Llama2 model files across various formats (both **GGUF** and **Hugging Face / Safetensors**), trained on the TinyStories dataset and optimized for compatibility with Andrej Karpathy's `llama2.c` and `llama.cpp`.
|
| 17 |
|
| 18 |
### Why this repository exists
|
| 19 |
-
When developing a custom LLM inference engine
|
| 20 |
|
| 21 |
---
|
| 22 |
|
|
@@ -68,8 +68,6 @@ To verify your local setup or compare tokens using the official native utilities
|
|
| 68 |
|
| 69 |
The `model.bin` is fully compatible with the 512-vocab `tokenizer.bin` derived from the `stories260k` asset pipeline.
|
| 70 |
|
| 71 |
-
> ⚠️ **Important Note for `llama2.c/run`:** When passing a prompt to the `run` binary, you must use the **`-i`** option. Do not use `-p`, as `-p` is reserved for the Top-p sampling threshold in `llama2.c`, which will cause the prompt to be ignored.
|
| 72 |
-
|
| 73 |
```bash
|
| 74 |
./run model.bin -z tokenizer.bin -i "Tom and Jerry are " -n 64
|
| 75 |
|
|
@@ -83,7 +81,7 @@ You can import the Hugging Face variant directly into Python using the `transfor
|
|
| 83 |
import torch
|
| 84 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 85 |
|
| 86 |
-
repo_id = "
|
| 87 |
|
| 88 |
# The library automatically looks into the hf/ folder using the subfolder argument
|
| 89 |
tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="hf")
|
|
|
|
| 16 |
This repository provides ultra-lightweight Llama2 model files across various formats (both **GGUF** and **Hugging Face / Safetensors**), trained on the TinyStories dataset and optimized for compatibility with Andrej Karpathy's `llama2.c` and `llama.cpp`.
|
| 17 |
|
| 18 |
### Why this repository exists
|
| 19 |
+
When developing a custom LLM inference engine, debugging with a full-sized model is slow. This suite offers a true **1M parameter scale model** (~1MB to ~4MB depending on the quantization format), allowing developers to validate their loaders, serialization, quantization kernels, and inference logic step-by-step with maximum efficiency.
|
| 20 |
|
| 21 |
---
|
| 22 |
|
|
|
|
| 68 |
|
| 69 |
The `model.bin` is fully compatible with the 512-vocab `tokenizer.bin` derived from the `stories260k` asset pipeline.
|
| 70 |
|
|
|
|
|
|
|
| 71 |
```bash
|
| 72 |
./run model.bin -z tokenizer.bin -i "Tom and Jerry are " -n 64
|
| 73 |
|
|
|
|
| 81 |
import torch
|
| 82 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 83 |
|
| 84 |
+
repo_id = "shibatch/tiny1m"
|
| 85 |
|
| 86 |
# The library automatically looks into the hf/ folder using the subfolder argument
|
| 87 |
tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="hf")
|