Instructions to use shibatch/tiny1m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use shibatch/tiny1m with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("shibatch/tiny1m", dtype="auto") - llama-cpp-python
How to use shibatch/tiny1m with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="shibatch/tiny1m", filename="tiny1m.BF16.gguf", )
output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use shibatch/tiny1m with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf shibatch/tiny1m:Q4_K_M # Run inference directly in the terminal: llama-cli -hf shibatch/tiny1m:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf shibatch/tiny1m:Q4_K_M # Run inference directly in the terminal: llama-cli -hf shibatch/tiny1m:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf shibatch/tiny1m:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf shibatch/tiny1m:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf shibatch/tiny1m:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf shibatch/tiny1m:Q4_K_M
Use Docker
docker model run hf.co/shibatch/tiny1m:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use shibatch/tiny1m with Ollama:
ollama run hf.co/shibatch/tiny1m:Q4_K_M
- Unsloth Studio
How to use shibatch/tiny1m with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for shibatch/tiny1m to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for shibatch/tiny1m to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for shibatch/tiny1m to start chatting
- Docker Model Runner
How to use shibatch/tiny1m with Docker Model Runner:
docker model run hf.co/shibatch/tiny1m:Q4_K_M
- Lemonade
How to use shibatch/tiny1m with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull shibatch/tiny1m:Q4_K_M
Run and chat with the model
lemonade run user.tiny1m-Q4_K_M
List all available models
lemonade list
Upload folder using huggingface_hub
Browse files- .gitattributes +21 -0
- hf/config.json +32 -0
- hf/generation_config.json +10 -0
- hf/model.safetensors +3 -0
- hf/tokenizer.model +3 -0
- model.bin +3 -0
- tiny1m.BF16.gguf +3 -0
- tiny1m.F16.gguf +3 -0
- tiny1m.F32.gguf +3 -0
- tiny1m.IQ3_S.gguf +3 -0
- tiny1m.IQ3_XXS.gguf +3 -0
- tiny1m.IQ4_NL.gguf +3 -0
- tiny1m.IQ4_XS.gguf +3 -0
- tiny1m.Q2_K.gguf +3 -0
- tiny1m.Q3_K_L.gguf +3 -0
- tiny1m.Q3_K_M.gguf +3 -0
- tiny1m.Q3_K_S.gguf +3 -0
- tiny1m.Q4_0.gguf +3 -0
- tiny1m.Q4_1.gguf +3 -0
- tiny1m.Q4_K_M.gguf +3 -0
- tiny1m.Q4_K_S.gguf +3 -0
- tiny1m.Q5_K_M.gguf +3 -0
- tiny1m.Q5_K_S.gguf +3 -0
- tiny1m.Q6_K.gguf +3 -0
- tiny1m.Q8_0.gguf +3 -0
- tiny1m.TQ1_0.gguf +3 -0
- tiny1m.TQ2_0.gguf +3 -0
- tokenizer.bin +3 -0
- tokenizer.model +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,24 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
tiny1m.BF16.gguf filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
tiny1m.F16.gguf filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
tiny1m.F32.gguf filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
tiny1m.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
tiny1m.IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
tiny1m.IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
tiny1m.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
tiny1m.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
tiny1m.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
tiny1m.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
tiny1m.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
tiny1m.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 48 |
+
tiny1m.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
| 49 |
+
tiny1m.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 50 |
+
tiny1m.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 51 |
+
tiny1m.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 52 |
+
tiny1m.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 53 |
+
tiny1m.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
| 54 |
+
tiny1m.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 55 |
+
tiny1m.TQ1_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 56 |
+
tiny1m.TQ2_0.gguf filter=lfs diff=lfs merge=lfs -text
|
hf/config.json
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"LlamaForCausalLM"
|
| 4 |
+
],
|
| 5 |
+
"attention_bias": false,
|
| 6 |
+
"attention_dropout": 0.0,
|
| 7 |
+
"bos_token_id": 1,
|
| 8 |
+
"dtype": "float32",
|
| 9 |
+
"eos_token_id": 2,
|
| 10 |
+
"head_dim": 64,
|
| 11 |
+
"hidden_act": "silu",
|
| 12 |
+
"hidden_size": 128,
|
| 13 |
+
"initializer_range": 0.02,
|
| 14 |
+
"intermediate_size": 352,
|
| 15 |
+
"max_position_embeddings": 256,
|
| 16 |
+
"mlp_bias": false,
|
| 17 |
+
"model_type": "llama",
|
| 18 |
+
"num_attention_heads": 2,
|
| 19 |
+
"num_hidden_layers": 4,
|
| 20 |
+
"num_key_value_heads": 2,
|
| 21 |
+
"pad_token_id": 2,
|
| 22 |
+
"pretraining_tp": 1,
|
| 23 |
+
"rms_norm_eps": 1e-06,
|
| 24 |
+
"rope_parameters": {
|
| 25 |
+
"rope_theta": 10000.0,
|
| 26 |
+
"rope_type": "default"
|
| 27 |
+
},
|
| 28 |
+
"tie_word_embeddings": false,
|
| 29 |
+
"transformers_version": "5.9.0",
|
| 30 |
+
"use_cache": false,
|
| 31 |
+
"vocab_size": 512
|
| 32 |
+
}
|
hf/generation_config.json
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_from_model_config": true,
|
| 3 |
+
"bos_token_id": 1,
|
| 4 |
+
"eos_token_id": 2,
|
| 5 |
+
"output_attentions": false,
|
| 6 |
+
"output_hidden_states": false,
|
| 7 |
+
"pad_token_id": 2,
|
| 8 |
+
"transformers_version": "5.9.0",
|
| 9 |
+
"use_cache": true
|
| 10 |
+
}
|
hf/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f30ae85c53797b5944da7f235162756b31987233b44eac36ed449afd1b82523b
|
| 3 |
+
size 3744288
|
hf/tokenizer.model
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dfff07d929db979913f166ec94a6f5ecad4c70cfed8eb5c9cbe7e464455e46f5
|
| 3 |
+
size 7645
|
model.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:80ba15bd7feed5f78c13c33bac83992617ff030661f16abbfed8fba0a6e335c1
|
| 3 |
+
size 3805724
|
tiny1m.BF16.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d977dcf0503f218daa648c4f75c217caa7105ff0b868a9748b8dbce16fcf1e7c
|
| 3 |
+
size 1886272
|
tiny1m.F16.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1521a0a32ef186dc10dcb45c3e775d3456eabb69c3168474f23c6ca9c0e23adc
|
| 3 |
+
size 1886272
|
tiny1m.F32.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:60fa2675b0531735c9724c9fcb871baa6e40594f451d1454981034be47d025d4
|
| 3 |
+
size 3754048
|
tiny1m.IQ3_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:65b9e571127377aafce098649c634a4332f41dbf9c377eb6d0d610ec5d695286
|
| 3 |
+
size 576576
|
tiny1m.IQ3_XXS.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e143756176671e53454e704a3814b604870adbb988f8b8e3877a8b3d9f1e5815
|
| 3 |
+
size 576576
|
tiny1m.IQ4_NL.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:005683cc0f1242911053214b0e4cf8289e1ef5e3551a9e44b58eb46dd7e0cef9
|
| 3 |
+
size 576576
|
tiny1m.IQ4_XS.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a2ea3a8a9649aabaf3366992954fd52e4d608b10e0804b0f09f57b3c816aaf59
|
| 3 |
+
size 576576
|
tiny1m.Q2_K.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dd40260dbe2d88fc5262acd508b35807c5c91fe479f8d113a7202e65e6190c61
|
| 3 |
+
size 576576
|
tiny1m.Q3_K_L.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:424e211ba566176cc0520d07a95c7dd8507329c023ade5695a5dbd8462387248
|
| 3 |
+
size 634944
|
tiny1m.Q3_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:717071ee94a35e17b6805795577a276436c8eb335673b199cbf62a876a600f7b
|
| 3 |
+
size 617536
|
tiny1m.Q3_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d98d22fd5ff3ca2e6c6797544f1ed199a9d2a14edeeadf55fe3c6eb7c2e2592e
|
| 3 |
+
size 576576
|
tiny1m.Q4_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:25ad51d8535ea39a985c5684a4654b46cff77e7008c08290a386c0c332c2872a
|
| 3 |
+
size 576576
|
tiny1m.Q4_1.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c6cbc29e3cdcf8f0e2b5b72900b347d383ec5a111a75a4bb1cdad392fba615bb
|
| 3 |
+
size 630848
|
tiny1m.Q4_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ae0bb9b0ca9bb62ccbef04fc1dc1217bbe9b7a2f82c9cbc475c30ee65a4030fb
|
| 3 |
+
size 731200
|
tiny1m.Q4_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1c960d5e9f01bfb42cca6285ff2739cdbef440942e63b95dae554d7612a50c6a
|
| 3 |
+
size 689216
|
tiny1m.Q5_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d46f51527b5ea49a0269f210a2b805eb27e5e5b0da78ce327d7dc1b6f0413280
|
| 3 |
+
size 777792
|
tiny1m.Q5_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:49955604e099ef3cc92db5eac88f2aaa41e16a79e8a89b7c6347cae31cdaa22f
|
| 3 |
+
size 739392
|
tiny1m.Q6_K.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fdbc43ac8b23484fb8706655d879e27f068c46acacbfc83b97ca84d6eeef07c9
|
| 3 |
+
size 1010752
|
tiny1m.Q8_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f11908a678e6308bb88b906739120cf436c795e31e5e1beaaed1cd4e0342a368
|
| 3 |
+
size 1010752
|
tiny1m.TQ1_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f14be5e93dd7f0edbdf92f015ac48832b8078ef6ebcef173ca0669905474cb81
|
| 3 |
+
size 584768
|
tiny1m.TQ2_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b61cd4fb4db6097327d0ac464df1976391c248c3cf57d5515abc5e2c5de2ab5b
|
| 3 |
+
size 584768
|
tokenizer.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:037cb335abb25d1fa9e8ecae30ed2a3a8ace9302862ebcdc05d51a6bbb10c312
|
| 3 |
+
size 6227
|
tokenizer.model
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dfff07d929db979913f166ec94a6f5ecad4c70cfed8eb5c9cbe7e464455e46f5
|
| 3 |
+
size 7645
|