codewithdark commited on
Commit
e676d4e
·
verified ·
1 Parent(s): b8527e9

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +14 -159
README.md CHANGED
@@ -1,160 +1,15 @@
1
  ---
2
- license: apache-2.0
3
- base_model: Qwen/Qwen2.5-7B-Instruct
4
- library_name: mlx
5
- language:
6
- - en
7
- tags:
8
- - quantllm
9
- - mlx
10
- - mlx-lm
11
- - apple-silicon
12
- - transformers
13
- - 4bit
14
- ---
15
-
16
- <div align="center">
17
-
18
- # [MLX] Qwen2.5-7B-Instruct-MLX
19
-
20
- **Qwen/Qwen2.5-7B-Instruct** converted to **MLX** format
21
-
22
- [![QuantLLM](https://img.shields.io/badge/MADE_WITH-QuantLLM-orange?style=for-the-badge)](https://github.com/codewithdark-git/QuantLLM)
23
- [![Format](https://img.shields.io/badge/Format-MLX-blue?style=for-the-badge)]()
24
- [![Quantization](https://img.shields.io/badge/Quant-4bit-green?style=for-the-badge)]()
25
-
26
- <a href="https://github.com/codewithdark-git/QuantLLM">[STAR] Star QuantLLM on GitHub</a>
27
-
28
- </div>
29
-
30
- ---
31
-
32
-
33
- ## [ABOUT] About This Model
34
-
35
- This model is **[Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)** converted to **MLX** format optimized for Apple Silicon (M1/M2/M3/M4) Macs with native acceleration.
36
-
37
- | Property | Value |
38
- |----------|-------|
39
- | **Base Model** | [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) |
40
- | **Format** | MLX |
41
- | **Quantization** | 4bit |
42
- | **License** | apache-2.0 |
43
- | **Created With** | [QuantLLM](https://github.com/codewithdark-git/QuantLLM) |
44
-
45
-
46
- ## [START] Quick Start
47
-
48
- ### Generate Text with mlx-lm
49
-
50
- ```python
51
- from mlx_lm import load, generate
52
-
53
- # Load the model
54
- model, tokenizer = load("QuantLLM/Qwen2.5-7B-Instruct-MLX")
55
-
56
- # Simple generation
57
- prompt = "Explain quantum computing in simple terms"
58
- messages = [{"role": "user", "content": prompt}]
59
- prompt_formatted = tokenizer.apply_chat_template(
60
- messages,
61
- add_generation_prompt=True
62
- )
63
-
64
- # Generate response
65
- text = generate(model, tokenizer, prompt=prompt_formatted, verbose=True)
66
- print(text)
67
- ```
68
-
69
- ### Streaming Generation
70
-
71
- ```python
72
- from mlx_lm import load, stream_generate
73
-
74
- model, tokenizer = load("QuantLLM/Qwen2.5-7B-Instruct-MLX")
75
-
76
- prompt = "Write a haiku about coding"
77
- messages = [{"role": "user", "content": prompt}]
78
- prompt_formatted = tokenizer.apply_chat_template(
79
- messages,
80
- add_generation_prompt=True
81
- )
82
-
83
- # Stream tokens as they're generated
84
- for token in stream_generate(model, tokenizer, prompt=prompt_formatted, max_tokens=200):
85
- print(token, end="", flush=True)
86
- ```
87
-
88
- ### Command Line Interface
89
-
90
- ```bash
91
- # Install mlx-lm
92
- pip install mlx-lm
93
-
94
- # Generate text
95
- python -m mlx_lm.generate --model QuantLLM/Qwen2.5-7B-Instruct-MLX --prompt "Hello!"
96
-
97
- # Interactive chat
98
- python -m mlx_lm.chat --model QuantLLM/Qwen2.5-7B-Instruct-MLX
99
- ```
100
-
101
- ### System Requirements
102
-
103
- | Requirement | Minimum |
104
- |-------------|---------|
105
- | **Chip** | Apple Silicon (M1/M2/M3/M4) |
106
- | **macOS** | 13.0 (Ventura) or later |
107
- | **Python** | 3.10+ |
108
- | **RAM** | 8GB+ (16GB recommended) |
109
-
110
- ```bash
111
- # Install dependencies
112
- pip install mlx-lm
113
- ```
114
-
115
-
116
- ## [DETAILS] Model Details
117
-
118
- | Property | Value |
119
- |----------|-------|
120
- | **Original Model** | [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) |
121
- | **Format** | MLX |
122
- | **Quantization** | 4bit |
123
- | **License** | `apache-2.0` |
124
- | **Export Date** | 2026-08-03 |
125
- | **Exported By** | [QuantLLM v2.1](https://github.com/codewithdark-git/QuantLLM) |
126
-
127
-
128
-
129
- ---
130
-
131
- ## [START] Created with QuantLLM
132
-
133
- <div align="center">
134
-
135
- [![QuantLLM](https://img.shields.io/badge/QuantLLM-Ultra_fast_LLM_Quantization-orange?style=for-the-badge)](https://github.com/codewithdark-git/QuantLLM)
136
-
137
- **Convert any model to GGUF, ONNX, or MLX in one line!**
138
-
139
- ```python
140
- from quantllm import turbo
141
-
142
- # Load any HuggingFace model
143
- model = turbo("Qwen/Qwen2.5-7B-Instruct")
144
-
145
- # Export to any format
146
- model.export("mlx", quantization="4bit")
147
-
148
- # Push to HuggingFace
149
- model.push("your-repo", format="mlx")
150
- ```
151
-
152
- <a href="https://github.com/codewithdark-git/QuantLLM">
153
- <img src="https://img.shields.io/github/stars/codewithdark-git/QuantLLM?style=social" alt="GitHub Stars">
154
- </a>
155
-
156
- **[DOCS](https://github.com/codewithdark-git/QuantLLM#readme)** ·
157
- **[ISSUE](https://github.com/codewithdark-git/QuantLLM/issues)** ·
158
- **[FEATURE](https://github.com/codewithdark-git/QuantLLM/issues)**
159
-
160
- </div>
 
1
  ---
2
+ {
3
+ "tags": [
4
+ "qwen"
5
+ ],
6
+ "params": 7.0,
7
+ "quality_score": 4.0,
8
+ "speed_score": 4.5,
9
+ "verified": true,
10
+ "recommended_quant": "4bit"
11
+ }
12
+ ---
13
+ # QuantLLM/Qwen2.5-7B-Instruct-MLX
14
+
15
+ QuantLLM-registered model.