algoscienceacademy commited on
Commit
c0b9f59
·
verified ·
1 Parent(s): a64607e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +250 -39
README.md CHANGED
@@ -1,66 +1,277 @@
 
1
  ---
2
  license: apache-2.0
3
- datasets:
4
- - cerebras/SlimPajama-627B
5
- - bigcode/starcoderdata
6
- - HuggingFaceH4/ultrachat_200k
7
- - HuggingFaceH4/ultrafeedback_binarized
8
  language:
9
  - en
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
  widget:
11
- - example_title: Fibonacci (Python)
12
  messages:
13
- - role: system
14
- content: You are a chatbot who can help code!
15
- - role: user
16
- content: Write me a function to calculate the first 10 digits of the fibonacci sequence in Python and print it out to the CLI.
17
  ---
 
18
  <div align="center">
19
 
20
- # TinyLlama-1.1B
 
 
 
 
 
 
 
21
  </div>
22
 
23
- https://github.com/jzhang38/TinyLlama
24
 
25
- The TinyLlama project aims to **pretrain** a **1.1B Llama model on 3 trillion tokens**. With some proper optimization, we can achieve this within a span of "just" 90 days using 16 A100-40G GPUs 🚀🚀. The training has started on 2023-09-01.
26
 
 
27
 
28
- We adopted exactly the same architecture and tokenizer as Llama 2. This means TinyLlama can be plugged and played in many open-source projects built upon Llama. Besides, TinyLlama is compact with only 1.1B parameters. This compactness allows it to cater to a multitude of applications demanding a restricted computation and memory footprint.
29
 
30
- #### This Model
31
- This is the chat model finetuned on top of [TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T). **We follow [HF's Zephyr](https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha)'s training recipe.** The model was " initially fine-tuned on a variant of the [`UltraChat`](https://huggingface.co/datasets/stingning/ultrachat) dataset, which contains a diverse range of synthetic dialogues generated by ChatGPT.
32
- We then further aligned the model with [🤗 TRL's](https://github.com/huggingface/trl) `DPOTrainer` on the [openbmb/UltraFeedback](https://huggingface.co/datasets/openbmb/UltraFeedback) dataset, which contain 64k prompts and model completions that are ranked by GPT-4."
33
 
 
34
 
35
- #### How to use
36
- You will need the transformers>=4.34
37
- Do check the [TinyLlama](https://github.com/jzhang38/TinyLlama) github page for more information.
38
 
39
- ```python
40
- # Install transformers from source - only needed for versions <= v4.34
41
- # pip install git+https://github.com/huggingface/transformers.git
42
- # pip install accelerate
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
43
 
44
- import torch
45
  from transformers import pipeline
46
 
47
- pipe = pipeline("text-generation", model="TinyLlama/TinyLlama-1.1B-Chat-v1.0", torch_dtype=torch.bfloat16, device_map="auto")
 
 
 
 
48
 
49
- # We use the tokenizer's chat template to format each message - see https://huggingface.co/docs/transformers/main/en/chat_templating
50
  messages = [
51
  {
52
  "role": "system",
53
- "content": "You are a friendly chatbot who always responds in the style of a pirate",
54
  },
55
- {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
 
 
 
56
  ]
57
- prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
58
- outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
59
- print(outputs[0]["generated_text"])
60
- # <|system|>
61
- # You are a friendly chatbot who always responds in the style of a pirate.</s>
62
- # <|user|>
63
- # How many helicopters can a human eat in one sitting?</s>
64
- # <|assistant|>
65
- # ...
66
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
  ---
3
  license: apache-2.0
 
 
 
 
 
4
  language:
5
  - en
6
+ library_name: transformers
7
+ pipeline_tag: text-generation
8
+ base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
9
+ tags:
10
+ - llm
11
+ - chatbot
12
+ - text-generation
13
+ - causal-lm
14
+ - harness
15
+ - algosciencelab
16
+ - vlsi
17
+ - coding
18
+ - reasoning
19
+ - pytorch
20
  widget:
21
+ - example_title: Python Example
22
  messages:
23
+ - role: system
24
+ content: You are Harness, an AI assistant created by Algo Science Lab.
25
+ - role: user
26
+ content: Write a Python function that calculates the factorial of a number.
27
  ---
28
+
29
  <div align="center">
30
 
31
+ # Harness-1B
32
+
33
+ ### Developed by Algo Science Lab
34
+
35
+ **Author:** Shahrear Hossain
36
+
37
+ **Hugging Face:** https://huggingface.co/algoscienceacademy
38
+
39
  </div>
40
 
41
+ ---
42
 
43
+ # Harness-1B
44
 
45
+ Harness-1B is a lightweight large language model developed by **Algo Science Lab** for instruction following, coding assistance, reasoning, mathematics, electronics, semiconductor engineering, VLSI design, and general conversational AI.
46
 
47
+ The model contains approximately **1 billion parameters**, making it suitable for local deployment while providing strong performance for everyday AI tasks.
48
 
49
+ Harness-1B has been fine-tuned to provide accurate, helpful, and concise responses across a wide variety of technical and general domains.
 
 
50
 
51
+ ---
52
 
53
+ # Features
 
 
54
 
55
+ - General conversation
56
+ - Code generation
57
+ - Python programming
58
+ - C programming
59
+ - C++ programming
60
+ - Rust programming
61
+ - Java programming
62
+ - JavaScript
63
+ - TypeScript
64
+ - Verilog HDL
65
+ - SystemVerilog
66
+ - VHDL
67
+ - RTL Design
68
+ - FPGA Development
69
+ - ASIC Design
70
+ - CMOS Digital Design
71
+ - Semiconductor Engineering
72
+ - VLSI Design
73
+ - Mathematics
74
+ - Electronics
75
+ - Physics
76
+ - Problem Solving
77
+ - Technical Documentation
78
+ - AI Research Assistance
79
+
80
+ ---
81
+
82
+ # Model Details
83
+
84
+ | Property | Value |
85
+ |----------|-------|
86
+ | Model Name | Harness-1B |
87
+ | Organization | Algo Science Lab |
88
+ | Hugging Face Username | algoscienceacademy |
89
+ | Parameters | ~1 Billion |
90
+ | Architecture | Llama-based |
91
+ | Model Type | Causal Language Model |
92
+ | Context Length | 2048 Tokens *(or your trained context size)* |
93
+ | Precision | FP16 / BF16 / GGUF |
94
+ | Framework | PyTorch |
95
+ | License | Apache-2.0 |
96
+
97
+ ---
98
+
99
+ # Intended Uses
100
+
101
+ Harness-1B is designed for:
102
+
103
+ - AI Chatbots
104
+ - Programming Assistant
105
+ - Educational Applications
106
+ - Research
107
+ - Embedded AI
108
+ - Local AI Deployment
109
+ - Engineering Assistance
110
+ - Electronics Design
111
+ - FPGA Development
112
+ - ASIC/VLSI Workflow
113
+ - RTL Development
114
+ - Automation
115
+ - Documentation
116
+
117
+ ---
118
+
119
+ # Example
120
 
121
+ ```python
122
  from transformers import pipeline
123
 
124
+ pipe = pipeline(
125
+ "text-generation",
126
+ model="algoscienceacademy/Harness-1B",
127
+ device_map="auto"
128
+ )
129
 
 
130
  messages = [
131
  {
132
  "role": "system",
133
+ "content": "You are Harness, an AI assistant created by Algo Science Lab."
134
  },
135
+ {
136
+ "role": "user",
137
+ "content": "Write a Python program to print Fibonacci numbers."
138
+ }
139
  ]
140
+
141
+ prompt = pipe.tokenizer.apply_chat_template(
142
+ messages,
143
+ tokenize=False,
144
+ add_generation_prompt=True
145
+ )
146
+
147
+ output = pipe(
148
+ prompt,
149
+ max_new_tokens=256,
150
+ temperature=0.7,
151
+ )
152
+
153
+ print(output[0]["generated_text"])
154
+ ```
155
+
156
+ ---
157
+
158
+ # GGUF Usage
159
+
160
+ Harness-1B is also available in GGUF format for:
161
+
162
+ - llama.cpp
163
+ - LM Studio
164
+ - Jan
165
+ - Open WebUI
166
+ - KoboldCpp
167
+ - Ollama (after conversion)
168
+
169
+ Example:
170
+
171
+ ```bash
172
+ ./main \
173
+ -m Harness-1B-Q4_K_M.gguf \
174
+ -p "Explain CMOS Inverter."
175
+ ```
176
+
177
+ ---
178
+
179
+ # Training
180
+
181
+ Harness-1B is trained and fine-tuned using open-source datasets and instruction-following techniques.
182
+
183
+ Possible data sources include:
184
+
185
+ - SlimPajama
186
+ - StarCoderData
187
+ - UltraChat
188
+ - UltraFeedback
189
+
190
+ Additional custom datasets may have been used during supervised fine-tuning.
191
+
192
+ ---
193
+
194
+ # Capabilities
195
+
196
+ Harness-1B can:
197
+
198
+ - Answer questions
199
+ - Explain concepts
200
+ - Generate code
201
+ - Debug code
202
+ - Write documentation
203
+ - Solve mathematics
204
+ - Explain algorithms
205
+ - Assist with VLSI
206
+ - Help with FPGA design
207
+ - Generate Verilog
208
+ - Generate SystemVerilog
209
+ - Produce technical reports
210
+
211
+ ---
212
+
213
+ # Limitations
214
+
215
+ Harness-1B may:
216
+
217
+ - Produce incorrect information.
218
+ - Generate outdated knowledge.
219
+ - Make reasoning mistakes.
220
+ - Require verification for safety-critical applications.
221
+ - Require human review for production environments.
222
+
223
+ ---
224
+
225
+ # Hardware Requirements
226
+
227
+ Recommended:
228
+
229
+ - 8 GB RAM (Q4 GGUF)
230
+ - 12 GB RAM (Q6 GGUF)
231
+ - 16 GB RAM (FP16)
232
+ - CUDA GPU recommended but optional
233
+
234
+ ---
235
+
236
+ # Citation
237
+
238
+ ```bibtex
239
+ @misc{Harness1B,
240
+ title={Harness-1B},
241
+ author={Algo Science Lab},
242
+ year={2026},
243
+ publisher={Hugging Face},
244
+ howpublished={https://huggingface.co/algoscienceacademy/Harness-1B}
245
+ }
246
+ ```
247
+
248
+ ---
249
+
250
+ # License
251
+
252
+ Apache License 2.0
253
+
254
+ ---
255
+
256
+ # Acknowledgements
257
+
258
+ Harness-1B builds upon open-source language model research and would not be possible without the work of the open-source AI community, including:
259
+
260
+ - Meta AI (Llama Architecture)
261
+ - TinyLlama Project
262
+ - Hugging Face
263
+ - PyTorch
264
+ - Transformers
265
+ - llama.cpp
266
+
267
+ ---
268
+
269
+ # Contact
270
+
271
+ **Organization:** Algo Science Lab
272
+
273
+ **Hugging Face:** https://huggingface.co/algoscienceacademy
274
+
275
+ ---
276
+
277
+ Made with ❤️ by Algo Science Lab.