Hugging Face's logo

wyt2000
/
InverseCoder-CL-13B

Text Generation
Transformers
Safetensors
llama
code
conversational
Eval Results (legacy)
text-generation-inference
Model card Files
xet
 Community
wyt2000 commited on
Commit
f23922c
·
verified ·
1 Parent(s): 109760e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +197 -3
README.md CHANGED
@@ -1,3 +1,197 @@
1
- ---
2
- license: llama2
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: llama2
3
+ datasets:
4
+ - wyt2000/InverseCoder-CL-13B-Evol-Instruct-90K
5
+ - ise-uiuc/Magicoder-Evol-Instruct-110K
6
+ library_name: transformers
7
+ pipeline_tag: text-generation
8
+ tags:
9
+ - code
10
+ model-index:
11
+ - name: InverseCoder-CL-13B
12
+ results:
13
+ - task:
14
+ type: text-generation
15
+ dataset:
16
+ type: openai_humaneval
17
+ name: HumanEval
18
+ metrics:
19
+ - name: pass@1
20
+ type: pass@1
21
+ value: 0.799
22
+ verified: false
23
+ - task:
24
+ type: text-generation
25
+ dataset:
26
+ type: openai_humaneval
27
+ name: HumanEval(+)
28
+ metrics:
29
+ - name: pass@1
30
+ type: pass@1
31
+ value: 0.744
32
+ verified: false
33
+ - task:
34
+ type: text-generation
35
+ dataset:
36
+ type: mbpp
37
+ name: MBPP
38
+ metrics:
39
+ - name: pass@1
40
+ type: pass@1
41
+ value: 0.746
42
+ verified: false
43
+ - task:
44
+ type: text-generation
45
+ dataset:
46
+ type: mbpp
47
+ name: MBPP(+)
48
+ metrics:
49
+ - name: pass@1
50
+ type: pass@1
51
+ value: 0.630
52
+ verified: false
53
+ - task:
54
+ type: text-generation
55
+ dataset:
56
+ type: ds1000
57
+ name: DS-1000 (Overall Completion)
58
+ metrics:
59
+ - name: pass@1
60
+ type: pass@1
61
+ value: 0.431
62
+ verified: false
63
+ - task:
64
+ type: text-generation
65
+ dataset:
66
+ type: nuprl/MultiPL-E
67
+ name: MultiPL-HumanEval (Java)
68
+ metrics:
69
+ - name: pass@1
70
+ type: pass@1
71
+ value: 0.545
72
+ verified: false
73
+ - task:
74
+ type: text-generation
75
+ dataset:
76
+ type: nuprl/MultiPL-E
77
+ name: MultiPL-HumanEval (JavaScript)
78
+ metrics:
79
+ - name: pass@1
80
+ type: pass@1
81
+ value: 0.654
82
+ verified: false
83
+ - task:
84
+ type: text-generation
85
+ dataset:
86
+ type: nuprl/MultiPL-E
87
+ name: MultiPL-HumanEval (C++)
88
+ metrics:
89
+ - name: pass@1
90
+ type: pass@1
91
+ value: 0.581
92
+ verified: false
93
+ - task:
94
+ type: text-generation
95
+ dataset:
96
+ type: nuprl/MultiPL-E
97
+ name: MultiPL-HumanEval (PHP)
98
+ metrics:
99
+ - name: pass@1
100
+ type: pass@1
101
+ value: 0.553
102
+ verified: false
103
+ - task:
104
+ type: text-generation
105
+ dataset:
106
+ type: nuprl/MultiPL-E
107
+ name: MultiPL-HumanEval (Swift)
108
+ metrics:
109
+ - name: pass@1
110
+ type: pass@1
111
+ value: 0.525
112
+ verified: false
113
+ - task:
114
+ type: text-generation
115
+ dataset:
116
+ type: nuprl/MultiPL-E
117
+ name: MultiPL-HumanEval (Rust)
118
+ metrics:
119
+ - name: pass@1
120
+ type: pass@1
121
+ value: 0.556
122
+ verified: false
123
+ - task:
124
+ type: text-generation
125
+ dataset:
126
+ type: nuprl/MultiPL-E
127
+ name: MultiPL-HumanEval (Average for non-python languages)
128
+ metrics:
129
+ - name: pass@1
130
+ type: pass@1
131
+ value: 0.569
132
+ verified: false
133
+ ---
134
+ <div align="center">
135
+ <img src="https://huggingface.co/wyt2000/InverseCoder-CL-7B/resolve/main/assets/logo.png" style="zoom:25%;" />
136
+ </div>
137
+
138
+ # InverseCoder: Unleashing the Power of Instruction-Tuned Code LLMs with Inverse-Instruct
139
+
140
+ <img src="https://huggingface.co/wyt2000/InverseCoder-CL-7B/resolve/main/assets/overview.png" style="zoom:50%;" />
141
+
142
+ InverseCoder is a series of code LLMs instruction-tuned by generating data from itself through Inverse-Instruct.
143
+
144
+ ## Models and Datasets
145
+ | | Base Model | InverseCoder | Dataset |
146
+ | --- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
147
+ | 6.7B | [deepseek-ai/deepseek-coder-6.7b-base](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-base) | [wyt2000/InverseCoder-DS-6.7B](https://huggingface.co/wyt2000/InverseCoder-DS-6.7B) | [wyt2000/InverseCoder-DS-6.7B-Evol-Instruct-90K](https://huggingface.co/datasets/wyt2000/InverseCoder-DS-6.7B-Evol-Instruct-90K) |
148
+ | 7B | [codellama/CodeLlama-7b-Python-hf](https://huggingface.co/codellama/CodeLlama-7b-Python-hf) | [wyt2000/InverseCoder-CL-7B](https://huggingface.co/wyt2000/InverseCoder-CL-7B) | [wyt2000/InverseCoder-CL-7B-Evol-Instruct-90K](https://huggingface.co/datasets/wyt2000/InverseCoder-CL-7B-Evol-Instruct-90K) |
149
+ | 13B | [codellama/CodeLlama-13b-Python-hf](https://huggingface.co/codellama/CodeLlama-13b-Python-hf) | [wyt2000/InverseCoder-CL-13B](https://huggingface.co/wyt2000/InverseCoder-CL-13B) **<= You are here** | [wyt2000/InverseCoder-CL-13B-Evol-Instruct-90K](https://huggingface.co/datasets/wyt2000/InverseCoder-CL-13B-Evol-Instruct-90K) |
150
+
151
+ ## Usage
152
+
153
+ Similar to [Magicoder-S-DS-6.7B](https://huggingface.co/ise-uiuc/Magicoder-S-DS-6.7B/), use the code below to get started with the model. Make sure you installed the [transformers](https://huggingface.co/docs/transformers/index) library.
154
+
155
+ ```python
156
+ from transformers import pipeline
157
+ import torch
158
+ INVERSECODER_PROMPT = """You are an exceptionally intelligent coding assistant that consistently delivers accurate and reliable responses to user instructions.
159
+ @@ Instruction
160
+ {instruction}
161
+ @@ Response
162
+ """
163
+ instruction = <Your code instruction here>
164
+ prompt = INVERSECODER_PROMPT.format(instruction=instruction)
165
+ generator = pipeline(
166
+ model="wyt2000/InverseCoder-CL-13B",
167
+ task="text-generation",
168
+ torch_dtype=torch.bfloat16,
169
+ device_map="auto",
170
+ )
171
+ result = generator(prompt, max_length=1024, num_return_sequences=1, temperature=0.0)
172
+ print(result[0]["generated_text"])
173
+ ```
174
+
175
+ ## Paper
176
+ **Arxiv:** <https://arxiv.org/abs/2407.05700>
177
+
178
+ Please cite the paper if you use the models or datasets from InverseCoder.
179
+
180
+ ```
181
+ @misc{wu2024inversecoderunleashingpowerinstructiontuned,
182
+ title={InverseCoder: Unleashing the Power of Instruction-Tuned Code LLMs with Inverse-Instruct},
183
+ author={Yutong Wu and Di Huang and Wenxuan Shi and Wei Wang and Lingzhe Gao and Shihao Liu and Ziyuan Nan and Kaizhao Yuan and Rui Zhang and Xishan Zhang and Zidong Du and Qi Guo and Yewen Pu and Dawei Yin and Xing Hu and Yunji Chen},
184
+ year={2024},
185
+ eprint={2407.05700},
186
+ archivePrefix={arXiv},
187
+ primaryClass={cs.CL},
188
+ url={https://arxiv.org/abs/2407.05700},
189
+ }
190
+ ```
191
+
192
+ ## Acknowledgements
193
+
194
+ * [Magicoder](https://github.com/ise-uiuc/magicoder): Training code, original datasets and data decontamination
195
+ * [DeepSeek-Coder](https://github.com/deepseek-ai/DeepSeek-Coder): Base model for InverseCoder-DS
196
+ * [CodeLlama](https://ai.meta.com/research/publications/code-llama-open-foundation-models-for-code/): Base model for InverseCoder-CL
197
+ * [AutoMathText](https://github.com/yifanzhang-pro/AutoMathText): Self-evaluation and data selection method