| | --- |
| | license: apache-2.0 |
| | datasets: |
| | - bigcode/starcoderdata |
| | --- |
| | # Model Card for DeciCoder-6B |
| |
|
| | DeciCoder-6B is a 6 billion parameter decoder-only code completion model |
| | trained on the Python, Java, Javascript, Rust, C++, C, and C# subset of [Starcoder Training Dataset](https://huggingface.co/datasets/bigcode/starcoderdata). |
| | The model uses variable Grouped Query Attention and has a context window of 2k |
| | tokens. It was trained using a Fill-in-the-Middle training objective. The model's |
| | architecture was generated by Deci's proprietary Neural Architecture |
| | Search-based technology, AutoNAC. |
| |
|
| | ## Model Details |
| |
|
| | - **Developed by:** Deci |
| | - **Model type:** DeciCoder-6B is an auto-regressive language model based on the transformer decoder architecture, using variable Grouped Query Attention. |
| | - **Language(s):** Python, Java, JavaScript, Rust, C++, C, C#, Go |
| | - **License:** Model checkpoints are licensed under the [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) |
| |
|
| | ## Documentation |
| |
|
| | - Blog Post: [Introducing DeciCoder-6B: Code LLM Engineered for Accuracy & Cost Efficiency At Scale](https://deci.ai/blog/decicoder-6b-the-best-multi-language-code-generation-llm-in-its-class/) |
| | - Tutorial: [How to Run DeciCoder-6B on Qualcomm Cloud AI 100](https://github.com/quic/cloud-ai-sdk/tree/1.12/models/language_processing/decoder) |
| | - Google Colab [Notebook](http://bit.ly/DeciCoder-6B-Notebook-1) |
| | - Run DeciCoder on [AWS DL2q instances using the Qualcomm Cloud AI Platform SDK](https://bit.ly/Amazon-EC2-DL2q-Instance) |
| | - Questions: Feel free to contact us via our [Discord Community!](https://discord.com/invite/p9ecgRhDR8/) |
| |
|
| | ## Model Architecture |
| |
|
| | | Parameters | Layers | Heads | Sequence Length | GQA num_key_value_heads | |
| | |:----------|:----------|:----------|:----------|:----------| |
| | | 6B | 32 | 32 | 2k | Variable | |
| | |
| | |
| | - **Decoder layer:** Variable Grouped Query Attention |
| | - **Position Embeddings:** Rotary Position Embeddings [Su et al., 2021](https://arxiv.org/abs/2104.09864) |
| | |
| | |
| | ### How to Use |
| | |
| | ```bibtex |
| | # pip install -q transformers |
| | import torch |
| | from transformers import AutoModelForCausalLM, AutoTokenizer |
| | |
| | checkpoint = "Deci/DeciCoder-6B" |
| | device = "cuda" # for GPU usage or "cpu" for CPU usage |
| | |
| | tokenizer = AutoTokenizer.from_pretrained(checkpoint) |
| | model = AutoModelForCausalLM.from_pretrained(checkpoint, torch_dtype=torch.bfloat16, trust_remote_code=True).to(device) |
| |
|
| | inputs = tokenizer.encode("def print_hello_world():", return_tensors="pt").to(device) |
| | outputs = model.generate(inputs, max_new_tokens=100) |
| | print(tokenizer.decode(outputs[0])) |
| | |
| | ### Attribution |
| | |
| | DeciCoder-6B was trained on StarCoder Training Dataset, filtered for |
| | Python, Java, JavaScript, Ruby, RUST, C++, C, and C#. For additional information, please |
| | refer to [https://huggingface.co/datasets/bigcode/starcoderdata](https://huggingface.co/datasets/bigcode/starcoderdata). |
| | |
| | ``` |
| | |
| | ### Limitations |
| | |
| | The model has undergone training with source code from Python, Java, |
| | JavaScript, RUST, C++, C, and C#, and Go. While the primary language in the source is English, it does |
| | contain other languages. Therefore, the model can produce code snippets |
| | given some context. However, there is no assurance that the resulting |
| | code will function as expected. It might be suboptimal, contain bugs, or |
| | even exploits. |
| | |
| | ## Evaluation |
| | |
| | Below are DeciCoder-6B's pass@1 on MultiPL HumanEval scores |
| | |
| | | Python | JavaScript | Java | C++ | C# | Rust | Go | |
| | |:----------|:----------|:----------|:----------|:----------|:----------|:----------| |
| | | 33.3% | 29.3% | 30.3% |29.93% |20.31% |20.5% |77.47% | |
| | |
| | |
| | ### Runtime Benchmarks |
| | |
| | |Inference Tool | Hardware | Prompt Length | Generation Length | Throughput (tokens/sec) | |
| | |:----------|:----------|:----------|:----------|:----------| |
| | | Qualcomm Cloud AI 100 SDK | Qualcomm Cloud AI 100 | 1024 | 1024 | 531.3 | |
| | |
| | - Measured for maximal batch size on the device |
| | |
| | ## How to Cite |
| | |
| | Please cite this model using this format. |
| | |
| | ```bibtex |
| | @misc{DeciFoundationModels, |
| | title = {DeciCoder-6B}, |
| | author = {DeciAI Research Team}, |
| | year = {2024} |
| | url={[https://huggingface.co/deci/decicoder-6B](https://huggingface.co/deci/decicoder-6B)}, |
| | } |
| | ``` |
