Text Generation
Transformers
PyTorch
Chinese
English
llama
translation
multilingual
large language model
instruction tuning
text-generation-inference
Instructions to use ICTNLP/bayling-13b-diff with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ICTNLP/bayling-13b-diff with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ICTNLP/bayling-13b-diff")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("ICTNLP/bayling-13b-diff") model = AutoModelForCausalLM.from_pretrained("ICTNLP/bayling-13b-diff") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ICTNLP/bayling-13b-diff with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ICTNLP/bayling-13b-diff" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ICTNLP/bayling-13b-diff", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/ICTNLP/bayling-13b-diff
- SGLang
How to use ICTNLP/bayling-13b-diff with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ICTNLP/bayling-13b-diff" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ICTNLP/bayling-13b-diff", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ICTNLP/bayling-13b-diff" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ICTNLP/bayling-13b-diff", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use ICTNLP/bayling-13b-diff with Docker Model Runner:
docker model run hf.co/ICTNLP/bayling-13b-diff
Vily1998 commited on
Commit ·
ab4a04c
1
Parent(s): bd5ca76
update
Browse files
README.md
CHANGED
|
@@ -21,7 +21,7 @@ tags:
|
|
| 21 |
|
| 22 |
💬 [**Demo**](http://nlp.ict.ac.cn/bayling/demo): Welcome to apply for a trial of BayLing's online demo (beta version).
|
| 23 |
|
| 24 |
-
📄 **Paper**:
|
| 25 |
|
| 26 |
🏠 [**Blog**](http://nlp.ict.ac.cn/bayling): BayLing's homepage. You can discover some case of BayLing here.
|
| 27 |
|
|
@@ -31,7 +31,8 @@ tags:
|
|
| 31 |
|
| 32 |
> BayLing is developed by [NLP Group](http://nlp.ict.ac.cn/) of [Institute of Computing Technology](https://www.cas.cn/), [Chinese Academy of Sciences](https://www.cas.cn/) (ICT/CAS)
|
| 33 |
>
|
| 34 |
-
>
|
|
|
|
| 35 |
|
| 36 |
|
| 37 |
**Refer to our [Github Repo](https://github.com/ictnlp/BayLing) for the detailed introduction to BayLing, including deploying BayLing, interacting with BayLing and BayLing's performance.**
|
|
@@ -54,10 +55,22 @@ We would like to express our gratitude to all those who have contributed to BayL
|
|
| 54 |
|
| 55 |
## <a id="Authors">Authors</a>
|
| 56 |
|
| 57 |
-
| [Shaolei Zhang](https://
|
| 58 |
|
| 59 |
-
| Yan Zhou | [Langlin Huang](https://nlp.ict.ac.cn/yjdw/xs/ssyjs/202210/t20221019_52686.html) | Mengyu Bu | Shangtong Gui |
|
| 60 |
|
| 61 |
-
| [Xilin Chen](http://www.ict.cas.cn/sourcedb_2018_ict_cas/cn/jssrck/200909/t20090917_2496595.html) | [Yang Feng \*](https://people.ucas.edu.cn/~yangfeng?language=en) |
|
| 62 |
|
| 63 |
-
## <a id="Citation">Citation</a>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
💬 [**Demo**](http://nlp.ict.ac.cn/bayling/demo): Welcome to apply for a trial of BayLing's online demo (beta version).
|
| 23 |
|
| 24 |
+
📄 [**Paper**](https://arxiv.org/abs/2306.10968): A comprehensive research paper of BayLing.
|
| 25 |
|
| 26 |
🏠 [**Blog**](http://nlp.ict.ac.cn/bayling): BayLing's homepage. You can discover some case of BayLing here.
|
| 27 |
|
|
|
|
| 31 |
|
| 32 |
> BayLing is developed by [NLP Group](http://nlp.ict.ac.cn/) of [Institute of Computing Technology](https://www.cas.cn/), [Chinese Academy of Sciences](https://www.cas.cn/) (ICT/CAS)
|
| 33 |
>
|
| 34 |
+
> BayLing is continuously optimizing 🆙
|
| 35 |
+
> If you have any suggestions, please contact `bayling@ict.ac.cn`. Thanks for your support!
|
| 36 |
|
| 37 |
|
| 38 |
**Refer to our [Github Repo](https://github.com/ictnlp/BayLing) for the detailed introduction to BayLing, including deploying BayLing, interacting with BayLing and BayLing's performance.**
|
|
|
|
| 55 |
|
| 56 |
## <a id="Authors">Authors</a>
|
| 57 |
|
| 58 |
+
| [Shaolei Zhang](https://vily1998.github.io/) | [Qingkai Fang](https://fangqingkai.github.io/) | [Zhuocheng Zhang](https://nlp.ict.ac.cn/yjdw/xs/bsyjs/202210/t20221019_52678.html) | [Zhengrui Ma](https://nlp.ict.ac.cn/yjdw/xs/bsyjs/202210/t20221019_52675.html) |
|
| 59 |
|
| 60 |
+
| [Yan Zhou](https://zhouyan19.github.io/zhouyan/) | [Langlin Huang](https://nlp.ict.ac.cn/yjdw/xs/ssyjs/202210/t20221019_52686.html) | [Mengyu Bu](https://bingo123122121.github.io/) | [Shangtong Gui](https://github.com/GhostofAdam) |
|
| 61 |
|
| 62 |
+
| [Yunji Chen](http://novel.ict.ac.cn/ychen/) | [Xilin Chen](http://www.ict.cas.cn/sourcedb_2018_ict_cas/cn/jssrck/200909/t20090917_2496595.html) | [Yang Feng \*](https://people.ucas.edu.cn/~yangfeng?language=en) |
|
| 63 |
|
| 64 |
+
## <a id="Citation">Citation</a>
|
| 65 |
+
|
| 66 |
+
If our work is helpful for you, please cite as:
|
| 67 |
+
|
| 68 |
+
```
|
| 69 |
+
@article{bayling,
|
| 70 |
+
title={BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Models},
|
| 71 |
+
author={Shaolei Zhang and Qingkai Fang and Zhuocheng Zhang and Zhengrui Ma and Yan Zhou and Langlin Huang and Mengyu Bu and Shangtong Gui and Yunji Chen and Xilin Chen and Yang Feng},
|
| 72 |
+
journal={arXiv preprint arXiv:2306.10968},
|
| 73 |
+
year={2023},
|
| 74 |
+
url={https://arxiv.org/abs/2306.10968}
|
| 75 |
+
}
|
| 76 |
+
```
|