Instructions to use NumbersStation/nsql-6B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use NumbersStation/nsql-6B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="NumbersStation/nsql-6B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("NumbersStation/nsql-6B") model = AutoModelForCausalLM.from_pretrained("NumbersStation/nsql-6B") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use NumbersStation/nsql-6B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "NumbersStation/nsql-6B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NumbersStation/nsql-6B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/NumbersStation/nsql-6B
- SGLang
How to use NumbersStation/nsql-6B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "NumbersStation/nsql-6B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NumbersStation/nsql-6B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "NumbersStation/nsql-6B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NumbersStation/nsql-6B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use NumbersStation/nsql-6B with Docker Model Runner:
docker model run hf.co/NumbersStation/nsql-6B
Example of prompt that GPT-3.5 solved, but nsql-6B did not.
Thanks for a great family of models!
You may want to add the below prompt as training material when developing the nsql further, as nsql-6B (and also nsql-350M) return an invalid SQL-query to it.
Prompt to nsql-6B
"""
text = """CREATE TABLE stadium (
stadium_id number,
location text,
name text,
capacity number,
highest number,
lowest number,
average number
)
CREATE TABLE singer (
singer_id number,
name text,
country text,
song_name text,
song_release_year text,
age number,
is_male others
)
CREATE TABLE concert (
concert_id number,
concert_name text,
theme text,
stadium_id text,
year text
)
CREATE TABLE singer_in_concert (
concert_id number,
singer_id text
)
-- Using valid SQLite, answer the following questions for the tables provided above.
-- Among the artists having concerts in year 2020, which artist has a song whose title equals the name of the stadium on which the concert takes place?
SELECT
"""
Invalid response, as retreived when using nsql-6B:
"""
SELECT T2.name
FROM concert AS T1
JOIN singer_in_concert AS T2 ON T1.concert_id = T2.concert_id
JOIN singer AS T3 ON T2.singer_id = T3.singer_id
WHERE T1.year = 2020 AND T3.song_name = (
SELECT T3.song_name
FROM stadium AS T4
JOIN concert AS T5 ON T4.stadium_id = T5.stadium_id
WHERE T5.year = 2020
);
"""
Valid response, as given by GPT-3.5:
"""
SELECT s.name AS artist_name
FROM singer s
JOIN singer_in_concert sic ON s.singer_id = sic.singer_id
JOIN concert c ON sic.concert_id = c.concert_id
JOIN stadium st ON c.stadium_id = st.stadium_id
WHERE c.year = '2020' AND s.song_name = st.name;
"""
Awesome! Will incorporate it in the future release. Thanks!