Instructions to use clouditera/secgpt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use clouditera/secgpt with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="clouditera/secgpt")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("clouditera/secgpt")
model = AutoModelForCausalLM.from_pretrained("clouditera/secgpt")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use clouditera/secgpt with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "clouditera/secgpt"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "clouditera/secgpt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/clouditera/secgpt

SGLang

How to use clouditera/secgpt with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "clouditera/secgpt" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "clouditera/secgpt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "clouditera/secgpt" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "clouditera/secgpt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use clouditera/secgpt with Docker Model Runner:
```
docker model run hf.co/clouditera/secgpt
```

secgpt

Commit History

Update README.md

b742806
verified

wygbb commited on Apr 14, 2025

Upload README.md

afaf935
verified

wygbb commited on Apr 14, 2025

Upload folder using huggingface_hub

1970c78
verified

wygbb commited on Apr 14, 2025

Delete quantizer.py

0b103b8
verified

wygbb commited on Apr 14, 2025

Delete special_tokens_map.json

29d10e7
verified

wygbb commited on Apr 14, 2025

Delete modeling_baichuan.py

03f811b
verified

wygbb commited on Apr 14, 2025

Delete generation_config.json

c29cdbc
verified

wygbb commited on Apr 14, 2025

Delete config.json

745a0c7
verified

wygbb commited on Apr 14, 2025

Delete .gitattributes

8b3cfa1
verified

wygbb commited on Apr 14, 2025

Delete tokenizer_config.json

322f0f1
verified

wygbb commited on Apr 14, 2025

Delete tokenizer.model

7692876
verified

wygbb commited on Apr 14, 2025

Delete pytorch_model.bin.index.json

a6e67b8
verified

wygbb commited on Apr 14, 2025

Delete configuration_baichuan.py

9bca9c6
verified

wygbb commited on Apr 14, 2025

Delete README.md

9ceab5a
verified

wygbb commited on Apr 14, 2025

Delete pytorch_model-00003-of-00003.bin

73edb86
verified

wygbb commited on Apr 14, 2025

Delete pytorch_model-00002-of-00003.bin

ac8cd72
verified

wygbb commited on Apr 14, 2025

Delete pytorch_model-00001-of-00003.bin

e7c8cc6
verified

wygbb commited on Apr 14, 2025

update README.md

86f5e6e

clouditera commited on Nov 22, 2023

update

de22c99

clouditera commited on Nov 22, 2023

Update README.md

1675edd

w8ay commited on Nov 20, 2023

Update README.md

1491351

w8ay commited on Nov 17, 2023

Create README.md

f3d9cf5

w8ay commited on Nov 17, 2023

first

a055a7a

clouditera commited on Nov 16, 2023

update

c7e2840

clouditera commited on Nov 22, 2023

init git-lfs config

7019d52

w8ay commited on Nov 22, 2023

Commit History

Update README.md b742806 verified

Upload README.md afaf935 verified

Upload folder using huggingface_hub 1970c78 verified

Delete quantizer.py 0b103b8 verified

Delete special_tokens_map.json 29d10e7 verified

Delete modeling_baichuan.py 03f811b verified

Delete generation_config.json c29cdbc verified

Delete config.json 745a0c7 verified

Delete .gitattributes 8b3cfa1 verified

Delete tokenizer_config.json 322f0f1 verified

Delete tokenizer.model 7692876 verified

Delete pytorch_model.bin.index.json a6e67b8 verified

Delete configuration_baichuan.py 9bca9c6 verified

Delete README.md 9ceab5a verified

Delete pytorch_model-00003-of-00003.bin 73edb86 verified

Delete pytorch_model-00002-of-00003.bin ac8cd72 verified

Delete pytorch_model-00001-of-00003.bin e7c8cc6 verified

update README.md 86f5e6e

update de22c99

Update README.md 1675edd

Update README.md 1491351

Create README.md f3d9cf5

first a055a7a

update c7e2840

init git-lfs config 7019d52

Update README.md

b742806
verified

Upload README.md

afaf935
verified

Upload folder using huggingface_hub

1970c78
verified

Delete quantizer.py

0b103b8
verified

Delete special_tokens_map.json

29d10e7
verified

Delete modeling_baichuan.py

03f811b
verified

Delete generation_config.json

c29cdbc
verified

Delete config.json

745a0c7
verified

Delete .gitattributes

8b3cfa1
verified

Delete tokenizer_config.json

322f0f1
verified

Delete tokenizer.model

7692876
verified

Delete pytorch_model.bin.index.json

a6e67b8
verified

Delete configuration_baichuan.py

9bca9c6
verified

Delete README.md

9ceab5a
verified

Delete pytorch_model-00003-of-00003.bin

73edb86
verified

Delete pytorch_model-00002-of-00003.bin

ac8cd72
verified

Delete pytorch_model-00001-of-00003.bin

e7c8cc6
verified

update README.md

86f5e6e

update

de22c99

Update README.md

1675edd

Update README.md

1491351

Create README.md

f3d9cf5

first

a055a7a

update

c7e2840

init git-lfs config

7019d52