Instructions to use my-ai-stack/Stack-2-9-finetuned with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use my-ai-stack/Stack-2-9-finetuned with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="my-ai-stack/Stack-2-9-finetuned")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("my-ai-stack/Stack-2-9-finetuned")
model = AutoModelForCausalLM.from_pretrained("my-ai-stack/Stack-2-9-finetuned", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use my-ai-stack/Stack-2-9-finetuned with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "my-ai-stack/Stack-2-9-finetuned"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "my-ai-stack/Stack-2-9-finetuned",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/my-ai-stack/Stack-2-9-finetuned

SGLang

How to use my-ai-stack/Stack-2-9-finetuned with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "my-ai-stack/Stack-2-9-finetuned" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "my-ai-stack/Stack-2-9-finetuned",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "my-ai-stack/Stack-2-9-finetuned" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "my-ai-stack/Stack-2-9-finetuned",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use my-ai-stack/Stack-2-9-finetuned with Docker Model Runner:
```
docker model run hf.co/my-ai-stack/Stack-2-9-finetuned
```

walidsobhie-code Claude Opus 4.6 commited on Apr 4

Commit

fb43392

1 Parent(s): 2ea2bcc

fix: support input_path in train_lora for JSONL files

Browse files

Handle input_path config option to load directly from JSONL file
and split into train/eval sets. Falls back to train_dir/eval_dir
disk datasets if input_path not provided.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Files changed (1) hide show

stack/training/train_lora.py +19 -5

stack/training/train_lora.py CHANGED Viewed

@@ -233,18 +233,32 @@ def train_lora(
     # Load datasets - handle local disk datasets
     print(f"\n📂 Loading datasets...")
-    train_dir = data_config["train_dir"]
-    eval_dir = data_config["eval_dir"]
     # Check if it's a local disk dataset (saved with save_to_disk)
     # save_to_disk creates dataset_info.json
-    if Path(train_dir).exists() and (Path(train_dir) / "dataset_info.json").exists():
         from datasets import load_from_disk
         train_dataset = load_from_disk(train_dir)
         eval_dataset = load_from_disk(eval_dir)
         print(f"   Loaded pre-processed datasets from disk")
     else:
-        # Try loading as JSONL or other format
         train_dataset = load_dataset(train_dir)
         eval_dataset = load_dataset(eval_dir)
         print(f"   Loaded datasets from: {train_dir}, {eval_dir}")

     # Load datasets - handle local disk datasets
     print(f"\n📂 Loading datasets...")
+    train_dir = data_config.get("train_dir")
+    eval_dir = data_config.get("eval_dir")
+    input_path = data_config.get("input_path")
+    # Check for input_path first (JSONL file)
+    if input_path and not train_dir:
+        print(f"   Loading from input_path: {input_path}")
+        # Load from JSONL file and split
+        raw_dataset = load_dataset("json", data_files=input_path, split="train")
+        train_split = data_config.get("train_split", 0.9)
+        test_split = data_config.get("test_split", 0.1)
+        # Split into train/eval
+        split_dataset = raw_dataset.train_test_split(test_size=test_split, seed=42)
+        train_dataset = split_dataset["train"]
+        eval_dataset = split_dataset["test"]
+        print(f"   Loaded and split JSONL dataset")
     # Check if it's a local disk dataset (saved with save_to_disk)
     # save_to_disk creates dataset_info.json
+    elif train_dir and eval_dir and Path(train_dir).exists() and (Path(train_dir) / "dataset_info.json").exists():
         from datasets import load_from_disk
         train_dataset = load_from_disk(train_dir)
         eval_dataset = load_from_disk(eval_dir)
         print(f"   Loaded pre-processed datasets from disk")
     else:
+        # Try loading as JSONL or other format from directories
         train_dataset = load_dataset(train_dir)
         eval_dataset = load_dataset(eval_dir)
         print(f"   Loaded datasets from: {train_dir}, {eval_dir}")