Instructions to use N-Bot-Int/OpenElla-StoryWriter-TypeB with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use N-Bot-Int/OpenElla-StoryWriter-TypeB with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("huihui-ai/Llama-3.2-1B-Instruct-abliterated")
model = PeftModel.from_pretrained(base_model, "N-Bot-Int/OpenElla-StoryWriter-TypeB")

Transformers

How to use N-Bot-Int/OpenElla-StoryWriter-TypeB with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="N-Bot-Int/OpenElla-StoryWriter-TypeB")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("N-Bot-Int/OpenElla-StoryWriter-TypeB")
model = AutoModelForCausalLM.from_pretrained("N-Bot-Int/OpenElla-StoryWriter-TypeB")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use N-Bot-Int/OpenElla-StoryWriter-TypeB with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "N-Bot-Int/OpenElla-StoryWriter-TypeB"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "N-Bot-Int/OpenElla-StoryWriter-TypeB",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/N-Bot-Int/OpenElla-StoryWriter-TypeB

SGLang

How to use N-Bot-Int/OpenElla-StoryWriter-TypeB with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "N-Bot-Int/OpenElla-StoryWriter-TypeB" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "N-Bot-Int/OpenElla-StoryWriter-TypeB",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "N-Bot-Int/OpenElla-StoryWriter-TypeB" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "N-Bot-Int/OpenElla-StoryWriter-TypeB",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio new

How to use N-Bot-Int/OpenElla-StoryWriter-TypeB with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for N-Bot-Int/OpenElla-StoryWriter-TypeB to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for N-Bot-Int/OpenElla-StoryWriter-TypeB to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for N-Bot-Int/OpenElla-StoryWriter-TypeB to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="N-Bot-Int/OpenElla-StoryWriter-TypeB",
    max_seq_length=2048,
)

Docker Model Runner
How to use N-Bot-Int/OpenElla-StoryWriter-TypeB with Docker Model Runner:
```
docker model run hf.co/N-Bot-Int/OpenElla-StoryWriter-TypeB
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

This is the Next Experimental OpenElla Architecture, Tested and Proven to Outcompete Minimaid-L2's and L1's

If you loved our Models, then please consider donating and supporting us through Ko-fi! Our Training Capabilities is limited to Google Colab's Free tier So any Help, Donation or Renting is very much appreciated!

OpenElla StoryWriter - TypeB

Introducing OpenElla StoryWriter - TypeB!
OpenElla StoryWriter TypeB is our brand new Model trained on a 1B llama Instruct model, capable of writing good RP responses, without sacrificing Speed or Performance! The New Architecture is trained on HERETICAL and finetuned for RP dataset. Which paves the path for the model's incredible performance compared to other 1B models, and are capable of Generating responses On-par to 3B counterparts

OpenElla StoryWriter - TypeB Base-Model Card Procedure:

TypeB TypeB Model contains 1B params, Type A is yet to be released and will contain 3B params Counterpart. There is NO PLAN to bump the training model to 7B, this is due to Google colab's free tier limit, limiting our training time and the intensity of training!

TypeB is trained using HERETICAL tool, its base model, Llama3.2 1B was taken from HF, and was trained on Heretical to remove as much Wastes as much as possible. However, please bear in mind that the model is UNCENSORED or atmost should be due to heretical training,

N-BOT-INT is NOT LIABLE FOR ANY DAMAGES, MISUSE, OR UNINTENDED OUTCOMES RESULTING FROM USER-GENERATED PROMPT

TypeB is then feeded 600K dataset aimed to increase the performance of the AI model in term of these categories
- ROMANCE
- MYSTERY / DETECTIVE
- SCI-FI
This, alongside the same 600K datasets Handwritten nature, gave the Model length and consistency which is Useful for RP purposes, HOWEVER DUE TO LACK OF PARAMS, EXPECT THE MODEL TO HALLUCINATE

ALL FINETUNING, ACCOUNTING DATASET PREPARATION ASWELL, TOOK 5 DAYS. HERETICAL TRAINING IS EXCLUDED BECAUSE THE HERETICAL HAS BEEN PREPPED LONG TIME AGO
OpenElla StoryWriter - TypeB is Another Instance of Our Smallest Model Yet! if you find any issue, then please don't hesitate to email us at: nexus.networkinteractives@gmail.com about any overfitting, or improvements for the future Model, Once again feel free to Modify the LORA to your likings, However please consider Adding this Page for credits and if you'll increase its Dataset, then please handle it with care and ethical considerations
OpenElla StoryWriter - TypeB is
- Developed by: N-Bot-Int
- License: apache-2.0
MiniMaid-L3 Official Metric Score "Minimaid Instances Vs. OpenElla StoryWriter - TypeB"

"OpenElla StoryWriter - TypeB vs. Several Different AI Models"
- METRIC SCORES ARE HIGHLY SUBJECTIVE, Feel free to use the model and judge them yourselves!
Notice
- For a Good Experience, Please use
  - (PLEASE CALIBRATE THE MODEL DEPENDING ON THE CHARACTER CARD)
  - USE CHATML, the AI MODEL IS FINETUNED TO USE CHATML more than any other format
Detail card:
- Parameter
  - 1 Billion Parameters
  - (Please check your GPU Core, CPU and RAM to see if you can confortably run 1B models)
- Finetuning tool:
- Unsloth AI
- This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
- Fine-tuned Using:
- Google Colab Free Tier