LH-Tech-AI commited on
Commit
2b11e79
·
verified ·
1 Parent(s): e964767

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +99 -0
README.md ADDED
@@ -0,0 +1,99 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: transformers
4
+ tags:
5
+ - router
6
+ - orchestrator
7
+ - slm
8
+ - edge-computing
9
+ - mixture-of-experts
10
+ - text-generation
11
+ pipeline_tag: text-generation
12
+ model_type: llama
13
+ datasets:
14
+ - SupraLabs/Prompt-Routing-Dataset
15
+ language:
16
+ - en
17
+ base_model:
18
+ - SupraLabs/Supra-1.5-50M-Base-exp
19
+ ---
20
+
21
+ <h1 align="center">Supra-Router-51M · Multi-Task Infrastructure Routing Model</h1>
22
+
23
+ ![logo](https://cdn-uploads.huggingface.co/production/uploads/697f2832c2c5e4daa93cece7/lClDSdp9BkKyv_VGWSQbt.png)
24
+
25
+ <h2 align="center">About the Model</h2>
26
+
27
+ **Supra-Router-51M** is an ultra-lightweight, high-speed infrastructure traffic controller optimized for localized edge orchestration. With only **51.7 million parameters**, this micro-LLM acts as a defensive gateway for multi-model ecosystems, accurately determining when user requests can be processed locally by an Edge SLM or when they must be triaged to a cloud-hosted frontier intelligence layer.
28
+
29
+ The model was built by fine-tuning a pre-trained 51M base on the `SupraLabs/Prompt-Routing-Dataset` (992 rows). Rather than acting as a naive binary classifier, the model uses **Multi-Task Sequence Generation** to map out the underlying properties of a prompt before predicting the final routing token, anchoring its attention heads to robust language and structural logic features.
30
+
31
+ ---
32
+
33
+ ## Multi-Task Decision Sequence
34
+
35
+ To run inference, wrap your user query inside the structural framing tokens used during training (`Task: [Prompt]\nAnalysis: `). The model will output a deterministic, pipe-separated string containing the full telemetry of the prompt's cognitive requirements:
36
+
37
+ ### Expected Output Target Schema:
38
+ ```text
39
+ Domain: [Semantic Field] | Complexity: [1-5] | Math: [True/False] | Code: [True/False] | Route: [small model/big model] | Justification: [Rule-driven infrastructure reasoning]
40
+ ```
41
+
42
+ ## Why this works:
43
+
44
+ By forcing a sub-100M parameter model to calculate the semantic domain, structural complexity, and technical flags before it emits the final Route token, the network effectively runs an internal feature-activation map. This multi-task sequence prevents localized weight collapse and guarantees stable routing boundaries.
45
+
46
+ ## Training Telemetry & Optimization
47
+
48
+ - Dataset Source: SupraLabs/Prompt-Routing-Dataset (992 samples)
49
+ - Training Duration: 5 Epochs
50
+ - Checkpoint Selection: Peak generalization was reached during Epoch 3 (eval_loss: 0.1342). To eliminate late-stage micro-model memorization and validation drift, the training state was automatically rewound and saved at this numerical peak.
51
+ - Precision: bfloat16
52
+ - Hardware Footprint: Optimized sequence processing length of 3840 tokens, ensuring rapid inference execution with negligible CPU/GPU overhead (sub-millisecond generation speeds).
53
+
54
+ ## Inference & Gateway Implementation
55
+
56
+ Use this direct script to test or wrap the model inside a live production orchestrator or FastAPI gateway. It enforces greedy decoding (do_sample=False) for maximum decision stability.
57
+
58
+ ```python
59
+ import torch
60
+ from transformers import AutoModelForCausalLM, AutoTokenizer
61
+
62
+ MODEL_ID = "SupraLabs/Supra-Router-51M"
63
+
64
+ print("[*] Initializing local infrastructure router...")
65
+ tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
66
+ model = AutoModelForCausalLM.from_pretrained(
67
+ MODEL_ID,
68
+ dtype=torch.bfloat16,
69
+ device_map="auto"
70
+ )
71
+ model.eval()
72
+
73
+ # Example prompt showcasing keyword-trap evasion
74
+ user_prompt = "Write a movie script about a chef who gets lost at sea."
75
+
76
+ # Format to match internal SFT attention alignment
77
+ formatted_input = f"Task: {user_prompt}\nAnalysis: "
78
+ inputs = tokenizer(formatted_input, return_tensors="pt").to(model.device)
79
+
80
+ with torch.no_grad():
81
+ outputs = model.generate(
82
+ **inputs,
83
+ max_new_tokens=128,
84
+ do_sample=False,
85
+ pad_token_id=tokenizer.pad_token_id,
86
+ eos_token_id=tokenizer.eos_token_id
87
+ )
88
+
89
+ generated_ids = outputs[0][inputs["input_ids"].shape[1]:]
90
+ print(tokenizer.decode(generated_ids, skip_special_tokens=True).strip())
91
+ ```
92
+
93
+ ## Proven Benchmarks & Defensive Boundaries
94
+
95
+ During edge validation testing, Supra-Router-51M demonstrated robust resilience against adversarial prompt strings:
96
+
97
+ - Keyword Trap Evasion: Successfully identifies semantic context rather than matching tokens. Prompts containing words like "script" or "calculus" are correctly parsed as creative writing (not programming/math code) and routed locally to the small model when complexity is low.
98
+ - Complexity-Driven Safety Net: In instances where programming syntax or technical boundaries are ambiguous (e.g., complex regex or architectural database frames), the model naturally scales its evaluation metrics to Complexity: 3, automatically triggering a big model route override.
99
+ - Deterministic Offloading: Safely captures multi-step logic paths, calculus concepts, and code generation scripts, instantly assigning them to cloud-scale frontier endpoints.