DeepAnalyze-8B (Q4_K_M GGUF)

DeepAnalyze-8B is a specialized, quantized large language model designed for autonomous in-memory data analytics. It translates natural language instructions into deterministic state-machine logic, executing complex operations on hierarchical spreadsheets, tabular datasets, and SQL databases.

This GGUF distribution is the official backend engine for the DeepAnalyze IPython Extension, an agentic execution harness that handles AST linting, runtime self-repair loops, and zero-copy DuckDB SQL querying directly inside Jupyter/IPython sessions.

---

Model Details

• Base Architecture: DeepSeek-Coder (8 Billion Parameters)
• Training Corpus: 500,000+ instruction-response pairs focusing on advanced Pandas operations, DuckDB, and code-to-chart synthesis.
• Quantization Format: Q4_K_M (4-bit Medium) via llama.cpp.
• Context Length: 8,192 tokens natively.
• Hardware Requirements: Extremely lightweight footprint requiring ~5.5 GB of RAM/VRAM for inference.

Why Q4_K_M?

The 4-bit medium quantization strikes the optimal balance between high-speed token generation and strict syntax adherence. Because the model operates within a closed-loop AST self-repair environment (via the IPython extension), the engine automatically catches and corrects the marginal syntax degradation introduced by 4-bit quantization, yielding maximum performance with minimal hardware overhead.

---

Intended Ecosystem & Use Case

While the model can be loaded into standard chat interfaces (like LM Studio or text-generation-webui), it is not designed for generic conversational chat.

DeepAnalyze is explicitly trained to output executable code payloads encased in specialized <Execute> tags. To utilize the model as intended, it must be paired with its agentic execution harness.

Primary Capabilities (via the Agent Harness):

• Autonomous unravelling of multi-row, non-rectangular ERP exports.
• Zero-copy in-memory querying via DuckDB.
• Automated feature engineering and vector transformations.
• Code-to-chart visualization synthesis.

👉 View the full system architecture and setup guide on GitHub

---

Model Serving (llama.cpp)

Initialize the model locally as an OpenAI-compatible API server using llama-server. This exposes port 8080 for the IPython agent to connect to.

llama-server \
  --hf-repo aboOod3d/deepanalyze-8b \
  --hf-file deepanalyze-8b.gguf \
  --port 8080 \
  -c 8192 \
  -ngl 99

---

Prompt Engineering & Chat Template

DeepAnalyze utilizes specific token formatting to trigger its analytical reasoning pathways. If you are building a custom client or API wrapper, you must strictly adhere to this template.

1. The Analytic Trigger

The prompt string must terminate with the <Analyze> token. This acts as a cognitive trigger, forcing the model out of conversational mode and into strict reasoning and code-generation mode.

<｜User｜> {Your data cleaning instructions} <｜Assistant｜><Analyze>

2. The Execution Sandbox Block

The model emits executable Python/SQL state-machine code strictly within <Execute> delimiters. The host application is expected to parse these tags, validate the Abstract Syntax Tree (AST), and execute the payload.

I have analyzed the schema. Here is the restructuring logic:

<Execute>
import duckdb

query = """
    SELECT 
        category, 
        SUM(revenue) AS total_revenue
    FROM df 
    WHERE status = 'Paid'
    GROUP BY 1
"""
cleaned_df = duckdb.query(query).df()
</Execute>

---

Citation & Attribution

This project builds upon the foundational research and dataset provided by RUC-DataLab.

• Base Weights & Research: RUC-DataLab/DeepAnalyze-8B
• Dataset: RUC-DataLab/DataScience-Instruct-500K
• License: MIT License