Teaching Code Models to Write Secure Code: The SecureCode Collection

When Equifax's 2017 breach exposed 147 million Americans' data, the root cause was a single vulnerable code pattern: unsanitized user input passed directly into an Apache Struts web application. Total cost: $1.4 billion in settlements and remediation. The vulnerable code? Something an LLM might generate today without hesitation.

Code generation models have transformed software development, but they carry a dangerous blind spot: they learn from existing code on the internet, which means they learn existing vulnerabilities. When GitHub Copilot was analyzed in 2021, researchers found it generated vulnerable code 40% of the time across common security scenarios. Your AI coding assistant might be actively introducing SQL injection, authentication bypasses, and command injection vulnerabilities into production systems.

I built the SecureCode model collection to solve this problem. Eight security-aware code models, trained on real-world breach patterns, designed to generate secure code by default.

What Makes SecureCode Different

Most code models learn from GitHub repositories and Stack Overflow answers. They get really good at syntax and common patterns. But security vulnerabilities are common patterns too—just the wrong kind.

SecureCode takes a different approach. Each model in the collection was fine-tuned on the SecureCode-v2 dataset, which contains 841 carefully curated examples built from actual security incidents. When criminals stole $25.6 million from Arup using deepfake social engineering, or when the MOVEit Transfer SQL injection affected 2,100+ organizations with an estimated $9.2 billion impact, those incidents became training examples.

Every conversation in the dataset follows the same structure:

1. Real-world context: The actual breach, CVE, impact in dollars and affected individuals
2. Vulnerable code: The pattern that caused the incident
3. Secure alternative: Production-ready code that prevents the vulnerability
4. Explanation: Why the fix works and how to apply it

This isn't theoretical security training. These are the exact code patterns that cost companies billions of dollars and exposed millions of individuals' data.

The Model Lineup

The collection spans eight models, from 3B to 20B parameters, covering different deployment scenarios:

Edge & Mobile: Llama 3.2 3B SecureCode

Use case: On-device code assistance, mobile development, edge deployment

At just 3 billion parameters, this model runs on smartphones and laptops without GPU acceleration. Security-aware coding doesn't require cloud infrastructure. When you're building a mobile app with limited connectivity, you still get protection against common authentication and input validation mistakes.

Model: llama-3.2-3b-securecode

Balanced Performance: DeepSeek Coder 6.7B & Qwen2.5-Coder 7B & CodeGemma 7B

Use case: Standard development workflows, CI/CD integration, team environments

The 7B models hit the sweet spot for most development teams. Fast enough for real-time suggestions, capable enough for complex security patterns, small enough to run on affordable infrastructure. DeepSeek excels at Python security patterns, Qwen handles multi-language projects exceptionally well, and CodeGemma brings Google's code understanding with security awareness.

Models:

• deepseek-coder-6.7b-securecode
• qwen-coder-7b-securecode
• codegemma-7b-securecode

Production Scale: CodeLlama 13B & StarCoder2 15B

Use case: Large codebases, enterprise deployment, complex security requirements

When you're working with microservices architectures, legacy code migrations, or compliance-critical applications, you need models that understand intricate security context. CodeLlama 13B handles cross-function vulnerability analysis. StarCoder2 15B excels at understanding how authentication flows through multiple layers of an application.

Models:

• codellama-13b-securecode
• starcoder2-15b-securecode

Enterprise Flagship: Granite 20B Code SecureCode

Use case: Maximum capability, regulated industries, critical infrastructure

IBM's Granite architecture with 20 billion parameters, trained on security-critical patterns. This model understands the nuances between secure and insecure stored procedure usage (the pattern behind the MOVEit breach), proper secrets management (the pattern behind the Uber breach), and secure authentication flows (the pattern behind countless API compromises).

When you're building financial services applications, healthcare systems, or government infrastructure, you need code generation that understands both correctness and security at the deepest level.

Model: granite-20b-code-securecode

Quick Start

All models use the same conversation format and are compatible with standard HuggingFace workflows:

from transformers import AutoModelForCausalLM, AutoTokenizer

# Pick your model based on your deployment scenario
model_name = "scthornton/qwen-coder-7b-securecode"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto",
    torch_dtype="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Ask about secure coding patterns
prompt = """### User:
I need to validate user input before using it in a database query. What's the secure approach?

### Assistant:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Each model will explain the vulnerability, show you the secure pattern, and reference the real-world incident that demonstrates why it matters.

Training Approach

Every model was fine-tuned using LoRA (Low-Rank Adaptation) on 4-bit quantized base models, making the training process accessible and the resulting models efficient to deploy. Training parameters were optimized for each architecture:

• Llama/Qwen models: 7 attention projection layers targeted
• StarCoder2: Specialized for code completion patterns
• Granite: GPT-BigCode architecture with 4 projection layers
• DeepSeek/CodeGemma: Optimized for Python/multi-language security

All training used the same high-quality dataset to ensure consistent security knowledge across the entire collection.

Research Foundation

The SecureCode approach is documented in the research paper "SecureCode v2.0: Secure Coding with Large Language Models", which details the dataset construction methodology, training approach, and evaluation results.

Choose Your Model

Starting out? Try Qwen2.5-Coder 7B SecureCode - excellent balance of capability and efficiency.

Resource constrained? Use Llama 3.2 3B SecureCode - runs anywhere, still security-aware.

Enterprise deployment? Deploy Granite 20B Code SecureCode - maximum security understanding, trusted architecture.

Need multiple languages? Choose CodeGemma 7B SecureCode - strong cross-language security patterns.

What's Next

The SecureCode collection represents a new approach to AI-assisted development: security by design, not security as an afterthought. These models understand that SELECT * FROM users WHERE username='${input}' isn't just inefficient SQL—it's the pattern that cost Equifax $1.4 billion.

Every model in this collection was trained on lessons learned from real breaches. When you use them, you're getting security knowledge that companies paid for with incident response costs, regulatory penalties, and reputational damage.

Start building secure code today. Pick your model, integrate it into your workflow, and stop introducing vulnerabilities that will become next year's headlines.

---

Dataset: scthornton/securecode-v2

Models: SecureCode Collection

Research: SecureCode v2.0 Paper

Organization: perfecXion.ai

Built by security researchers, for developers who care about writing secure code.