kernel-code-embed

A 42.6M-parameter bi-encoder for retrieving Linux kernel C from natural-language queries. Queries and code go through the same encoder, with no prefix or instruction prompt.

512-dim output, mean pooling, L2-normalised — score with cosine similarity.

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("nethunter2023/kernel-code-embed")

query = "how are free pages coalesced into larger blocks"
code = "static inline void __free_one_page(struct page *page, ...) { ... }"

emb = model.encode([query, code], normalize_embeddings=True)
print(emb @ emb.T)          # cosine similarity

Sequence length

max_seq_length is 320, and should not be raised even though max_position_embeddings is 512. Pretraining saw 512 tokens but the retrieval stage trained at 320, so positions 320–511 are undertrained; feeding longer inputs measurably degrades ranking. Chunk longer functions instead.

Results

Two different protocols. Read them separately — the candidate pools differ by two orders of magnitude, so the numbers are not comparable across tables.

1. Open retrieval — the whole kernel

Every .c/.h chunk of Linux v7.1-rc5 as the index (914,554 candidates), N = 400 held-out queries, one correct answer each.

	this model	+ BM25 fusion
recall@1	0.8125	0.9050
recall@5	0.9375	0.9725
recall@10	0.9575	0.9775
recall@50	0.9850	0.9950
MRR	0.8682	0.9374

The model alone ranks the correct function first 81% of the time out of 914,554 candidates (95% CI ±3.8pp at N=400).

The second column is reciprocal-rank fusion of this model with BM25. It is higher, but part of that lift is BM25's, so the 0.8125 figure is the one that belongs to this model.

2. Closed set — against other encoders

N = 2,000 held-out queries, 4,000 candidates, where each distractor is a sibling function from the same source file as the answer.

	params	accuracy@1	NDCG@10
BM25 (lexical)	—	0.7115	0.8297
jina-embeddings-v2-base-code	161M	0.9035	0.9541
this model	42.6M	0.9230	0.9661

Scored with the identical query set, candidate pool and metric code. The general-purpose code embedder was given a 512-token budget — more than this model's 320.

The +2.0pp accuracy@1 margin over jina-embeddings-v2-base-code is significant under a paired McNemar test: p = 0.003, 95% CI on the paired difference +0.7pp to +3.4pp (N = 2,000). A 42.6M domain model edging out a 161M general-purpose one — at 3.8× fewer parameters — is the result worth having.

How the evaluation set was built

This matters for reading the numbers above.

Queries are kernel-doc comments written by kernel developers, not generated by a language model — so they are not distribution-matched to the training data by construction. They are held out from training. The symbol name is stripped from the query: left in, kmalloc_node would appear in both the query and the answer's signature and the task would collapse to identifier matching.

Training used InfoNCE over in-batch negatives (batch size 24, softmax scale 20.0), with hard negatives drawn as sibling functions from the same file.

Limitations

• Domain-specific. Linux kernel C only. Not a general-purpose code or text embedding model; do not expect transfer to other languages or codebases.
• Short context — 320 tokens. Long functions must be chunked.
• kernel-doc phrasing. Queries are developer-written documentation, which is more precise than typical end-user questions. Expect lower accuracy on casual or ambiguous phrasing.
• Single held-out split, no seed variance reported.

License

GPL-2.0, matching the Linux kernel corpus it was trained on. Whether a GPL training corpus propagates to model weights is legally unsettled; this is the conservative reading, chosen deliberately rather than by default. If you need different terms for commercial use, treat that as an open question to resolve with counsel rather than an answered one.