nethunter2023 commited on
Commit
35584e2
·
verified ·
1 Parent(s): 0e4e5d6

results, no methodology

Browse files
Files changed (1) hide show
  1. README.md +29 -4
README.md CHANGED
@@ -17,7 +17,13 @@ language:
17
  # kernel-code-embed
18
 
19
  A 42.6M-parameter bi-encoder for retrieving **Linux kernel C** from
20
- natural-language queries. Queries and code use the same encoder, no prefixes.
 
 
 
 
 
 
21
 
22
  ```python
23
  from sentence_transformers import SentenceTransformer
@@ -31,7 +37,26 @@ emb = model.encode([query, code], normalize_embeddings=True)
31
  print(emb @ emb.T) # cosine similarity
32
  ```
33
 
34
- 512-dim output, mean pooling, L2-normalised — score with cosine similarity.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
35
 
36
- `max_seq_length` is **320** and should not be raised; longer inputs degrade
37
- retrieval. Chunk longer functions instead.
 
17
  # kernel-code-embed
18
 
19
  A 42.6M-parameter bi-encoder for retrieving **Linux kernel C** from
20
+ natural-language queries. Queries and code go through the same encoder, with no
21
+ prefix or instruction prompt.
22
+
23
+ 512-dim output, mean pooling, L2-normalised. `max_seq_length` is **320** and
24
+ should not be raised — longer inputs degrade retrieval. Chunk longer functions.
25
+
26
+ ## Usage
27
 
28
  ```python
29
  from sentence_transformers import SentenceTransformer
 
37
  print(emb @ emb.T) # cosine similarity
38
  ```
39
 
40
+ ## Results
41
+
42
+ Retrieval over the whole kernel — **914,554 candidate chunks**:
43
+
44
+ | | dense | hybrid (+ BM25) |
45
+ |---|---|---|
46
+ | recall@1 | 0.8125 | **0.9050** |
47
+ | recall@5 | 0.9375 | **0.9725** |
48
+ | recall@10 | 0.9575 | **0.9775** |
49
+ | recall@50 | 0.9850 | **0.9950** |
50
+ | MRR | 0.8682 | **0.9374** |
51
+ | median rank | 1 | 1 |
52
+
53
+ The correct function ranks first out of 914,554 candidates 90% of the time.
54
+
55
+ Against a lexical baseline on a held-out set:
56
+
57
+ | | BM25 | this model |
58
+ |---|---|---|
59
+ | accuracy@1 | 0.7115 | **0.9225** |
60
+ | NDCG@10 | 0.8297 | **0.9659** |
61
 
62
+ Trained on Linux kernel C only; not a general-purpose code embedding model.