Hugging Face's logo

spidercob
/
code-risk-classifier

Text Classification
setfit
Safetensors
sentence-transformers
bert
generated_from_setfit_trainer
Eval Results (legacy)
text-embeddings-inference
Model card Files
xet
 Community
Sumeetgpt commited on
Commit
8a09d4c
·
verified ·
1 Parent(s): d6afa7c

Push model using huggingface_hub.

Browse files
Files changed (4) hide show
  1. README.md +37 -46
  2. config_setfit.json +2 -2
  3. model.safetensors +1 -1
  4. model_head.pkl +1 -1
README.md CHANGED
@@ -5,27 +5,20 @@ tags:
5
  - text-classification
6
  - generated_from_setfit_trainer
7
  widget:
8
- - text: "class TestAuth(unittest.TestCase):\n def setUp(self):\n webhook_secret\
9
- \ = \"test_abcdef1234567890\""
10
- - text: "Analyze this vulnerable_pattern: function solveSum(obj) {\n\tif (\"answer\"\
11
- \ in obj) {\n\t\tdocument.getElementById(\"answer\").innerHTML = obj['answer'];\n\
12
- \t}\n}"
13
- - text: "def handle_request(request):\n sql = f\"DELETE FROM sessions WHERE token='{token}'\"\
14
- \n return result"
15
- - text: 'Analyze this vulnerable_pattern: }
16
-
17
-
18
- $create_tb = "CREATE TABLE users (user_id int(6),first_name varchar(15),last_name
19
- varchar(15), user varchar(15), password varchar(32),avatar varchar(70), last_login
20
- TIMESTAMP, failed_login INT(3), PRIMARY KEY (user_id));";
21
-
22
- if( !mysqli_query($GLOBALS["___mysqli_ston"],'
23
- - text: 'Analyze this clean_code: from __future__ import annotations
24
-
25
-
26
- import importlib.util
27
-
28
- import os'
29
  metrics:
30
  - accuracy
31
  pipeline_tag: text-classification
@@ -43,7 +36,7 @@ model-index:
43
  split: test
44
  metrics:
45
  - type: accuracy
46
- value: 0.9746192893401016
47
  name: Accuracy
48
  ---
49
 
@@ -75,19 +68,19 @@ The model has been trained using an efficient few-shot learning technique that i
75
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
76
 
77
  ### Model Labels
78
- | Label | Examples |
79
- |:-----------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
80
- | TEST_MOCK | <ul><li>"client_secret: str = 'abc123'"</li><li>'# conftest.py\njwt_secret = "MY_API_KEY"'</li><li>"private_key: str = 'yyy'"</li></ul> |
81
- | REAL_SECRET | <ul><li>'Analyze this hardcoded_secret: aws_access_key_id: AKIAI44QH8DHBEXAMPLE'</li><li>"Analyze this hardcoded_secret: // data: {\n // format: 'json',\n // method: 'flickr.interestingness.getList',\n // api_key: '7617adae70159d09ba78cfec73c13be3'\n // },\n\t // dataType: 'jsonp',\n // jsonp: 'json"</li><li>'Analyze this hardcoded_secret: name: "Blake2b with \'Hello, World!\'",\n\t\t\thasher: NewBlake2B(),\n\t\t\tinput: []byte("Hello, World!"),\n\t\t\texpectedHex: "511bc81dde11180838c562c82bb35f3223f46061ebde4a955c27b3f489cf1e03",\n\t\t},\n\t\t{\n\t\t\tname: "Blake2b input at max size",'</li></ul> |
82
- | VULNERABLE_LOGIC | <ul><li>'Analyze this vulnerable_pattern: }\n\n$create_db = "CREATE DATABASE {$_DVWA[ \'db_database\' ]};";\nif( !@mysqli_query($GLOBALS["___mysqli_ston"], $create_db ) ) {\n\tdvwaMessagePush( "Could not create database<br />SQL: " . ((is_object($GLOBALS["___mysqli_ston"])) ? mysqli_error($GLOBALS["___mysqli_ston"]) : (($___mysqli_res = mysqli_connect_error()) ? $___mysqli_res : false)) );\n\tdvwaPageReload();\n}'</li><li>'Analyze this vulnerable_pattern: $user = stripslashes( $user );\n\t$user = mysqli_real_escape_string($GLOBALS["___mysqli_ston"], $user);\n\n\t$pass = $_POST[ \'password\' ];\n\t$pass = stripslashes( $pass );\n\t$pass = mysqli_real_escape_string($GLOBALS["___mysqli_ston"], $pass);\n\t$pass = md5( $pass );'</li><li>'Analyze this vulnerable_pattern: case MYSQL:\n\t\t\t// Check database\n\t\t\t$query = "SELECT first_name, last_name FROM users WHERE user_id = \'$id\' LIMIT 1;";\n\t\t\t$result = mysqli_query($GLOBALS["___mysqli_ston"], $query ) or die( \'<pre>Something went wrong.</pre>\' );\n\n\t\t\t// Get results\n\t\t\twhile( $row = mysqli_fetch_assoc( $result ) ) {'</li></ul> |
83
- | SAFE_CODE | <ul><li>"Analyze this clean_code: [\n 500,\n {\n 'content-type' => content_type,\n 'content-length' => body.bytesize.to_s\n },\n [body]"</li><li>"Analyze this clean_code: it 'should allow switching on the include_subdomains option' do\n mock_app do\n use Rack::Protection::StrictTransport, include_subdomains: true\n run DummyApp\n end"</li><li>"Analyze this clean_code: check(@foo, Rack::Lock, @bar)\n end\n\n it 'works as middleware behind Rack::Lock, with lock disabled' do\n @foo.disable :lock\n check(Rack::Lock, @foo, @bar)\n end"</li></ul> |
84
 
85
  ## Evaluation
86
 
87
  ### Metrics
88
  | Label | Accuracy |
89
  |:--------|:---------|
90
- | **all** | 0.9746 |
91
 
92
  ## Uses
93
 
@@ -107,10 +100,7 @@ from setfit import SetFitModel
107
  # Download from the 🤗 Hub
108
  model = SetFitModel.from_pretrained("setfit_model_id")
109
  # Run inference
110
- preds = model("Analyze this clean_code: from __future__ import annotations
111
-
112
- import importlib.util
113
- import os")
114
  ```
115
 
116
  <!--
@@ -142,11 +132,11 @@ import os")
142
  ### Training Set Metrics
143
  | Training set | Min | Median | Max |
144
  |:-------------|:----|:--------|:----|
145
- | Word count | 1 | 27.6734 | 157 |
146
 
147
  | Label | Training Sample Count |
148
  |:-----------------|:----------------------|
149
- | REAL_SECRET | 67 |
150
  | VULNERABLE_LOGIC | 240 |
151
  | TEST_MOCK | 240 |
152
  | SAFE_CODE | 240 |
@@ -173,17 +163,18 @@ import os")
173
  ### Training Results
174
  | Epoch | Step | Training Loss | Validation Loss |
175
  |:------:|:----:|:-------------:|:---------------:|
176
- | 0.0020 | 1 | 0.0639 | - |
177
- | 0.1016 | 50 | 0.0831 | - |
178
- | 0.2033 | 100 | 0.0272 | - |
179
- | 0.3049 | 150 | 0.0108 | - |
180
- | 0.4065 | 200 | 0.0124 | - |
181
- | 0.5081 | 250 | 0.0125 | - |
182
- | 0.6098 | 300 | 0.0069 | - |
183
- | 0.7114 | 350 | 0.0047 | - |
184
- | 0.8130 | 400 | 0.0048 | - |
185
- | 0.9146 | 450 | 0.0096 | - |
186
- | 1.0 | 492 | - | 0.0246 |
 
187
 
188
  ### Framework Versions
189
  - Python: 3.12.12
 
5
  - text-classification
6
  - generated_from_setfit_trainer
7
  widget:
8
+ - text: "Analyze this clean_code: perms = models.Permission.objects.filter(\n \
9
+ \ codename__in=(\"add_customuser\", \"change_customuser\")\n )\n\
10
+ \ user.user_permissions.add(*perms)\n request = self.factory.get(\"\
11
+ /rand\")\n request.user = user"
12
+ - text: 'Analyze this hardcoded_secret: SECRET_KEY = "super-secret-jwt-key-do-not-share" #
13
+ Django production secret'
14
+ - text: "columns_str = ', '.join(BOOKMARK_COLUMNS)\n curs.execute('SELECT ' + columns_str\
15
+ \ + ' FROM bookmarks ' + where_clause\\"
16
+ - text: 'Analyze this hardcoded_secret: const db = new Pool({ password: ''Pr0duct10n#2024''
17
+ });'
18
+ - text: "Analyze this hardcoded_secret: name: \"Blake2b with 'Hello, World!'\"\
19
+ ,\n\t\t\thasher: NewBlake2B(),\n\t\t\tinput: []byte(\"Hello, World!\"\
20
+ ),\n\t\t\texpectedHex: \"511bc81dde11180838c562c82bb35f3223f46061ebde4a955c27b3f489cf1e03\"\
21
+ ,\n\t\t},\n\t\t{\n\t\t\tname: \"Blake2b input at max size\","
 
 
 
 
 
 
 
22
  metrics:
23
  - accuracy
24
  pipeline_tag: text-classification
 
36
  split: test
37
  metrics:
38
  - type: accuracy
39
+ value: 0.9760765550239234
40
  name: Accuracy
41
  ---
42
 
 
68
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
69
 
70
  ### Model Labels
71
+ | Label | Examples |
72
+ |:-----------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
73
+ | SAFE_CODE | <ul><li>'Analyze this clean_code: },\n "summary": "Read Items",\n "operationId": "read_items_items__get",\n "security": [{"OAuth2PasswordBearer": []}],'</li><li>'Analyze this clean_code: :license: Apache 2.0, see LICENSE for more details.\n"""\n\nfrom __future__ import annotations\n\nimport warnings'</li><li>'Analyze this clean_code: it \'sets rel to stylesheet\' do\n get \'/style\'\n expect(headers[\'Link\']).to include(\'rel="stylesheet"\')\n end\n\n it \'returns html tag\' do'</li></ul> |
74
+ | VULNERABLE_LOGIC | <ul><li>'try {\n await exec(`${ezu} apikey-get --apikey hello`);'</li><li>"Analyze this vulnerable_pattern: $user = $_GET[ 'username' ];\n\n\t// Get password\n\t$pass = $_GET[ 'password' ];\n\t$pass = md5( $pass );\n\n\t// Check the database"</li><li>"Analyze this vulnerable_pattern: if( isset( $_GET[ 'Login' ] ) ) {\n\t// Check Anti-CSRF token\n\tcheckToken( $_REQUEST[ 'user_token' ], $_SESSION[ 'session_token' ], 'index.php' );\n\n\t// Sanitise username input\n\t$user = $_GET[ 'username' ];"</li></ul> |
75
+ | REAL_SECRET | <ul><li>'Analyze this hardcoded_secret: OPENAI_API_KEY=sk-proj-Tz9mK2nP7qR4sT6uW8vE5fG'</li><li>'Analyze this hardcoded_secret: $this->data = array (\n\t\t\t1 => new User (1, "tony", 0, \'1c8bfe8f801d79745c4631d09fff36c82aa37fc4cce4fc946683d7b336b63032\'),\n\t\t\t2 => new User (2, "morph", 1, \'e5326ba4359f77c2623244acb04f6ac35c4dfca330ebcccdf9b734e5b1df90a8\'),\n\t\t\t3 => new User (3, "chas", 1, \'a89237fc1f9dd8d424d8b8b98b890dbc4a817bfde59af17c39debcc4a14c21de\'),\n\t\t);\n\t\t$this->requestMethod = $requestMethod;\n\t\t$this->userId = $userId;'</li><li>"openai_key: ['Use sk-abcdefghijklmnopqrstuvwxyz0123456789 for calls', 'sk-abcdefghijklmnopqrstuvwxyz0123456789'],\n github_token: ['Push with ghp_abcdefghijklmnopqrstuvwxyz0123456789', 'ghp_abcdefghijklmnopqrstuvwxyz0123456789'],"</li></ul> |
76
+ | TEST_MOCK | <ul><li>'Analyze this test_fixture: expected2 = "y=b\'\\\\xe2\\\\xe1\', x=b\'\\\\xe1\\\\xe2\'"\n self.assertIn(txt, (expected1, expected2))\n\n def test_text_kwargs(self):\n txt = str(utils.log_pprint(kwargs={\'x\': \'ŧêßŧ\', \'y\': \'ŧßêŧ\'}))\n expected1 = "x=\'ŧêßŧ\', y=\'ŧßêŧ\'"\n expected2 = "y=\'ŧßêŧ\', x=\'ŧêßŧ\'"'</li><li>'const ACCESS_TOKEN = "AKIAIOSFODNN7EXAMPLE"'</li><li>"Analyze this test_fixture: os.pardir, # /path/to/fboy\n)\n\nMEDIA_ROOT = os.path.join(FACTORY_ROOT, 'tmp_test')\n\nDATABASES = {\n 'default': {"</li></ul> |
77
 
78
  ## Evaluation
79
 
80
  ### Metrics
81
  | Label | Accuracy |
82
  |:--------|:---------|
83
+ | **all** | 0.9761 |
84
 
85
  ## Uses
86
 
 
100
  # Download from the 🤗 Hub
101
  model = SetFitModel.from_pretrained("setfit_model_id")
102
  # Run inference
103
+ preds = model("Analyze this hardcoded_secret: const db = new Pool({ password: 'Pr0duct10n#2024' });")
 
 
 
104
  ```
105
 
106
  <!--
 
132
  ### Training Set Metrics
133
  | Training set | Min | Median | Max |
134
  |:-------------|:----|:--------|:----|
135
+ | Word count | 1 | 22.5870 | 224 |
136
 
137
  | Label | Training Sample Count |
138
  |:-----------------|:----------------------|
139
+ | REAL_SECRET | 113 |
140
  | VULNERABLE_LOGIC | 240 |
141
  | TEST_MOCK | 240 |
142
  | SAFE_CODE | 240 |
 
163
  ### Training Results
164
  | Epoch | Step | Training Loss | Validation Loss |
165
  |:------:|:----:|:-------------:|:---------------:|
166
+ | 0.0019 | 1 | 0.0334 | - |
167
+ | 0.0960 | 50 | 0.0839 | - |
168
+ | 0.1919 | 100 | 0.0608 | - |
169
+ | 0.2879 | 150 | 0.0444 | - |
170
+ | 0.3839 | 200 | 0.0285 | - |
171
+ | 0.4798 | 250 | 0.0267 | - |
172
+ | 0.5758 | 300 | 0.0143 | - |
173
+ | 0.6718 | 350 | 0.0141 | - |
174
+ | 0.7678 | 400 | 0.0116 | - |
175
+ | 0.8637 | 450 | 0.0099 | - |
176
+ | 0.9597 | 500 | 0.0092 | - |
177
+ | 1.0 | 521 | - | 0.0186 |
178
 
179
  ### Framework Versions
180
  - Python: 3.12.12
config_setfit.json CHANGED
@@ -1,9 +1,9 @@
1
  {
 
2
  "labels": [
3
  "REAL_SECRET",
4
  "VULNERABLE_LOGIC",
5
  "TEST_MOCK",
6
  "SAFE_CODE"
7
- ],
8
- "normalize_embeddings": false
9
  }
 
1
  {
2
+ "normalize_embeddings": false,
3
  "labels": [
4
  "REAL_SECRET",
5
  "VULNERABLE_LOGIC",
6
  "TEST_MOCK",
7
  "SAFE_CODE"
8
+ ]
 
9
  }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:07f14a301e379e2bec9340847b50bb389d2d197df126ee6baa1052364a996c02
3
  size 90864192
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:90f75ef471b407684d35375cad34efdf0962e4ead6b1472c3a9d6eef4040c5f7
3
  size 90864192
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9405abda38afa737bd06c713716d4f43881c6914f53a12cad272d1dbccfc734a
3
  size 13191
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9f474dc11bfae94ec29f21ea6cee07d08b605b7b0b55f732d4cb62b0ed937dac
3
  size 13191