dimsmauls/code4ml-sq-filtered

Code4ML with SonarQube Maintainability Ratings

Python code snippets from the Code4ML corpus (Zenodo record 7312803) — machine-learning notebooks harvested from Kaggle — each annotated with structural code-quality metrics produced by SonarQube static analysis.

Every code_block was scanned individually and joined back to the source snippet as an audit trail. The maintainability_rating (A–E) is the headline label; supporting raw metrics (code smells, complexity, bugs, technical debt) are kept for transparency.

Dataset summary

Records	2,599,355
Rated (non-empty maintainability_rating)	2,596,565 (99.89%)
Format	single comma-delimited CSV, one header
File	code4ml_sonarqube_rated.csv (~986 MB)
Language	Python

The ~0.11% unrated rows are snippets SonarQube could not analyze (empty or unparseable).

Columns

column	type	description
(unnamed index)	int	row index inherited from source shards
kernel_id	int	Kaggle kernel/notebook id
code_block_id	int	id of the code block within the kernel
code_block	string	the Python source snippet (may span multiple lines)
sqale_rating	float	raw SQALE rating (1.0 = A … 5.0 = E)
maintainability_rating	string	maintainability grade A–E (headline label)
code_smells	int	number of code smells
complexity	int	cyclomatic complexity
cognitive_complexity	int	cognitive complexity
bugs	int	number of bugs detected
sqale_index	int	technical-debt index (minutes)
ncloc	int	non-comment lines of code

Usage

from datasets import load_dataset

ds = load_dataset("<your-username>/code4ml-sonarqube-maintainability", split="train")
print(ds[0]["code_block"], ds[0]["maintainability_rating"])

Filter to a quality floor (keep A–C):

good = ds.filter(lambda r: r["maintainability_rating"] in {"A", "B", "C"})

How it was built

1. Split the source code_blocks.csv into 12 balanced shards (Rust CSV splitter).
2. Scan each snippet through SonarQube via a FastAPI wrapper + sonar-scanner.
3. Join the resulting id;maintainability_rating ratings back onto each shard (add-rating, LEFT JOIN on code_block_id).
4. Merge all 12 rated shards into this single CSV.

Provenance & attribution

• Source corpus: Code4ML — Zenodo record 7312803.
• Original code: the code_block snippets are Kaggle notebook cells, each published under the Apache-2.0 license (see per-kernel links in Code4ML's kernels_meta.csv). Cells were split and stripped of redundant comments/markdown; PEP8 fixes may have been applied. No code-affecting changes were made. See NOTICE.txt.
• This derivative adds only the SonarQube static-analysis columns (maintainability rating + supporting metrics). If you use this dataset, please cite the original Code4ML work as well.

License

The dataset (compilation + SonarQube annotation columns) is released under CC-BY-4.0. See LICENSE.

The underlying code_block snippets are Kaggle notebook code, each under the Apache-2.0 license; that attribution is preserved in NOTICE.txt. Redistributing the Apache-2.0 code within this CC-BY-4.0 dataset is permitted as long as the NOTICE is kept.