Create README.md

README.md

@@ -0,0 +1,171 @@
+---
+
+---
+language:
+- code
+tags:
+- code
+- programming-language
+- classification
+- bert
+- text-classification
+license: apache-2.0
+datasets:
+- kaushik-harsh-99/Code-Language-Classification
+metrics:
+- accuracy
+- f1
+- precision
+- recall
+model-index:
+- name: code-lang-bert-small
+  results:
+  - task:
+      type: text-classification
+      name: Programming Language Identification
+    dataset:
+      type: kaushik-harsh-99/Code-Language-Classification
+      name: Code Language Classification
+      split: test
+    metrics:
+    - type: accuracy
+      value: 0.9663
+    - type: f1 (macro)
+      value: 0.9662
+    - type: f1 (weighted)
+      value: 0.9662
+    - type: precision (macro)
+      value: 0.9663
+    - type: recall (macro)
+      value: 0.9663
+---
+
+# Model Card for code-lang-bert-small
+
+A fine-tuned BERT-small model for identifying programming languages from code snippets. The model classifies raw source code into one of 16 supported languages with high accuracy.
+
+## Model Details
+
+### Model Description
+
+This model is a fine-tuned version of `prajjwal1/bert-small` (29M parameters) designed for the task of programming language identification. By analyzing the syntax, keywords, and structural patterns of source code, it accurately predicts the programming language of a given snippet.
+
+- **Developed by:** Pankaj8922
+- **Model type:** Encoder-only Transformer (BERT-small) for sequence classification
+- **Language(s):** 16 programming and markup languages (see below)
+- **License:** Apache 2.0
+- **Finetuned from model:** [prajjwal1/bert-small](https://huggingface.co/prajjwal1/bert-small)
+
+### Supported Languages
+
+Rust, Java, Dart, Python, Go, HTML, JavaScript, Typescript, C, CSS, C#, Markdown, Assembly, Lua, C++, Kotlin
+
+## Uses
+
+### Direct Use
+
+The model is intended for classifying code snippets. It can be used directly with the Hugging Face `pipeline` API or integrated into applications for code tagging, automated documentation, or content filtering.
+
+```python
+from transformers import pipeline
+
+classifier = pipeline(
+    "text-classification",
+    model="Pankaj8922/code-lang-bert-small"
+)
+
+code_snippet = """
+def quicksort(arr):
+    if len(arr) <= 1:
+        return arr
+    pivot = arr[len(arr) // 2]
+    return quicksort(left) + mid + quicksort(right)
+"""
+
+result = classifier(code_snippet)
+print(result)
+# [{'label': 'Python', 'score': 0.99}]
+```
+
+### Out-of-Scope Use
+
+The model is trained to classify full files or substantial code snippets. It may not perform well on:
+- Very short, ambiguous one-liners.
+- Heavily obfuscated or minified code.
+- Code containing multiple languages (e.g., a Python file with extensive embedded SQL).
+- Languages not present in the 16 supported classes.
+
+## Bias, Risks, and Limitations
+
+The model may exhibit biases present in the training data distribution. Languages with syntactically similar constructs (e.g., C and C++, JavaScript and TypeScript) are the most common sources of confusion, as reflected in the confusion matrix. Performance on code from very niche or domain-specific libraries may be lower.
+
+## Training Details
+
+### Training Data
+
+The model was trained on the [Code-Language-Classification](https://huggingface.co/datasets/kaushik-harsh-99/Code-Language-Classification) dataset. The official `train`, `validation`, and `test` splits were used.
+- **Train samples:** 1,600,000
+- **Validation samples:** 32,000
+- **Test samples:** 32,000
+- **Classes:** 16 (perfectly balanced, 2000 samples per class in test set)
+
+### Training Procedure
+
+The BERT-small model was fine-tuned on 2 x T4 GPUs with dynamic padding for efficiency. Training was configured for 5 epochs with early stopping, but was manually stopped after 4 epochs as the model had already converged.
+
+- **Batch size:** 256 (128 per device x 2 GPUs)
+- **Learning rate:** 3e-5
+- **Optimizer:** AdamW (weight decay: 0.01)
+- **Max sequence length:** 512 tokens
+- **Early stopping patience:** 2 epochs
+- **Checkpointing:** Best model based on validation accuracy saved to the Hub.
+
+## Evaluation
+
+The evaluation was performed on the held-out test set of 32,000 samples using the official script provided in the repository.
+
+### Testing Metrics
+
+| Metric           | Value    |
+|------------------|----------|
+| Accuracy         | 96.63%   |
+| Macro F1         | 96.62%   |
+| Weighted F1      | 96.62%   |
+| Macro Precision  | 96.63%   |
+| Macro Recall     | 96.63%   |
+| Eval Loss        | 0.1147   |
+
+### Per-Class Performance
+
+| Language   | Precision | Recall | F1-Score |
+|------------|-----------|--------|----------|
+| Rust       | 0.9885    | 0.9925 | 0.9905   |
+| Java       | 0.9731    | 0.9785 | 0.9758   |
+| Dart       | 0.9772    | 0.9850 | 0.9811   |
+| Python     | 0.9890    | 0.9880 | 0.9885   |
+| Go         | 0.9859    | 0.9800 | 0.9829   |
+| HTML       | 0.9279    | 0.8885 | 0.9078   |
+| JavaScript | 0.8859    | 0.8930 | 0.8894   |
+| TypeScript | 0.9466    | 0.9580 | 0.9523   |
+| C          | 0.9566    | 0.9375 | 0.9470   |
+| CSS        | 0.9728    | 0.9845 | 0.9786   |
+| C#         | 0.9895    | 0.9870 | 0.9882   |
+| Markdown   | 0.9671    | 0.9695 | 0.9683   |
+| Assembly   | 0.9935    | 0.9945 | 0.9940   |
+| Lua        | 0.9885    | 0.9915 | 0.9900   |
+| C++        | 0.9770    | 0.9760 | 0.9765   |
+| Kotlin     | 0.9840    | 0.9870 | 0.9855   |
+
+### Key Observations
+- The model performs exceptionally well on most languages, with 11 of 16 classes achieving an F1-score of 97% or higher.
+- **JavaScript** (F1: 0.89) and **HTML** (F1: 0.91) are the most challenging classes, commonly confused with each other and with TypeScript/CSS.
+- The model is highly confident in distinguishing structurally unique languages like **Assembly** (F1: 0.994) and **Python** (F1: 0.989).
+
+## Environmental Impact
+
+- **Hardware Type:** 2 x NVIDIA T4 GPUs
+- **Hours used:** Approx. 4 epochs of training
+- **Cloud Provider:** Not specified
+- **Compute Region:** Not specified
+
+*Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute).*