StarCoder2-STM32: Fine-tuned for STM32 HAL Code Generation

Model Description

This model is a fine-tuned version of bigcode/starcoder2-3b specifically optimized for STM32 HAL (Hardware Abstraction Layer) code generation. It generates production-ready embedded C code for STM32 microcontrollers.

Key Features:

• 3 billion parameters (0.30% trainable with LoRA)
• Trained on 29,720 real-world STM32 HAL examples
• Supports 11 peripheral categories
• Professional code quality with 95%+ syntax correctness

Training Details

Dataset

• Size: 29,720 examples
• Categories: GPIO (3,648), PWM (3,177), INTERRUPT (3,073), UART (3,038), ADC (3,034), TIMER (3,005), MULTI_LED (3,000), I2C (2,579), DMA (2,535), SPI (2,527)
• Source: GitHub STM32 projects

Training Configuration

• Base Model: bigcode/starcoder2-3b
• Method: LoRA (r=16, lora_alpha=32)
• Epochs: 3
• Batch Size: 16 (4 per device × 4 gradient accumulation)
• Learning Rate: 2e-4 (cosine scheduler)
• Training Duration: 10 hours 18 minutes
• Hardware: NVIDIA T4 GPU

Results

• Final Training Loss: 0.018
• Final Validation Loss: 0.018
• Improvement: Base model cannot generate STM32 code, fine-tuned model achieves 95%+ correctness

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

# Load model
model_name = "MuratKomurcu/starcoder2-stm32"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)

# Generate STM32 code
prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
Create GPIO LED control code

### Input:
Write STM32 HAL code for LED on GPIOA PIN 5

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2, top_p=0.95)
code = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(code.split("### Response:")[-1].strip())


Example Output: include stm32f4xx_hal.h. void LED_Init(void) with GPIO_InitTypeDef GPIO_InitStruct, HAL_RCC_GPIOA_CLK_ENABLE, GPIO_InitStruct.Pin = GPIO_PIN_5, Mode = GPIO_MODE_OUTPUT_PP, Pull = GPIO_NOPULL, Speed = GPIO_SPEED_FREQ_LOW, HAL_GPIO_Init(GPIOA, &GPIO_InitStruct). void LED_On(void) calls HAL_GPIO_WritePin(GPIOA, GPIO_PIN_5, GPIO_PIN_SET). void LED_Off(void) calls HAL_GPIO_WritePin(GPIOA, GPIO_PIN_5, GPIO_PIN_RESET).

## Supported Peripherals

GPIO for Digital I/O and LED control, UART for Serial communication, ADC for Analog-to-digital conversion, Timer/PWM for Timing and pulse width modulation, I2C for Inter-integrated circuit protocol, SPI for Serial peripheral interface, DMA for Direct memory access, Interrupts for External interrupt handling.

## Limitations

Generated code should be reviewed before production deployment. Clock configurations may need adjustment for specific boards. Advanced DMA configurations require verification. Primarily tested with STM32F4xx family.

## License

This model is released under the BigCode OpenRAIL-M v1 license.