VADRK155 Compactbot commited on
Commit
47ca349
·
1 Parent(s): 8a45d49

Fix card: param count 30m → 36.5m (untied lm_head omitted), weights fp16 → fp32 (#1)

Browse files

- Fix card: param count 30m → 36.5m (untied lm_head omitted), weights fp16 → fp32 (ca22aa45fd5ecf4ba5a1b24e7d29e1c97a57cc3e)


Co-authored-by: Compactbot <Compactbot@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -13,8 +13,8 @@ tags:
13
 
14
  A small GPT model for code generation, trained completely from scratch (custom PyTorch architecture, no `transformers` library involved).
15
 
16
- - **Parameters**: 30m
17
- - **Weights format**: fp16
18
  - **Architecture**: Decoder-only Transformer (custom implementation)
19
  - **Training**: from scratch, no pretrained weights used
20
  - **Language**: writes Python code, all comments and output in English
 
13
 
14
  A small GPT model for code generation, trained completely from scratch (custom PyTorch architecture, no `transformers` library involved).
15
 
16
+ - **Parameters**: 36.5m (36,505,728 learnable; the "30m" figure omitted the untied output head)
17
+ - **Weights format**: fp32
18
  - **Architecture**: Decoder-only Transformer (custom implementation)
19
  - **Training**: from scratch, no pretrained weights used
20
  - **Language**: writes Python code, all comments and output in English