LiLTInvoiceCzechV3

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0643
• Precision: 0.8479
• Recall: 0.8464
• F1: 0.8471
• Accuracy: 0.9875

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 2
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.1
• num_epochs: 40
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	12	2.4543	0.0	0.0	0.0	0.9376
No log	2.0	24	0.3935	0.0	0.0	0.0	0.9473
No log	3.0	36	0.3048	0.0	0.0	0.0	0.9473
No log	4.0	48	0.2729	0.0	0.0	0.0	0.9473
No log	5.0	60	0.2406	0.2267	0.1536	0.1831	0.9501
No log	6.0	72	0.2033	0.2516	0.2713	0.2611	0.9477
No log	7.0	84	0.1644	0.4503	0.4403	0.4452	0.9609
No log	8.0	96	0.1413	0.5264	0.4420	0.4805	0.9643
No log	9.0	108	0.1255	0.5960	0.5563	0.5755	0.9683
No log	10.0	120	0.1264	0.5579	0.5751	0.5664	0.9669
No log	11.0	132	0.1059	0.6198	0.6092	0.6145	0.9700
No log	12.0	144	0.0942	0.6203	0.6689	0.6437	0.9733
No log	13.0	156	0.0820	0.6545	0.6758	0.6650	0.9757
No log	14.0	168	0.0759	0.6805	0.6980	0.6891	0.9790
No log	15.0	180	0.0732	0.6328	0.6911	0.6607	0.9782
No log	16.0	192	0.0766	0.6141	0.6980	0.6534	0.9774
No log	17.0	204	0.0653	0.7132	0.7765	0.7435	0.9828
No log	18.0	216	0.0639	0.7039	0.7747	0.7376	0.9824
No log	19.0	228	0.0673	0.7537	0.7833	0.7682	0.9842
No log	20.0	240	0.0687	0.7684	0.8208	0.7937	0.9839
No log	21.0	252	0.0591	0.7689	0.8003	0.7843	0.9846
No log	22.0	264	0.0602	0.8083	0.8345	0.8212	0.9862
No log	23.0	276	0.0625	0.8162	0.8259	0.8210	0.9860
No log	24.0	288	0.0685	0.7772	0.7918	0.7844	0.9844
No log	25.0	300	0.0610	0.8194	0.8515	0.8351	0.9867
No log	26.0	312	0.0642	0.7885	0.8652	0.8251	0.9852
No log	27.0	324	0.0667	0.7961	0.8328	0.8140	0.9851
No log	28.0	336	0.0650	0.8112	0.8430	0.8268	0.9858
No log	29.0	348	0.0639	0.8249	0.8362	0.8305	0.9866
No log	30.0	360	0.0643	0.8479	0.8464	0.8471	0.9875
No log	31.0	372	0.0631	0.8345	0.8345	0.8345	0.9868
No log	32.0	384	0.0630	0.7978	0.8618	0.8285	0.9858
No log	33.0	396	0.0605	0.8331	0.8515	0.8422	0.9873
No log	34.0	408	0.0638	0.8230	0.8413	0.8321	0.9866
No log	35.0	420	0.0602	0.8367	0.8567	0.8465	0.9877
No log	36.0	432	0.0603	0.8439	0.8396	0.8417	0.9876
No log	37.0	444	0.0615	0.8339	0.8481	0.8409	0.9873
No log	38.0	456	0.0604	0.8344	0.8515	0.8429	0.9874
No log	39.0	468	0.0603	0.8401	0.8515	0.8458	0.9876
No log	40.0	480	0.0602	0.8375	0.8532	0.8453	0.9876

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2