| 08/03/2026 21:28:59 - INFO - Config: {'model': {'llm_path': '/netscratch/fkallel/models/Qwen3.5-4B/', 'llm_type': 'qwen3.5', 'audio_encoder_path': '/ds-slt/audio_weights/xlsr2_300m.pt', 'audio_encoder_type': 'wav2vec2', 'audio_encoder_source': 'fairseq', 'freeze_audio_encoder': False, 'lora': True, 'lora_rank': 64, 'lora_alpha': 16, 'lora_dropout': 0.05, 'special_tokens': []}, 'data': {'train_path': 'data/my_hir_sdd_binary_train_unique_2.json', 'val_path': 'data/my_hir_sdd_binary_val_unique_2.json', 'prompt_path': 'prompts/train_prompt.json'}, 'train': {'batch_size': 8, 'epochs': 5, 'lr': '1e-4', 'lr_adapter': '1e-5', 'lr_audio': '1e-6', 'lr_lora': '1e-5', 'num_workers': 4, 'seed': 42, 'grad_accum_steps': 1}, 'output_dir': 'output_v6/qwen3.5_4B_lora_42'} |
| 08/03/2026 21:28:59 - INFO - Device: cuda |
| 08/03/2026 21:28:59 - INFO - Loading qwen3.5 Tokenizer & Model from /netscratch/fkallel/models/Qwen3.5-4B/ |
| 08/03/2026 21:29:00 - INFO - Creating wav2vec2 encoder from /ds-slt/audio_weights/xlsr2_300m.pt |
| 08/03/2026 21:29:02 - INFO - Please install tensorboardX: pip install tensorboardX |
| 08/03/2026 21:29:10 - INFO - ============================== |
| 08/03/2026 21:29:10 - INFO - Trainable parameters: |
| 08/03/2026 21:29:10 - INFO - Audio encoder: 317,390,592 (lr=1e-6) |
| 08/03/2026 21:29:10 - INFO - Speech adapter: 2,624,000 (lr=1e-5) |
| 08/03/2026 21:29:10 - INFO - LLM / LoRA: 84,934,656 (lr=1e-5) |
| 08/03/2026 21:29:10 - INFO - Total: 404,949,248 |
| 08/03/2026 21:29:10 - INFO - ============================== |
| 08/03/2026 21:29:43 - INFO - Epoch 1/5 | step 0/4003 | loss 14.2598 |
| 08/03/2026 21:48:41 - INFO - Epoch 1/5 | step 1000/4003 | loss 0.2046 |
| 08/03/2026 22:07:43 - INFO - Epoch 1/5 | step 2000/4003 | loss 0.0018 |
| 08/03/2026 22:26:46 - INFO - Epoch 1/5 | step 3000/4003 | loss 0.0079 |
| 08/03/2026 22:45:46 - INFO - Epoch 1/5 | step 4000/4003 | loss 0.0007 |
| 08/03/2026 22:45:48 - INFO - Epoch 1 train loss 0.1405 (4598.8s) |
| 08/03/2026 22:48:18 - INFO - Epoch 1 val loss 0.0241 |
| 08/03/2026 22:48:21 - INFO - Saved best checkpoint epoch 1 (val_loss=0.0241) |
| 08/03/2026 22:48:24 - INFO - Epoch 2/5 | step 0/4003 | loss 0.0042 |
| 08/03/2026 23:07:21 - INFO - Epoch 2/5 | step 1000/4003 | loss 0.0006 |
| 08/03/2026 23:26:20 - INFO - Epoch 2/5 | step 2000/4003 | loss 0.0298 |
| 08/03/2026 23:45:20 - INFO - Epoch 2/5 | step 3000/4003 | loss 0.0003 |
| 08/04/2026 00:04:18 - INFO - Epoch 2/5 | step 4000/4003 | loss 0.0270 |
| 08/04/2026 00:04:20 - INFO - Epoch 2 train loss 0.0145 (4558.9s) |
| 08/04/2026 00:06:50 - INFO - Epoch 2 val loss 0.0119 |
| 08/04/2026 00:06:52 - INFO - Saved best checkpoint epoch 2 (val_loss=0.0119) |
| 08/04/2026 00:06:56 - INFO - Epoch 3/5 | step 0/4003 | loss 0.2902 |
| 08/04/2026 00:25:57 - INFO - Epoch 3/5 | step 1000/4003 | loss 0.0001 |
| 08/04/2026 00:44:57 - INFO - Epoch 3/5 | step 2000/4003 | loss 0.0003 |
| 08/04/2026 01:04:00 - INFO - Epoch 3/5 | step 3000/4003 | loss 0.0000 |
| 08/04/2026 01:22:59 - INFO - Epoch 3/5 | step 4000/4003 | loss 0.1279 |
| 08/04/2026 01:23:02 - INFO - Epoch 3 train loss 0.0068 (4569.2s) |
| 08/04/2026 01:25:32 - INFO - Epoch 3 val loss 0.0153 |
| 08/04/2026 01:25:36 - INFO - Epoch 4/5 | step 0/4003 | loss 0.0001 |
| 08/04/2026 01:44:39 - INFO - Epoch 4/5 | step 1000/4003 | loss 0.0001 |
| 08/04/2026 02:03:41 - INFO - Epoch 4/5 | step 2000/4003 | loss 0.0002 |
| 08/04/2026 02:22:41 - INFO - Epoch 4/5 | step 3000/4003 | loss 0.0000 |
| 08/04/2026 02:41:44 - INFO - Epoch 4/5 | step 4000/4003 | loss 0.0000 |
| 08/04/2026 02:41:46 - INFO - Epoch 4 train loss 0.0038 (4574.6s) |
| 08/04/2026 02:44:16 - INFO - Epoch 4 val loss 0.0135 |
| 08/04/2026 02:44:20 - INFO - Epoch 5/5 | step 0/4003 | loss 0.0008 |
| 08/04/2026 03:03:24 - INFO - Epoch 5/5 | step 1000/4003 | loss 0.0000 |
| 08/04/2026 03:22:27 - INFO - Epoch 5/5 | step 2000/4003 | loss 0.0000 |
| 08/04/2026 03:41:29 - INFO - Epoch 5/5 | step 3000/4003 | loss 0.0000 |
| 08/04/2026 04:00:34 - INFO - Epoch 5/5 | step 4000/4003 | loss 0.0001 |
| 08/04/2026 04:00:36 - INFO - Epoch 5 train loss 0.0031 (4580.1s) |
| 08/04/2026 04:03:07 - INFO - Epoch 5 val loss 0.0091 |
| 08/04/2026 04:03:09 - INFO - Saved best checkpoint epoch 5 (val_loss=0.0091) |
| 08/04/2026 04:03:09 - INFO - Done. Best val loss 0.0091 at epoch 5 |
|
|
