IVPL
/

CaReDiff

@@ -1,98 +0,0 @@
-# CaReDiff Personalised Models, Offline Track (REACT 2026)
-Three personalised models for the offline MAFRG track. Each model is the same
-frozen generic offline backbone plus a Personalised Residual Adapter (PRA)
-trained under a different listener condition. The backbone weights are shared
-by all three and are identical to the generic offline submission.
-## Layout
-```
-offline/
-  backbone/                 frozen generic backbone (shared by all conditions)
-    CausalTransformerDenoiser/checkpoint_120.pth
-    DiffusionPriorNetwork/checkpoint_120.pth
-    EEGPredictionHead/checkpoint_120.pth
-  adapters/
-    personality/ModifierNetwork/checkpoint_best.pth
-    lhfb/ModifierNetwork/checkpoint_best.pth
-    both/ModifierNetwork/checkpoint_best.pth
-```
-The adapter file also contains the fine-tuned EEG head, which overwrites the
-backbone EEG head at load time.
-## Checksums (SHA-256)
-| File | SHA-256 |
-|---|---|
-| backbone/CausalTransformerDenoiser/checkpoint_120.pth | 68faca9700415c949eecbe7bd3e381877a76b5e1b24bdab9c30e6fd5b628faa2 |
-| backbone/DiffusionPriorNetwork/checkpoint_120.pth | d1b66e87f51afd9bb93bdcef1b9e350e6366aa8f995920e400d7e7dd4e299357 |
-| backbone/EEGPredictionHead/checkpoint_120.pth | 750c49999a180cda330b88d771f99d1dca0fd94a810470ea77a45561cfd58780 |
-| adapters/personality/ModifierNetwork/checkpoint_best.pth | 8e0a501237c9b80b8c9e9524bd089fa5ca54ad747bdf9ed65dd97b8d883bf928 |
-| adapters/lhfb/ModifierNetwork/checkpoint_best.pth | 0ddfde5284c580c3cc461006b2b7cd4df73d2715a8700d3838d8d5e5db8eb7f4 |
-| adapters/both/ModifierNetwork/checkpoint_best.pth | 73434669c633bc6384acc9845e62c0c4302c9322be27f04e30005e55dda3ab92 |
-## Conditions
-| Folder | Listener condition | Config value |
-|---|---|---|
-| adapters/personality | Big-Five personality (5-d) | `personality_only` |
-| adapters/lhfb | Listener historical facial behaviour (3DMM) | `3dmm_only` |
-| adapters/both | Both, gated fusion | `3dmm_personality` |
-Training: AdamW, learning rate 2e-4, weight decay 1e-4, gradient clipping 1.0,
-30 epochs, batch size 32, seed 1234, counterfactual listener-swap loss
-(weight 0.5, margin 0.05). The backbone stays frozen throughout.
-## Test performance (MARS test set, official evaluation code, num_gts=10)
-| Condition | FRCorr | FRDist | FRDiv | FRVar | FRRea | FRSyn |
-|---|---|---|---|---|---|---|
-| personality | 0.7786 | 173.63 | 0.1221 | 0.0782 | 50.91 | 48.37 |
-| lhfb | 0.7824 | 173.11 | 0.1200 | 0.0766 | 51.23 | 48.26 |
-| both | 0.7822 | 171.41 | 0.1187 | 0.0761 | 50.82 | 48.28 |
-FRRea is the FID between rendered generated frames and ground-truth frames
-(56,100 frames per side, frame stride 30).
-## How to run
-The source code is in the CaReDiff GitHub repository
-(https://github.com/smu-ivpl/CaReDiff, `personalised/code/`). Example for
-the personality condition (set `PKG` to the absolute path of the
-`personalised` folder containing the checkpoints):
-```bash
-cd code
-python main.py --config-name g2p_delta stage=test task=offline \
-  data_dir=<MARS_ROOT> run_id=eval_offline_personality \
-  trainer.batch_size=4 num_gts=10 \
-  trainer.generic.eval_condition_mode=matched \
-  trainer.generic.eval_eeg=false \
-  trainer.main_model.args.personal_condition_mode=personality_only \
-  resume_id=personality \
-  trainer.ckpt_dir=$PKG/offline/adapters \
-  trainer.pretrained.diffusion_decoder=$PKG/offline/backbone/CausalTransformerDenoiser/checkpoint_120.pth \
-  trainer.pretrained.diffusion_prior=$PKG/offline/backbone/DiffusionPriorNetwork/checkpoint_120.pth \
-  trainer.pretrained.eeg_head_checkpoint=$PKG/offline/backbone/EEGPredictionHead/checkpoint_120.pth
-```
-The adapter is loaded from `<trainer.ckpt_dir>/<resume_id>/ModifierNetwork/`,
-which maps directly onto the `adapters/` layout above. For the other two
-conditions, change `personal_condition_mode` and `resume_id` (`lhfb` or
-`both`) according to the table. The loader verifies that the checkpoint was
-trained with the configured condition mode and stops with an error on a
-mismatch.
-## Notes
-- Large assets shared with the official baseline are not duplicated here.
-  The post-processor EmotionVAE checkpoint (517 MB) is required for
-  evaluation and must be placed at
-  `code/pretrained_models/post_processor/checkpoint.pth`. The PIRender
-  renderer (234 MB) is needed only for FRRea rendering. Take both from the
-  official baseline_react2026 repository.
-- Python dependencies: `code/requirements.txt`.
-- The MARS dataset is not included and must be obtained through the
-  challenge organisers.

personalised/online/README.md DELETED Viewed

@@ -1,108 +0,0 @@
-# CaReDiff Personalised Models, Online Track (REACT 2026)
-Three personalised models for the online MAFRG track. Each model is the same
-frozen generic online backbone plus a Personalised Residual Adapter (PRA)
-trained under a different listener condition. Each online adapter was
-warm-started from its offline counterpart of the same condition and adapted
-with scheduled sampling (probability ramping to 0.5 over 25 epochs). The
-backbone weights are shared by all three and are identical to the generic
-online submission.
-## Layout
-```
-online/
-  backbone/                 frozen generic backbone (shared by all conditions)
-    CausalTransformerDenoiser/checkpoint_120.pth
-    DiffusionPriorNetwork/checkpoint_120.pth
-    EEGPredictionHead/checkpoint_120.pth
-  adapters/
-    personality/ModifierNetwork/checkpoint_best.pth
-    lhfb/ModifierNetwork/checkpoint_best.pth
-    both/ModifierNetwork/checkpoint_best.pth
-```
-The adapter file also contains the fine-tuned EEG head, which overwrites the
-backbone EEG head at load time.
-## Checksums (SHA-256)
-| File | SHA-256 |
-|---|---|
-| backbone/CausalTransformerDenoiser/checkpoint_120.pth | f4fc53506fc94a65e86b52bfe1491669a73ca3429ad0b1ab51c62488854242f0 |
-| backbone/DiffusionPriorNetwork/checkpoint_120.pth | 8b717d619cd37fc793f80f37a4af607bda5e9709c83b82f17916b2467a4380a6 |
-| backbone/EEGPredictionHead/checkpoint_120.pth | 60c7a7ae4e6a233fdb59c0ee1e099daf1158931d876a5f46386a781fa2a52a52 |
-| adapters/personality/ModifierNetwork/checkpoint_best.pth | fad7691aea8c1895a11fcc7d40873d83ea44e0692e6dfb75f736d89cdd9e62d0 |
-| adapters/lhfb/ModifierNetwork/checkpoint_best.pth | c5d8c774774b8a1994be3149c3bb475384326e24a9af64edbf15cda0292da868 |
-| adapters/both/ModifierNetwork/checkpoint_best.pth | 77a744f486c46484dc7a357c484bd5c3582345ea5f6947aea1aa625ba00e660a |
-## Conditions
-| Folder | Listener condition | Config value |
-|---|---|---|
-| adapters/personality | Big-Five personality (5-d) | `personality_only` |
-| adapters/lhfb | Listener historical facial behaviour (3DMM) | `3dmm_only` |
-| adapters/both | Both, gated fusion | `3dmm_personality` |
-Training: AdamW, learning rate 2e-4, weight decay 1e-4, gradient clipping 1.0,
-30 epochs, batch size 32, seed 1234, counterfactual listener-swap loss
-(weight 0.5, margin 0.05). The backbone stays frozen throughout. Generation
-is autoregressive over 30-frame windows; each of the 10 predictions
-conditions on its own previous window.
-## Test performance (MARS test set, official evaluation code, num_gts=10)
-| Condition | FRCorr | FRDist | FRDiv | FRVar | FRRea | FRSyn |
-|---|---|---|---|---|---|---|
-| personality | 0.6485 | 185.17 | 0.1521 | 0.0831 | 50.58 | 47.92 |
-| lhfb | 0.6481 | 191.88 | 0.1521 | 0.0828 | 50.89 | 47.92 |
-| both | 0.6355 | 181.11 | 0.1451 | 0.0790 | 52.09 | 48.16 |
-FRRea is the FID between rendered generated frames and ground-truth frames
-(56,100 frames per side, frame stride 30).
-## How to run
-The source code is in the CaReDiff GitHub repository
-(https://github.com/smu-ivpl/CaReDiff, `personalised/code/`). Example for
-the personality condition (set `PKG` to the absolute path of the
-`personalised` folder containing the checkpoints):
-```bash
-cd code
-python main.py --config-name g2p_delta_online stage=test task=online \
-  data_dir=<MARS_ROOT> run_id=eval_online_personality \
-  trainer.batch_size=4 num_gts=10 \
-  trainer.generic.eval_condition_mode=matched \
-  trainer.generic.eval_eeg=false \
-  trainer.main_model.args.personal_condition_mode=personality_only \
-  trainer.model.diff_model.diffusion_decoder.args.past_l_emotion_drop_prob=0.2 \
-  resume_id=personality \
-  trainer.ckpt_dir=$PKG/online/adapters \
-  trainer.pretrained.diffusion_decoder=$PKG/online/backbone/CausalTransformerDenoiser/checkpoint_120.pth \
-  trainer.pretrained.diffusion_prior=$PKG/online/backbone/DiffusionPriorNetwork/checkpoint_120.pth \
-  trainer.pretrained.eeg_head_checkpoint=$PKG/online/backbone/EEGPredictionHead/checkpoint_120.pth
-```
-The adapter is loaded from `<trainer.ckpt_dir>/<resume_id>/ModifierNetwork/`,
-which maps directly onto the `adapters/` layout above. For the other two
-conditions, change `personal_condition_mode` and `resume_id` (`lhfb` or
-`both`) according to the table. The loader verifies that the checkpoint was
-trained with the configured condition mode and stops with an error on a
-mismatch.
-Keep `past_l_emotion_drop_prob=0.2`. This flag enables the past-listener
-conditioning pathway; the config default disables it and does not reproduce
-the reported numbers.
-## Notes
-- Large assets shared with the official baseline are not duplicated here.
-  The post-processor EmotionVAE checkpoint (517 MB) is required for
-  evaluation and must be placed at
-  `code/pretrained_models/post_processor/checkpoint.pth`. The PIRender
-  renderer (234 MB) is needed only for FRRea rendering. Take both from the
-  official baseline_react2026 repository.
-- Python dependencies: `code/requirements.txt`.
-- The MARS dataset is not included and must be obtained through the
-  challenge organisers.