Sync from GitHub

.github/workflows/sync-hf.yml

@@ -0,0 +1,30 @@
+name: Sync to Hugging Face
+
+on:
+  push:
+    branches:
+      - main
+
+
+jobs:
+  sync:
+    runs-on: ubuntu-latest
+
+    steps:
+
+    - name: Checkout
+      uses: actions/checkout@v4
+
+
+    - name: Upload to Hugging Face
+      env:
+        HF_TOKEN: ${{ secrets.HF_TOKEN }}
+      run: |
+
+        pip install huggingface_hub
+
+        hf upload \
+        RL-MIND/LibContinual \
+        . \
+        --repo-type model \
+        --commit-message "Sync from GitHub"

.gitignore

@@ -0,0 +1,17 @@
+*.log
+/log
+/output/
+.vscode
+__pycache__
+*.py[cod]
+.venv-3.8/
+.history
+/data/
+.DS_Store
+
+# Local dependency snapshots kept for environment comparison.
+/old_requirements.txt
+/old_requirements.yaml
+/IDER_EXPERIMENT_REPORT.md
+/run.sh
+/reproduce/ider/scripts/

README.md

@@ -6,6 +6,7 @@
 </div>
 
 ## Introduction
+[![arXiv](https://img.shields.io/badge/arXiv-2512.22029-b31b1b.svg)](https://arxiv.org/abs/2512.22029)
 <div style="text-align: justify">
 LibContinual is an open-source continual learning toolbox based on PyTorch. The framework currently supports PyTorch 1.13+ (compatibility with earlier versions not fully guaranteed) and provides comprehensive implementations of state-of-the-art continual learning algorithms.
 </div>
@@ -27,6 +28,8 @@ LibContinual is an open-source continual learning toolbox based on PyTorch. The
 + [TRGP (ICLR 2022)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/trgp/README.md): Trust Region Gradient Projection for Continual Learning.
 + [API (CVPR 2023)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/api/README.md): Adaptive Plasticity Improvement for Continual Learning.
 + [RanPAC (NeurIPS 2023)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/ranpac/README.md): Random Projections and Pre-trained Models for Continual Learning.
++ [EWC-DR (CVPR 2026)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/ewcdr/README.md): Elastic Weight Consolidation Done Right for Continual Learning.
++ [IDER (ICLR 2026)](reproduce/ider/README.md): IDempotent Experience Replay for Reliable Continual Learning.
 <!-- + [PRAKA (ICCV 2023)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/praka/README.md): Prototype Reminiscence and Augmented Asymmetric Knowledge Aggregation for Non-Exemplar Class-Incremental Learning. -->
 <!-- + [DER (CVPR 2021)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/der/README.md): Dynamically Expandable Representation for Class Incremental Learning. -->
 
@@ -40,7 +43,7 @@ LibContinual is an open-source continual learning toolbox based on PyTorch. The
 + [SD_LoRA (ICLR 2025)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/sd_lora/README.md): Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning
 + [LoRA_Sub_DRS (CVPR 2025)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/lora_sub_drs/README.md): LoRA Subtraction for Drift-Resistant Space in Exemplar-Free Continual Learning
 + [CL-LoRA (CVPR 2025)](https://github.com/RL-VIG/LibContinual/tree/master/reproduce/cl_lora/README.md): Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning
-
++ [Fly-CL (ICLR 2026)](https://github.com/gfyddha/Fly-CL): A Fly-Inspired Framework for Efficient Decorrelation in Pre-trained Model-based Continual Representation Learning.
 <!-- ## Quick Installation
 
 Please refer to [`install.md`](https://libcontinual.readthedocs.io/en/latest/docs/install.html) <br>

config/fly_cl_cifar100.yaml

@@ -0,0 +1,93 @@
+
+dataset: cifar100
+data_root: ./data/cifar100
+image_size: 224
+
+init_cls_num: 10
+inc_cls_num: 10
+total_cls_num: 100
+task_num: 10
+
+
+init_epoch: 1
+epoch: 1
+batch_size: 128
+val_per_epoch: 1
+testing_times: 1
+setting: task-agnostic
+testing_per_task: true
+
+seed: 1993
+n_gpu: 1
+
+class_order: [61, 79, 33, 57, 4, 14, 21, 42, 44, 19,
+              51, 73, 45, 89, 35, 85, 39, 56, 0, 24,
+              65, 29, 9, 18, 13, 95, 41, 80, 96, 32,
+              15, 49, 22, 99, 63, 68, 1, 62, 46, 59,
+              23, 60, 7, 86, 3, 27, 67, 69, 50, 92,
+              31, 98, 76, 84, 97, 93, 43, 16, 30, 83,
+              12, 5, 66, 72, 48, 78, 54, 81, 53, 26,
+              20, 94, 74, 47, 88, 38, 90, 10, 36, 11,
+              40, 52, 64, 87, 91, 6, 8, 55, 77, 82,
+              25, 75, 17, 28, 70, 2, 58, 71, 37, 34]
+
+
+train_trfms:
+  - Resize:
+      size: 224
+      interpolation: BICUBIC
+  - CenterCrop:
+      size: 224
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5, 0.5, 0.5]
+      std: [0.5, 0.5, 0.5]
+
+test_trfms:
+  - Resize:
+      size: 224
+      interpolation: BICUBIC
+  - CenterCrop:
+      size: 224
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5, 0.5, 0.5]
+      std: [0.5, 0.5, 0.5]
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.0
+
+lr_scheduler:
+  name: Constant
+
+backbone:
+  name: timm_backbone
+  kwargs:
+    pretrained: true
+    model_name: vit_base_patch16_224
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 0
+    batch_size: 128
+    strategy: random
+
+classifier:
+  name: FlyCL
+  kwargs:
+    embedding_dim: 768
+    expand_dim: 10000
+    synaptic_degree: 300
+    coding_level: 0.3
+    ridge_lower: 6
+    ridge_upper: 10
+
+    gcv_backend: dual_eigh
+    disable_cudnn: true
+    total_cls_num: 100
+    init_cls_num: 10
+    inc_cls_num: 10
+    mask_unseen_classes: false

config/fly_cl_cub200.yaml

@@ -0,0 +1,101 @@
+dataset: cub200
+data_root: ./data/cub200
+image_size: 224
+
+init_cls_num: 20
+inc_cls_num: 20
+total_cls_num: 200
+task_num: 10
+
+init_epoch: 1
+epoch: 1
+batch_size: 128
+val_per_epoch: 1
+testing_times: 1
+setting: task-agnostic
+testing_per_task: true
+
+seed: 2023
+n_gpu: 1
+
+class_order: [98, 179, 114, 99, 81, 156, 146, 86, 162, 25,
+              31, 166, 77, 178, 136, 183, 135, 187, 45, 59,
+              181, 163, 4, 53, 92, 17, 159, 32, 9, 51,
+              39, 122, 197, 185, 87, 128, 16, 19, 137, 118,
+              127, 7, 189, 193, 60, 27, 148, 18, 174, 35,
+              196, 168, 52, 46, 188, 54, 74, 69, 30, 14,
+              95, 121, 169, 75, 22, 129, 103, 80, 70, 195,
+              72, 34, 40, 106, 175, 56, 171, 125, 198, 71,
+              104, 29, 152, 65, 44, 120, 82, 15, 2, 150,
+              184, 105, 0, 36, 117, 88, 190, 21, 113, 172,
+              55, 116, 89, 12, 144, 153, 62, 119, 139, 11,
+              83, 173, 42, 140, 8, 93, 61, 84, 123, 134,
+              143, 49, 157, 3, 91, 133, 115, 177, 24, 96,
+              85, 164, 109, 110, 26, 5, 37, 76, 167, 57,
+              23, 79, 1, 73, 63, 102, 124, 186, 165, 43,
+              158, 176, 67, 149, 100, 78, 151, 108, 111, 192,
+              33, 13, 180, 170, 130, 10, 64, 161, 94, 154,
+              132, 191, 48, 145, 101, 50, 66, 126, 141, 155,
+              28, 160, 68, 112, 38, 90, 147, 47, 182, 199,
+              20, 58, 107, 131, 142, 194, 97, 138, 6, 41]
+
+train_trfms:
+  - Resize:
+      size: 256
+      interpolation: BICUBIC
+  - CenterCrop:
+      size: 224
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5, 0.5, 0.5]
+      std: [0.5, 0.5, 0.5]
+
+test_trfms:
+  - Resize:
+      size: 256
+      interpolation: BICUBIC
+  - CenterCrop:
+      size: 224
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5, 0.5, 0.5]
+      std: [0.5, 0.5, 0.5]
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.0
+
+lr_scheduler:
+  name: Constant
+
+backbone:
+  name: timm_backbone
+  kwargs:
+    pretrained: true
+    model_name: vit_base_patch16_224
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 0
+    batch_size: 128
+    strategy: random
+
+classifier:
+  name: FlyCL
+  kwargs:
+    embedding_dim: 768
+    expand_dim: 10000
+    synaptic_degree: 300
+    coding_level: 0.3
+    ridge_lower: 6
+    ridge_upper: 10
+    gcv_backend: dual_eigh
+    disable_cudnn: true
+    use_projection: true
+    use_ridge: true
+    total_cls_num: 200
+    init_cls_num: 20
+    inc_cls_num: 20
+    mask_unseen_classes: false

config/zz_EWCDR/ewcdr-resnet18-cifar100-b10-10-10.yaml

@@ -0,0 +1,86 @@
+dataset: &dataset binary_cifar100
+data_root: ./data/binary_cifar100
+shuffle: True
+
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+task_num: &task_num 10
+image_size: &image_size 32
+
+init_epoch: 200
+epoch: 180
+val_per_epoch: 5
+batch_size: 128
+num_workers: 4
+testing_times: 1
+seed: 1993
+
+train_trfms:
+  - RandomCrop:
+      size: *image_size
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ColorJitter:
+      brightness: 0.24705882352941178
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+init_optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0005
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0002
+
+init_lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [60, 120, 170]
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [70, 120, 150]
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: *total_cls_num
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 0
+    batch_size: 128
+    strategy: herding
+
+classifier:
+  name: EWCDR
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    lamda: 10000
+    omega_max: 0.0001

config/zz_EWCDR/ewcdr-resnet18-cifar100-b20-20-5.yaml

@@ -0,0 +1,86 @@
+dataset: &dataset binary_cifar100
+data_root: ./data/binary_cifar100
+shuffle: True
+
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+task_num: &task_num 5
+image_size: &image_size 32
+
+init_epoch: 200
+epoch: 180
+val_per_epoch: 5
+batch_size: 128
+num_workers: 4
+testing_times: 1
+seed: 1993
+
+train_trfms:
+  - RandomCrop:
+      size: *image_size
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ColorJitter:
+      brightness: 0.24705882352941178
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+init_optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0005
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0002
+
+init_lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [60, 120, 170]
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [70, 120, 150]
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: *total_cls_num
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 0
+    batch_size: 128
+    strategy: herding
+
+classifier:
+  name: EWCDR
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    lamda: 10000
+    omega_max: 0.0001

config/zz_EWCDR/ewcdr-resnet18-cifar100-b40-3-21.yaml

@@ -0,0 +1,86 @@
+dataset: &dataset binary_cifar100
+data_root: ./data/binary_cifar100
+shuffle: True
+
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 40
+inc_cls_num: &inc_cls_num 3
+task_num: &task_num 21
+image_size: &image_size 32
+
+init_epoch: 200
+epoch: 180
+val_per_epoch: 5
+batch_size: 128
+num_workers: 4
+testing_times: 1
+seed: 1993
+
+train_trfms:
+  - RandomCrop:
+      size: *image_size
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ColorJitter:
+      brightness: 0.24705882352941178
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+init_optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0005
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0002
+
+init_lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [60, 120, 170]
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [70, 120, 150]
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: *total_cls_num
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 0
+    batch_size: 128
+    strategy: herding
+
+classifier:
+  name: EWCDR
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    lamda: 10000
+    omega_max: 0.0001

config/zz_EWCDR/ewcdr-resnet18-cifar100-b5-5-20.yaml

@@ -0,0 +1,86 @@
+dataset: &dataset binary_cifar100
+data_root: ./data/binary_cifar100
+shuffle: True
+
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 5
+inc_cls_num: &inc_cls_num 5
+task_num: &task_num 20
+image_size: &image_size 32
+
+init_epoch: 200
+epoch: 180
+val_per_epoch: 5
+batch_size: 128
+num_workers: 4
+testing_times: 1
+seed: 1993
+
+train_trfms:
+  - RandomCrop:
+      size: *image_size
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ColorJitter:
+      brightness: 0.24705882352941178
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+init_optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0005
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0002
+
+init_lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [60, 120, 170]
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [70, 120, 150]
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: *total_cls_num
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 0
+    batch_size: 128
+    strategy: herding
+
+classifier:
+  name: EWCDR
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    lamda: 10000
+    omega_max: 0.0001

config/zz_EWCDR/ewcdr-resnet18-cifar100-b50-10-6.yaml

@@ -0,0 +1,86 @@
+dataset: &dataset binary_cifar100
+data_root: ./data/binary_cifar100
+shuffle: True
+
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 50
+inc_cls_num: &inc_cls_num 10
+task_num: &task_num 6
+image_size: &image_size 32
+
+init_epoch: 200
+epoch: 180
+val_per_epoch: 5
+batch_size: 128
+num_workers: 4
+testing_times: 1
+seed: 1993
+
+train_trfms:
+  - RandomCrop:
+      size: *image_size
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ColorJitter:
+      brightness: 0.24705882352941178
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+init_optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0005
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0002
+
+init_lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [60, 120, 170]
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [70, 120, 150]
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: *total_cls_num
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 0
+    batch_size: 128
+    strategy: herding
+
+classifier:
+  name: EWCDR
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    lamda: 10000
+    omega_max: 0.0001

config/zz_EWCDR/ewcdr-resnet18-cifar100-b50-5-11.yaml

@@ -0,0 +1,86 @@
+dataset: &dataset binary_cifar100
+data_root: ./data/binary_cifar100
+shuffle: True
+
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 50
+inc_cls_num: &inc_cls_num 5
+task_num: &task_num 11
+image_size: &image_size 32
+
+init_epoch: 200
+epoch: 180
+val_per_epoch: 5
+batch_size: 128
+num_workers: 4
+testing_times: 1
+seed: 1993
+
+train_trfms:
+  - RandomCrop:
+      size: *image_size
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ColorJitter:
+      brightness: 0.24705882352941178
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+init_optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0005
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0002
+
+init_lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [60, 120, 170]
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [70, 120, 150]
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: *total_cls_num
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 0
+    batch_size: 128
+    strategy: herding
+
+classifier:
+  name: EWCDR
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    lamda: 10000
+    omega_max: 0.0001

config/zz_EWCDR/ewcdr-resnet18-tinyimagenet-b10-10-20.yaml

@@ -0,0 +1,27 @@
+includes:
+  - zz_EWCDR/tinyimagenet_common.yaml
+
+init_cls_num: 10
+inc_cls_num: 10
+task_num: 20
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: 200
+    args:
+      dataset: tiny-imagenet
+      init_cls_num: 10
+      inc_cls_num: 10
+
+classifier:
+  name: EWCDR
+  kwargs:
+    num_class: 200
+    feat_dim: 512
+    init_cls_num: 10
+    inc_cls_num: 10
+    task_num: 20
+    lamda: 10000
+    omega_max: 0.0001
+

config/zz_EWCDR/ewcdr-resnet18-tinyimagenet-b100-10-11.yaml

@@ -0,0 +1,27 @@
+includes:
+  - zz_EWCDR/tinyimagenet_common.yaml
+
+init_cls_num: 100
+inc_cls_num: 10
+task_num: 11
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: 200
+    args:
+      dataset: tiny-imagenet
+      init_cls_num: 100
+      inc_cls_num: 10
+
+classifier:
+  name: EWCDR
+  kwargs:
+    num_class: 200
+    feat_dim: 512
+    init_cls_num: 100
+    inc_cls_num: 10
+    task_num: 11
+    lamda: 10000
+    omega_max: 0.0001
+

config/zz_EWCDR/ewcdr-resnet18-tinyimagenet-b100-20-6.yaml

@@ -0,0 +1,27 @@
+includes:
+  - zz_EWCDR/tinyimagenet_common.yaml
+
+init_cls_num: 100
+inc_cls_num: 20
+task_num: 6
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: 200
+    args:
+      dataset: tiny-imagenet
+      init_cls_num: 100
+      inc_cls_num: 20
+
+classifier:
+  name: EWCDR
+  kwargs:
+    num_class: 200
+    feat_dim: 512
+    init_cls_num: 100
+    inc_cls_num: 20
+    task_num: 6
+    lamda: 10000
+    omega_max: 0.0001
+

config/zz_EWCDR/ewcdr-resnet18-tinyimagenet-b100-5-21.yaml

@@ -0,0 +1,27 @@
+includes:
+  - zz_EWCDR/tinyimagenet_common.yaml
+
+init_cls_num: 100
+inc_cls_num: 5
+task_num: 21
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: 200
+    args:
+      dataset: tiny-imagenet
+      init_cls_num: 100
+      inc_cls_num: 5
+
+classifier:
+  name: EWCDR
+  kwargs:
+    num_class: 200
+    feat_dim: 512
+    init_cls_num: 100
+    inc_cls_num: 5
+    task_num: 21
+    lamda: 10000
+    omega_max: 0.0001
+

config/zz_EWCDR/ewcdr-resnet18-tinyimagenet-b20-20-10.yaml

@@ -0,0 +1,27 @@
+includes:
+  - zz_EWCDR/tinyimagenet_common.yaml
+
+init_cls_num: 20
+inc_cls_num: 20
+task_num: 10
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: 200
+    args:
+      dataset: tiny-imagenet
+      init_cls_num: 20
+      inc_cls_num: 20
+
+classifier:
+  name: EWCDR
+  kwargs:
+    num_class: 200
+    feat_dim: 512
+    init_cls_num: 20
+    inc_cls_num: 20
+    task_num: 10
+    lamda: 10000
+    omega_max: 0.0001
+

config/zz_EWCDR/ewcdr-resnet18-tinyimagenet-b40-40-5.yaml

@@ -0,0 +1,27 @@
+includes:
+  - zz_EWCDR/tinyimagenet_common.yaml
+
+init_cls_num: 40
+inc_cls_num: 40
+task_num: 5
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: 200
+    args:
+      dataset: tiny-imagenet
+      init_cls_num: 40
+      inc_cls_num: 40
+
+classifier:
+  name: EWCDR
+  kwargs:
+    num_class: 200
+    feat_dim: 512
+    init_cls_num: 40
+    inc_cls_num: 40
+    task_num: 5
+    lamda: 10000
+    omega_max: 0.0001
+

config/zz_EWCDR/tinyimagenet_common.yaml

@@ -0,0 +1,42 @@
+dataset: tiny-imagenet
+data_root: ./data
+shuffle: true
+pin_memory: true
+
+total_cls_num: 200
+image_size: 64
+init_epoch: 200
+epoch: 180
+val_per_epoch: 5
+batch_size: 128
+num_workers: 8
+testing_times: 1
+seed: 1993
+
+train_trfms:
+  - RandomResizedCrop: {size: 64}
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize: {mean: [0.485, 0.456, 0.406], std: [0.229, 0.224, 0.225]}
+test_trfms:
+  - Resize: {size: 64}
+  - CenterCrop: {size: 56}
+  - ToTensor: {}
+  - Normalize: {mean: [0.485, 0.456, 0.406], std: [0.229, 0.224, 0.225]}
+
+init_optimizer:
+  name: SGD
+  kwargs: {lr: 0.1, momentum: 0.9, weight_decay: 0.0005}
+optimizer:
+  name: SGD
+  kwargs: {lr: 0.1, momentum: 0.9, weight_decay: 0.0002}
+init_lr_scheduler:
+  name: MultiStepLR
+  kwargs: {gamma: 0.1, milestones: [60, 120, 170]}
+lr_scheduler:
+  name: MultiStepLR
+  kwargs: {gamma: 0.1, milestones: [70, 120, 150]}
+
+buffer:
+  name: LinearBuffer
+  kwargs: {buffer_size: 0, batch_size: 128, strategy: herding}

config/zz_IDER/er_cifar100_buf2000.yaml

@@ -0,0 +1,76 @@
+dataset: &dataset binary_cifar100
+data_root: ./datasets
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+task_num: 10
+image_size: 32
+save_path: ./
+
+train_trfms:
+  - RandomCrop:
+      size: 32
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+seed: 0
+deterministic: True
+device_ids: auto
+n_gpu: 1
+num_workers: 4
+pin_memory: False
+
+batch_size: 32
+init_epoch: 50
+epoch: 50
+val_per_epoch: 50
+testing_times: 1
+testing_per_task: True
+setting: task-agnostic
+eval_with_test: True
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.0
+    weight_decay: 0.0
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [35, 45]
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: *total_cls_num
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 2000
+    batch_size: 32
+    strategy: random
+
+classifier:
+  name: PaperER
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    minibatch_size: 32
+    buffer_storage: tensor

config/zz_IDER/er_cifar100_buf500.yaml

@@ -0,0 +1,76 @@
+dataset: &dataset binary_cifar100
+data_root: ./datasets
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+task_num: 10
+image_size: 32
+save_path: ./
+
+train_trfms:
+  - RandomCrop:
+      size: 32
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+seed: 0
+deterministic: True
+device_ids: auto
+n_gpu: 1
+num_workers: 4
+pin_memory: False
+
+batch_size: 32
+init_epoch: 50
+epoch: 50
+val_per_epoch: 50
+testing_times: 1
+testing_per_task: True
+setting: task-agnostic
+eval_with_test: True
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.0
+    weight_decay: 0.0
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [35, 45]
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: *total_cls_num
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 500
+    batch_size: 32
+    strategy: random
+
+classifier:
+  name: PaperER
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    minibatch_size: 32
+    buffer_storage: tensor

config/zz_IDER/er_cifar10_buf200.yaml

@@ -0,0 +1,74 @@
+dataset: &dataset binary_cifar10
+data_root: ./datasets
+total_cls_num: &total_cls_num 10
+init_cls_num: &init_cls_num 2
+inc_cls_num: &inc_cls_num 2
+task_num: 5
+image_size: 32
+save_path: ./
+
+train_trfms:
+  - RandomCrop:
+      size: 32
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4914, 0.4822, 0.4465]
+      std: [0.2470, 0.2435, 0.2615]
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4914, 0.4822, 0.4465]
+      std: [0.2470, 0.2435, 0.2615]
+
+seed: 0
+deterministic: True
+device_ids: auto
+n_gpu: 1
+num_workers: 4
+pin_memory: False
+
+batch_size: 32
+init_epoch: 50
+epoch: 50
+val_per_epoch: 50
+testing_times: 1
+testing_per_task: True
+setting: task-agnostic
+eval_with_test: True
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.0
+    weight_decay: 0.0
+
+lr_scheduler:
+  name: Constant
+  kwargs: {}
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: *total_cls_num
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 200
+    batch_size: 32
+    strategy: random
+
+classifier:
+  name: PaperER
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    minibatch_size: 32
+    buffer_storage: tensor

config/zz_IDER/er_cifar10_buf500.yaml

@@ -0,0 +1,74 @@
+dataset: &dataset binary_cifar10
+data_root: ./datasets
+total_cls_num: &total_cls_num 10
+init_cls_num: &init_cls_num 2
+inc_cls_num: &inc_cls_num 2
+task_num: 5
+image_size: 32
+save_path: ./
+
+train_trfms:
+  - RandomCrop:
+      size: 32
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4914, 0.4822, 0.4465]
+      std: [0.2470, 0.2435, 0.2615]
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4914, 0.4822, 0.4465]
+      std: [0.2470, 0.2435, 0.2615]
+
+seed: 0
+deterministic: True
+device_ids: auto
+n_gpu: 1
+num_workers: 4
+pin_memory: False
+
+batch_size: 32
+init_epoch: 50
+epoch: 50
+val_per_epoch: 50
+testing_times: 1
+testing_per_task: True
+setting: task-agnostic
+eval_with_test: True
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.0
+    weight_decay: 0.0
+
+lr_scheduler:
+  name: Constant
+  kwargs: {}
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: *total_cls_num
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 500
+    batch_size: 32
+    strategy: random
+
+classifier:
+  name: PaperER
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    minibatch_size: 32
+    buffer_storage: tensor

config/zz_IDER/er_tinyimagenet_buf4000.yaml

@@ -0,0 +1,76 @@
+dataset: &dataset processed_tinyimg
+data_root: ./datasets/TINYIMG
+total_cls_num: &total_cls_num 200
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+task_num: 10
+image_size: 64
+save_path: ./
+
+train_trfms:
+  - RandomCrop:
+      size: 64
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4802, 0.4480, 0.3975]
+      std: [0.2770, 0.2691, 0.2821]
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4802, 0.4480, 0.3975]
+      std: [0.2770, 0.2691, 0.2821]
+
+seed: 0
+deterministic: True
+device_ids: auto
+n_gpu: 1
+num_workers: 4
+pin_memory: False
+
+batch_size: 32
+init_epoch: 50
+epoch: 50
+val_per_epoch: 50
+testing_times: 1
+testing_per_task: True
+setting: task-agnostic
+eval_with_test: True
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.0
+    weight_decay: 0.0
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [35, 45]
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: *total_cls_num
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 4000
+    batch_size: 32
+    strategy: random
+
+classifier:
+  name: PaperER
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    minibatch_size: 32
+    buffer_storage: tensor

config/zz_IDER/er_tinyimagenet_buf500.yaml

@@ -0,0 +1,76 @@
+dataset: &dataset processed_tinyimg
+data_root: ./datasets/TINYIMG
+total_cls_num: &total_cls_num 200
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+task_num: 10
+image_size: 64
+save_path: ./
+
+train_trfms:
+  - RandomCrop:
+      size: 64
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4802, 0.4480, 0.3975]
+      std: [0.2770, 0.2691, 0.2821]
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4802, 0.4480, 0.3975]
+      std: [0.2770, 0.2691, 0.2821]
+
+seed: 0
+deterministic: True
+device_ids: auto
+n_gpu: 1
+num_workers: 4
+pin_memory: False
+
+batch_size: 32
+init_epoch: 100
+epoch: 100
+val_per_epoch: 100
+testing_times: 1
+testing_per_task: True
+setting: task-agnostic
+eval_with_test: True
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.03
+    momentum: 0.0
+    weight_decay: 0.0
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [35, 60, 75]
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: *total_cls_num
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 500
+    batch_size: 32
+    strategy: random
+
+classifier:
+  name: PaperER
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    minibatch_size: 32
+    buffer_storage: tensor

config/zz_IDER/ider_cifar100_buf2000.yaml

@@ -0,0 +1,91 @@
+dataset: &dataset binary_cifar100
+data_root: ./datasets
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+task_num: &task_num 10
+image_size: 32
+save_path: ./
+
+train_trfms:
+  - RandomCrop:
+      size: 32
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+seed: 0
+deterministic: True
+device_ids: auto
+n_gpu: 1
+num_workers: 4
+pin_memory: False
+
+batch_size: 32
+init_epoch: 50
+epoch: 50
+val_per_epoch: 50
+testing_times: 1
+testing_per_task: True
+setting: task-agnostic
+eval_with_test: True
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.03
+    momentum: 0.0
+    weight_decay: 0.0
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [35, 45]
+
+backbone:
+  name: resnet18_id2
+  kwargs:
+    num_classes: *total_cls_num
+    nf: 64
+    use_cos: False
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 2000
+    batch_size: 32
+    strategy: random
+
+classifier:
+  name: IDER
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    weighta: 0.5
+    weightb: 0.4
+    weightc: 0.5
+    weightmask: 1.0
+    refine_inference: False
+    mask_unseen: False
+    online_buffer_update: True
+    class_balance: True
+    print_buffer_stats: True
+    buffer_update_after_step: True
+    buffer_storage: tensor
+    ce_detach_signal: False
+    distill_detach_signal: False
+    old_model_eval: False
+    freeze_old_model: True

config/zz_IDER/ider_cifar100_buf500.yaml

@@ -0,0 +1,91 @@
+dataset: &dataset binary_cifar100
+data_root: ./datasets
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+task_num: &task_num 10
+image_size: 32
+save_path: ./
+
+train_trfms:
+  - RandomCrop:
+      size: 32
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+seed: 0
+deterministic: True
+device_ids: auto
+n_gpu: 1
+num_workers: 4
+pin_memory: False
+
+batch_size: 32
+init_epoch: 50
+epoch: 50
+val_per_epoch: 50
+testing_times: 1
+testing_per_task: True
+setting: task-agnostic
+eval_with_test: True
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.03
+    momentum: 0.0
+    weight_decay: 0.0
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [35, 45]
+
+backbone:
+  name: resnet18_id2
+  kwargs:
+    num_classes: *total_cls_num
+    nf: 64
+    use_cos: False
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 500
+    batch_size: 32
+    strategy: random
+
+classifier:
+  name: IDER
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    weighta: 0.5
+    weightb: 0.4
+    weightc: 0.5
+    weightmask: 1.0
+    refine_inference: False
+    mask_unseen: False
+    online_buffer_update: True
+    class_balance: True
+    print_buffer_stats: True
+    buffer_update_after_step: True
+    buffer_storage: tensor
+    ce_detach_signal: False
+    distill_detach_signal: False
+    old_model_eval: False
+    freeze_old_model: True

config/zz_IDER/ider_cifar100_buf500_til.yaml

@@ -0,0 +1,91 @@
+dataset: &dataset binary_cifar100
+data_root: ./datasets
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+task_num: &task_num 10
+image_size: 32
+save_path: ./
+
+train_trfms:
+  - RandomCrop:
+      size: 32
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4867, 0.4408]
+      std: [0.2675, 0.2565, 0.2761]
+
+seed: 0
+deterministic: True
+device_ids: auto
+n_gpu: 1
+num_workers: 4
+pin_memory: False
+
+batch_size: 32
+init_epoch: 50
+epoch: 50
+val_per_epoch: 50
+testing_times: 1
+testing_per_task: True
+setting: task-aware
+eval_with_test: True
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.03
+    momentum: 0.0
+    weight_decay: 0.0
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [35, 45]
+
+backbone:
+  name: resnet18_id2
+  kwargs:
+    num_classes: *total_cls_num
+    nf: 64
+    use_cos: False
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 500
+    batch_size: 32
+    strategy: random
+
+classifier:
+  name: IDER
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    weighta: 0.5
+    weightb: 0.4
+    weightc: 0.5
+    weightmask: 1.0
+    refine_inference: False
+    mask_unseen: False
+    online_buffer_update: True
+    class_balance: True
+    print_buffer_stats: True
+    buffer_update_after_step: True
+    buffer_storage: tensor
+    ce_detach_signal: False
+    distill_detach_signal: False
+    old_model_eval: False
+    freeze_old_model: True

config/zz_IDER/ider_cifar10_buf200.yaml

@@ -0,0 +1,89 @@
+dataset: &dataset binary_cifar10
+data_root: ./datasets
+total_cls_num: &total_cls_num 10
+init_cls_num: &init_cls_num 2
+inc_cls_num: &inc_cls_num 2
+task_num: &task_num 5
+image_size: 32
+save_path: ./
+
+train_trfms:
+  - RandomCrop:
+      size: 32
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4914, 0.4822, 0.4465]
+      std: [0.2470, 0.2435, 0.2615]
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4914, 0.4822, 0.4465]
+      std: [0.2470, 0.2435, 0.2615]
+
+seed: 0
+deterministic: True
+device_ids: auto
+n_gpu: 1
+num_workers: 4
+pin_memory: False
+
+batch_size: 32
+init_epoch: 50
+epoch: 50
+val_per_epoch: 50
+testing_times: 1
+testing_per_task: True
+setting: task-agnostic
+eval_with_test: True
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.03
+    momentum: 0.0
+    weight_decay: 0.0
+
+lr_scheduler:
+  name: Constant
+  kwargs: {}
+
+backbone:
+  name: resnet18_id2
+  kwargs:
+    num_classes: *total_cls_num
+    nf: 64
+    use_cos: False
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 200
+    batch_size: 32
+    strategy: random
+
+classifier:
+  name: IDER
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    weighta: 0.05
+    weightb: 0.05
+    weightc: 0.5
+    weightmask: 1.0
+    refine_inference: False
+    mask_unseen: False
+    online_buffer_update: True
+    class_balance: True
+    print_buffer_stats: True
+    buffer_update_after_step: True
+    buffer_storage: tensor
+    ce_detach_signal: False
+    distill_detach_signal: False
+    old_model_eval: False
+    freeze_old_model: True

config/zz_IDER/ider_cifar10_buf500.yaml

@@ -0,0 +1,89 @@
+dataset: &dataset binary_cifar10
+data_root: ./datasets
+total_cls_num: &total_cls_num 10
+init_cls_num: &init_cls_num 2
+inc_cls_num: &inc_cls_num 2
+task_num: &task_num 5
+image_size: 32
+save_path: ./
+
+train_trfms:
+  - RandomCrop:
+      size: 32
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4914, 0.4822, 0.4465]
+      std: [0.2470, 0.2435, 0.2615]
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4914, 0.4822, 0.4465]
+      std: [0.2470, 0.2435, 0.2615]
+
+seed: 0
+deterministic: True
+device_ids: auto
+n_gpu: 1
+num_workers: 4
+pin_memory: False
+
+batch_size: 32
+init_epoch: 50
+epoch: 50
+val_per_epoch: 50
+testing_times: 1
+testing_per_task: True
+setting: task-agnostic
+eval_with_test: True
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.03
+    momentum: 0.0
+    weight_decay: 0.0
+
+lr_scheduler:
+  name: Constant
+  kwargs: {}
+
+backbone:
+  name: resnet18_id2
+  kwargs:
+    num_classes: *total_cls_num
+    nf: 64
+    use_cos: False
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 500
+    batch_size: 32
+    strategy: random
+
+classifier:
+  name: IDER
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    weighta: 0.05
+    weightb: 0.05
+    weightc: 0.5
+    weightmask: 1.0
+    refine_inference: False
+    mask_unseen: False
+    online_buffer_update: True
+    class_balance: True
+    print_buffer_stats: True
+    buffer_update_after_step: True
+    buffer_storage: tensor
+    ce_detach_signal: False
+    distill_detach_signal: False
+    old_model_eval: False
+    freeze_old_model: True

config/zz_IDER/ider_tinyimagenet_buf4000.yaml

@@ -0,0 +1,91 @@
+dataset: &dataset processed_tinyimg
+data_root: ./datasets/TINYIMG
+total_cls_num: &total_cls_num 200
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+task_num: &task_num 10
+image_size: 64
+save_path: ./
+
+train_trfms:
+  - RandomCrop:
+      size: 64
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4802, 0.4480, 0.3975]
+      std: [0.2770, 0.2691, 0.2821]
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4802, 0.4480, 0.3975]
+      std: [0.2770, 0.2691, 0.2821]
+
+seed: 0
+deterministic: True
+device_ids: auto
+n_gpu: 1
+num_workers: 4
+pin_memory: False
+
+batch_size: 32
+init_epoch: 50
+epoch: 50
+val_per_epoch: 50
+testing_times: 1
+testing_per_task: True
+setting: task-agnostic
+eval_with_test: True
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.03
+    momentum: 0.0
+    weight_decay: 0.0
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [35, 45]
+
+backbone:
+  name: resnet18_id2
+  kwargs:
+    num_classes: *total_cls_num
+    nf: 64
+    use_cos: False
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 4000
+    batch_size: 32
+    strategy: random
+
+classifier:
+  name: IDER
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    weighta: 0.5
+    weightb: 0.4
+    weightc: 0.5
+    weightmask: 1.0
+    refine_inference: False
+    mask_unseen: False
+    online_buffer_update: True
+    class_balance: True
+    print_buffer_stats: True
+    buffer_update_after_step: True
+    buffer_storage: tensor
+    ce_detach_signal: False
+    distill_detach_signal: False
+    old_model_eval: False
+    freeze_old_model: True

config/zz_IDER/ider_tinyimagenet_buf500.yaml

@@ -0,0 +1,91 @@
+dataset: &dataset processed_tinyimg
+data_root: ./datasets/TINYIMG
+total_cls_num: &total_cls_num 200
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+task_num: &task_num 10
+image_size: 64
+save_path: ./
+
+train_trfms:
+  - RandomCrop:
+      size: 64
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4802, 0.4480, 0.3975]
+      std: [0.2770, 0.2691, 0.2821]
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4802, 0.4480, 0.3975]
+      std: [0.2770, 0.2691, 0.2821]
+
+seed: 0
+deterministic: True
+device_ids: auto
+n_gpu: 1
+num_workers: 4
+pin_memory: False
+
+batch_size: 32
+init_epoch: 50
+epoch: 50
+val_per_epoch: 50
+testing_times: 1
+testing_per_task: True
+setting: task-agnostic
+eval_with_test: True
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.03
+    momentum: 0.0
+    weight_decay: 0.0
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [35, 45]
+
+backbone:
+  name: resnet18_id2
+  kwargs:
+    num_classes: *total_cls_num
+    nf: 64
+    use_cos: False
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 500
+    batch_size: 32
+    strategy: random
+
+classifier:
+  name: IDER
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    weighta: 0.5
+    weightb: 0.4
+    weightc: 0.5
+    weightmask: 1.0
+    refine_inference: False
+    mask_unseen: False
+    online_buffer_update: True
+    class_balance: True
+    print_buffer_stats: True
+    buffer_update_after_step: True
+    buffer_storage: tensor
+    ce_detach_signal: False
+    distill_detach_signal: False
+    old_model_eval: False
+    freeze_old_model: True

config/zz_IDER/ider_tinyimagenet_buf500_paper.yaml

@@ -0,0 +1,91 @@
+dataset: &dataset processed_tinyimg
+data_root: ./datasets/TINYIMG
+total_cls_num: &total_cls_num 200
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+task_num: &task_num 10
+image_size: 64
+save_path: ./
+
+train_trfms:
+  - RandomCrop:
+      size: 64
+      padding: 4
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4802, 0.4480, 0.3975]
+      std: [0.2770, 0.2691, 0.2821]
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.4802, 0.4480, 0.3975]
+      std: [0.2770, 0.2691, 0.2821]
+
+seed: 0
+deterministic: True
+device_ids: auto
+n_gpu: 1
+num_workers: 4
+pin_memory: False
+
+batch_size: 32
+init_epoch: 100
+epoch: 100
+val_per_epoch: 100
+testing_times: 1
+testing_per_task: True
+setting: task-agnostic
+eval_with_test: True
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.03
+    momentum: 0.0
+    weight_decay: 0.0
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [35, 60, 75]
+
+backbone:
+  name: resnet18_id2
+  kwargs:
+    num_classes: *total_cls_num
+    nf: 64
+    use_cos: False
+    args:
+      dataset: *dataset
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 500
+    batch_size: 32
+    strategy: random
+
+classifier:
+  name: IDER
+  kwargs:
+    num_class: *total_cls_num
+    feat_dim: 512
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    weighta: 0.5
+    weightb: 0.4
+    weightc: 0.5
+    weightmask: 1.0
+    refine_inference: False
+    mask_unseen: False
+    online_buffer_update: True
+    class_balance: True
+    print_buffer_stats: True
+    buffer_update_after_step: True
+    buffer_storage: tensor
+    ce_detach_signal: False
+    distill_detach_signal: False
+    old_model_eval: False
+    freeze_old_model: True

core/data/dataloader.py

@@ -108,7 +108,18 @@ def get_dataloader(config, mode, cls_map=None):
                 _, cls_code, cls_name = line.strip().split('\t')
                 cls_map[cls_code] = cls_name
 
-    elif cls_map is None and dataset != 'binary_cifar100':
+    elif cls_map is None and dataset in ['binary_cifar10', 'binary_cifar100']:
+        num_classes = 10 if dataset == 'binary_cifar10' else 100
+        if 'class_order' in config.keys():
+            class_order = config['class_order']
+        elif config.get('shuffle', False):
+            rng = np.random.RandomState(config['seed'])
+            class_order = rng.permutation(num_classes).tolist()
+        else:
+            class_order = list(range(num_classes))
+        cls_map = {label: ori_label for label, ori_label in enumerate(class_order)}
+
+    elif cls_map is None and dataset not in ['binary_cifar10', 'binary_cifar100', 'processed_tinyimg']:
         # Apply class_order for debugging
         cls_list = sorted(os.listdir(os.path.join(data_root, mode)))
         #random.shuffle(cls_list)

core/data/dataset.py

@@ -1,7 +1,6 @@
 import os
 import torch
 import pickle
-import random
 import numpy as np
 
 from PIL import Image
@@ -11,6 +10,15 @@ from torch.utils.data.distributed import DistributedSampler
 from continuum.datasets import TinyImageNet200
 from continuum import ClassIncremental
 
+
+def _processed_tiny_dir(data_root):
+    processed_dir = os.path.join(data_root, "processed")
+    if os.path.isdir(processed_dir):
+        return processed_dir
+    if os.path.basename(os.path.normpath(data_root)) == "processed":
+        return data_root
+    return processed_dir
+
 class ContinualDatasets:
     def __init__(self, dataset, mode, task_num, init_cls_num, inc_cls_num, data_root, cls_map, trfms, batchsize, num_workers, config):
         self.mode = mode
@@ -25,8 +33,18 @@ class ContinualDatasets:
         self.config = config
         self.dataset = dataset
 
-        if self.dataset == 'binary_cifar100':
-            datasets.CIFAR100(self.data_root, download = True)
+        if self.dataset in ['binary_cifar10', 'binary_cifar100']:
+            num_classes = 10 if self.dataset == 'binary_cifar10' else 100
+            if self.cls_map is None:
+                class_order = self.config.get('class_order', list(range(num_classes)))
+                self.cls_map = {label: ori_label for label, ori_label in enumerate(class_order)}
+            if self.dataset == 'binary_cifar10':
+                datasets.CIFAR10(self.data_root, download=True)
+            else:
+                datasets.CIFAR100(self.data_root, download=True)
+        elif self.dataset == 'processed_tinyimg':
+            class_order = self.config.get('class_order', list(range(200)))
+            self.cls_map = {label: ori_label for label, ori_label in enumerate(class_order)}
 
         self.create_loaders()
 
@@ -37,10 +55,11 @@ class ContinualDatasets:
 
             if 'class_order' in self.config:
                 class_order = self.config['class_order']
+            elif self.config.get('shuffle', False):
+                rng = np.random.RandomState(self.config['seed'])
+                class_order = rng.permutation(200).tolist()
             else:
                 class_order = list(range(200))
-                random.seed(self.config['seed'])
-                random.shuffle(class_order)
 
             scenario = ClassIncremental(
                 TinyImageNet200(self.data_root, train=self.mode == 'train', download=True),
@@ -69,7 +88,7 @@ class ContinualDatasets:
 
                 self.dataloaders.append(DataLoader(
                     dataset,
-                    shuffle = True,
+                    shuffle = self.mode == 'train',
                     batch_size = self.batchsize,
                     drop_last = False,
                     num_workers = self.num_workers,
@@ -84,11 +103,11 @@ class ContinualDatasets:
                 end_idx = start_idx + (self.init_cls_num if i ==0 else self.inc_cls_num)
                 self.dataloaders.append(DataLoader(
                     SingleDataset(self.dataset, self.data_root, self.mode, self.init_cls_num, self.inc_cls_num, self.cls_map, self.trfms, start_idx, end_idx),
-                    shuffle = True,
+                    shuffle = self.mode == 'train',
                     batch_size = self.batchsize,
                     drop_last = False,
                     num_workers = self.num_workers,
-                    pin_memory=False
+                    pin_memory=self.config['pin_memory']
                 ))
 
     def get_loader(self, task_idx):
@@ -246,7 +265,7 @@ class SingleDataset(Dataset):
             self.images, self.labels, self.labels_name = self._init_datalist()
 
     def __getitem__(self, idx):
-        if self.dataset == 'binary_cifar100':
+        if self.dataset in ['binary_cifar10', 'binary_cifar100']:
 
             image = self.images[idx]
             image = Image.fromarray(np.uint8(image))
@@ -255,6 +274,12 @@ class SingleDataset(Dataset):
             img_path = self.images[idx]
             image = Image.open(img_path).convert("RGB")
 
+        elif self.dataset == 'processed_tinyimg':
+            image = self.images[idx]
+            if np.max(image) <= 1.0:
+                image = np.uint8(255 * image)
+            image = Image.fromarray(np.uint8(image)).convert("RGB")
+
         else:
             
             img_path = self.images[idx]
@@ -263,7 +288,7 @@ class SingleDataset(Dataset):
         label = self.labels[idx]
         image = self.trfms(image)
 
-        return {"image": image, "label": label}
+        return {"image": image, "label": label, "index": idx}
     
     def __len__(self,):
         return len(self.labels)
@@ -272,22 +297,67 @@ class SingleDataset(Dataset):
 
         imgs, labels, labels_name = [], [], []
 
-        if self.dataset == 'binary_cifar100':
-            
-            with open(os.path.join(self.data_root, 'cifar-100-python', self.mode), 'rb') as f:
-                load_data = pickle.load(f, encoding='latin1')
+        if self.dataset in ['binary_cifar10', 'binary_cifar100']:
+            if self.dataset == 'binary_cifar10':
+                data_dir = os.path.join(self.data_root, 'cifar-10-batches-py')
+                file_names = [f'data_batch_{idx}' for idx in range(1, 6)] if self.mode == 'train' else ['test_batch']
+                data_chunks, label_chunks = [], []
+                for file_name in file_names:
+                    with open(os.path.join(data_dir, file_name), 'rb') as f:
+                        load_data = pickle.load(f, encoding='latin1')
+                    data_chunks.append(load_data['data'])
+                    label_chunks.extend(load_data['labels'])
+                raw_data = np.concatenate(data_chunks, axis=0)
+                raw_labels = label_chunks
+            else:
+                with open(os.path.join(self.data_root, 'cifar-100-python', self.mode), 'rb') as f:
+                    load_data = pickle.load(f, encoding='latin1')
+                raw_data = load_data['data']
+                raw_labels = load_data['fine_labels']
+
+            num_classes = 10 if self.dataset == 'binary_cifar10' else 100
+            cls_map = self.cls_map or {label: label for label in range(num_classes)}
+            new_label_by_old = {old_label: new_label for new_label, old_label in cls_map.items()}
 
-            for data, label in zip(load_data['data'], load_data['fine_labels']):
+            for data, label in zip(raw_data, raw_labels):
 
-                if label in range(self.start_idx, self.end_idx):
+                if label not in new_label_by_old:
+                    continue
+
+                new_label = new_label_by_old[label]
+                if new_label in range(self.start_idx, self.end_idx):
                     r = data[:1024].reshape(32, 32)
                     g = data[1024:2048].reshape(32, 32)
                     b = data[2048:].reshape(32, 32)
 
                     tt_data = np.dstack((r, g, b))
-
                     imgs.append(tt_data)
-                    labels.append(label)
+                    labels.append(new_label)
+                    labels_name.append(new_label)
+
+        elif self.dataset == 'processed_tinyimg':
+            split = 'train' if self.mode == 'train' else 'val'
+            processed_dir = _processed_tiny_dir(self.data_root)
+            label_range = range(self.start_idx, self.end_idx)
+            ori_to_label = {self.cls_map[id]: id for id in label_range}
+            target_classes = np.array(list(ori_to_label.keys()))
+
+            for num in range(20):
+                data_path = os.path.join(processed_dir, f'x_{split}_{num + 1:02d}.npy')
+                label_path = os.path.join(processed_dir, f'y_{split}_{num + 1:02d}.npy')
+                raw_labels = np.load(label_path)
+                keep_mask = np.isin(raw_labels, target_classes)
+                if not np.any(keep_mask):
+                    continue
+
+                raw_data = np.load(data_path, mmap_mode='r')
+                kept_data = raw_data[keep_mask]
+                kept_labels = raw_labels[keep_mask]
+
+                for data, label in zip(kept_data, kept_labels):
+                    label = int(label)
+                    imgs.append(np.asarray(data))
+                    labels.append(ori_to_label[label])
                     labels_name.append(label)
 
         else:
@@ -301,4 +371,4 @@ class SingleDataset(Dataset):
         return imgs, labels, labels_name
 
     def get_class_names(self):
-        return self.labels_name
+        return self.labels_name

core/model/__init__.py

@@ -8,10 +8,13 @@ from .lwf import LWF
 from .wa import WA
 from .bic import bic
 from .ewc import EWC
+from .ewcdr import EWCDR
 from .ocm import OCM
 from .eraml import ERAML
 from .erace import ERACE
 from .der import DER
+from .ider import IDER
+from .paper_er import PaperER
 from .dualprompt import DualPrompt
 from .l2p import L2P
 from .codaprompt import CodaPrompt
@@ -31,4 +34,5 @@ from .api import API
 from .dap import DAP
 from .sd_lora import SD_LoRA
 from .lora_sub import LoRAsub_DRS
-from .cl_lora import CL_LoRA
+from .cl_lora import CL_LoRA
+from .fly_cl import FlyCL

core/model/backbone/__init__.py

@@ -1,4 +1,5 @@
 from .resnet import *
+from .resnet_id import *
 from .vit import vit_pt_imnet
 from .vit import vit_pt_imnet_in21k_adapter
 from .vit import vit_cl_lora
@@ -9,6 +10,7 @@ from .SiNet import SiNet_vit
 from .resnet_cbam import *
 from .alexnet import AlexNet_TRGP, AlexNet_API
 from .clip import clip
+from .timm_backbone import timm_backbone
 
 def get_backbone(config):
     """

core/model/backbone/resnet.py

@@ -133,6 +133,12 @@ class ResNet(nn.Module):
         if 'cifar' in args["dataset"] or '5-datasets' in args["dataset"]:
             self.conv1 = nn.Sequential(nn.Conv2d(3, self.inplanes, kernel_size=3, stride=1, padding=1, bias=False),
                                        nn.BatchNorm2d(self.inplanes), nn.ReLU(inplace=True))
+        elif 'tinyimg' in args["dataset"] or 'tiny-imagenet' in args["dataset"]:
+            # The original IDER/Mammoth ResNet18 uses a CIFAR-style 3x3
+            # stem for TinyImageNet instead of the torchvision ImageNet
+            # 7x7 stride-2 stem.
+            self.conv1 = nn.Sequential(nn.Conv2d(3, self.inplanes, kernel_size=3, stride=1, padding=1, bias=False),
+                                       nn.BatchNorm2d(self.inplanes), nn.ReLU(inplace=True))
         elif 'imagenet' in args["dataset"]:
             if args["init_cls_num"] == args["inc_cls_num"]:
                 self.conv1 = nn.Sequential(

core/model/backbone/resnet_id.py

@@ -0,0 +1,165 @@
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+__all__ = ["resnet18_id2"]
+
+
+def conv3x3(in_planes, out_planes, stride=1):
+    return nn.Conv2d(
+        in_planes,
+        out_planes,
+        kernel_size=3,
+        stride=stride,
+        padding=1,
+        bias=False,
+    )
+
+
+class BasicBlock(nn.Module):
+    expansion = 1
+
+    def __init__(self, in_planes, planes, stride=1):
+        super().__init__()
+        self.conv1 = conv3x3(in_planes, planes, stride)
+        self.bn1 = nn.BatchNorm2d(planes)
+        self.conv2 = conv3x3(planes, planes)
+        self.bn2 = nn.BatchNorm2d(planes)
+
+        self.shortcut = nn.Sequential()
+        if stride != 1 or in_planes != self.expansion * planes:
+            self.shortcut = nn.Sequential(
+                nn.Conv2d(
+                    in_planes,
+                    self.expansion * planes,
+                    kernel_size=1,
+                    stride=stride,
+                    bias=False,
+                ),
+                nn.BatchNorm2d(self.expansion * planes),
+            )
+
+    def forward(self, x):
+        out = F.relu(self.bn1(self.conv1(x)), inplace=True)
+        out = self.bn2(self.conv2(out))
+        out = out + self.shortcut(x)
+        return F.relu(out, inplace=True)
+
+
+class CosineClassifier(nn.Module):
+    def __init__(self, feat_dim, num_classes, temperature=12.0):
+        super().__init__()
+        fc = nn.Linear(feat_dim, num_classes)
+        self.weight = nn.Parameter(fc.weight.t())
+        self.bias = nn.Parameter(fc.bias)
+        self.temperature = nn.Parameter(torch.tensor([temperature]), requires_grad=False)
+
+    def forward(self, features):
+        features = F.normalize(features, p=2, dim=1, eps=1e-12)
+        weight = F.normalize(self.weight, p=2, dim=0, eps=1e-12)
+        return self.temperature * torch.mm(features, weight)
+
+
+class ResNetStage1(nn.Module):
+    def __init__(self, block, num_blocks, nf):
+        super().__init__()
+        self.in_planes = nf
+        self.conv1 = conv3x3(3, nf)
+        self.bn1 = nn.BatchNorm2d(nf)
+        self.layer1 = self._make_layer(block, nf, num_blocks[0], stride=1)
+        self.layer2 = self._make_layer(block, nf * 2, num_blocks[1], stride=2)
+
+    def _make_layer(self, block, planes, num_blocks, stride):
+        strides = [stride] + [1] * (num_blocks - 1)
+        layers = []
+        for stride in strides:
+            layers.append(block(self.in_planes, planes, stride))
+            self.in_planes = planes * block.expansion
+        return nn.Sequential(*layers)
+
+    def forward(self, x):
+        out = F.relu(self.bn1(self.conv1(x)), inplace=True)
+        out = self.layer1(out)
+        return self.layer2(out)
+
+
+class ResNetStage2(nn.Module):
+    def __init__(self, block, num_blocks, num_classes, nf, use_cos=False):
+        super().__init__()
+        self.in_planes = nf * 2
+        self.num_classes = num_classes
+        self.out_dim = nf * 8 * block.expansion
+        self.label_fc = nn.Sequential(
+            nn.Linear(num_classes, nf * 2),
+            nn.LeakyReLU(inplace=True),
+        )
+        self.layer3 = self._make_layer(block, nf * 4, num_blocks[2], stride=2)
+        self.layer4 = self._make_layer(block, nf * 8, num_blocks[3], stride=2)
+        if use_cos:
+            self.classifier = CosineClassifier(self.out_dim, num_classes)
+        else:
+            self.classifier = nn.Linear(self.out_dim, num_classes)
+
+    def _make_layer(self, block, planes, num_blocks, stride):
+        strides = [stride] + [1] * (num_blocks - 1)
+        layers = []
+        for stride in strides:
+            layers.append(block(self.in_planes, planes, stride))
+            self.in_planes = planes * block.expansion
+        return nn.Sequential(*layers)
+
+    def forward(self, features, label_signal):
+        out = features + self.label_fc(label_signal)[..., None, None]
+        out = self.layer3(out)
+        out = self.layer4(out)
+        pooled = F.avg_pool2d(out, out.shape[2])
+        flat = torch.flatten(pooled, 1)
+        logits = self.classifier(flat)
+        return logits[:, : self.num_classes], flat, out
+
+
+class IdempotentResNet(nn.Module):
+    """ResNet-18 split for IDER.
+
+    The image path produces an intermediate feature map. The second input is a
+    label/probability signal projected to the same channel dimension and added
+    before the later ResNet stages, following the official IDER implementation.
+    """
+
+    def __init__(self, block, num_blocks, num_classes=100, nf=64, use_cos=False, **kwargs):
+        super().__init__()
+        self.num_classes = num_classes
+        self.out_dim = nf * 8 * block.expansion
+        self.f1 = ResNetStage1(block, num_blocks, nf)
+        self.f2 = ResNetStage2(block, num_blocks, num_classes, nf, use_cos=use_cos)
+
+    def empty_signal(self, batch_size, device, upto=None):
+        upto = self.num_classes if upto is None else min(upto, self.num_classes)
+        signal = torch.zeros(batch_size, self.num_classes, device=device)
+        signal[:, :upto] = 1.0 / max(upto, 1)
+        return signal
+
+    def forward(self, x, y=None, upto=None, returnt="all"):
+        if y is None:
+            y = self.empty_signal(x.size(0), x.device, upto=upto)
+
+        stage1_features = self.f1(x)
+        logits, features, fmap = self.f2(stage1_features, y)
+
+        if returnt == "logits":
+            return logits
+        if returnt == "features":
+            return {"features": features}
+        return {"logits": logits, "features": features, "fmaps": [stage1_features, fmap]}
+
+
+def resnet18_id2(pretrained=False, progress=True, num_classes=100, nf=64, use_cos=False, **kwargs):
+    if "nclasses" in kwargs:
+        num_classes = kwargs["nclasses"]
+    return IdempotentResNet(
+        BasicBlock,
+        [2, 2, 2, 2],
+        num_classes=num_classes,
+        nf=nf,
+        use_cos=use_cos,
+    )

core/model/backbone/timm_backbone.py

@@ -0,0 +1,22 @@
+import timm
+import torch.nn as nn
+
+
+class TimmBackbone(nn.Module):
+
+    def __init__(self, model_name="vit_base_patch16_224", pretrained=False, **kwargs):
+        super().__init__()
+        self.feat = timm.create_model(
+            model_name,
+            pretrained=pretrained,
+            num_classes=0,
+            **kwargs,
+        )
+        self.feat_dim = self.feat.num_features
+
+    def forward(self, x):
+        return self.feat(x)
+
+
+def timm_backbone(pretrained=False, **kwargs):
+    return TimmBackbone(pretrained=pretrained, **kwargs)

core/model/ewcdr.py

@@ -0,0 +1,158 @@
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from .finetune import Finetune
+
+
+class IncrementalLinearNet(nn.Module):
+    def __init__(self, backbone, feat_dim, num_class):
+        super().__init__()
+        self.backbone = backbone
+        self.classifier = nn.Linear(feat_dim, num_class)
+
+    def _features(self, x):
+        output = self.backbone(x)
+        if isinstance(output, dict):
+            return output["features"]
+        return output
+
+    def forward(self, x):
+        return self.classifier(self._features(x))
+
+    def update_classifier(self, num_class):
+        old_classifier = self.classifier
+        if old_classifier.out_features == num_class:
+            return
+
+        new_classifier = nn.Linear(old_classifier.in_features, num_class)
+        new_classifier = new_classifier.to(old_classifier.weight.device)
+        with torch.no_grad():
+            old_out = old_classifier.out_features
+            new_classifier.weight[:old_out].copy_(old_classifier.weight)
+            new_classifier.bias[:old_out].copy_(old_classifier.bias)
+        self.classifier = new_classifier
+
+
+class EWCDR(Finetune):
+    """EWC Done Right with logits reversal during importance estimation."""
+
+    def __init__(self, backbone, feat_dim, num_class, **kwargs):
+        super().__init__(backbone, feat_dim, num_class, **kwargs)
+        self.init_cls_num = kwargs["init_cls_num"]
+        self.inc_cls_num = kwargs["inc_cls_num"]
+        self.lamda = kwargs["lamda"]
+        self.omega_max = kwargs.get("omega_max", 1e-4)
+
+        self.task_idx = 0
+        self.known_cls_num = 0
+        self.total_cls_num = self.init_cls_num
+        self.network = IncrementalLinearNet(backbone, feat_dim, self.init_cls_num)
+        self.ref_param = {}
+        self.omega = None
+
+    def before_task(self, task_idx, buffer, train_loader, test_loaders):
+        self.task_idx = task_idx
+        self.known_cls_num = (
+            0 if task_idx == 0 else self.init_cls_num + (task_idx - 1) * self.inc_cls_num
+        )
+        self.total_cls_num = self.init_cls_num + task_idx * self.inc_cls_num
+        self.network.update_classifier(self.total_cls_num)
+        self.network.to(self.device)
+
+    def observe(self, data):
+        x = data["image"].to(self.device)
+        y = data["label"].to(self.device)
+        logits = self.network(x)
+
+        if self.task_idx == 0:
+            loss = F.cross_entropy(logits, y)
+        else:
+            loss = F.cross_entropy(
+                logits[:, self.known_cls_num:], y - self.known_cls_num
+            )
+            loss = loss + self.lamda * self.compute_ewc()
+
+        pred = torch.argmax(logits, dim=1)
+        acc = torch.sum(pred == y).item()
+        return pred, acc / x.size(0), loss
+
+    def inference(self, data):
+        x = data["image"].to(self.device)
+        y = data["label"].to(self.device)
+        logits = self.network(x)
+        pred = torch.argmax(logits, dim=1)
+        acc = torch.sum(pred == y).item()
+        return pred, acc / x.size(0)
+
+    def after_task(self, task_idx, buffer, train_loader, test_loaders):
+        new_omega = self.get_importance(train_loader)
+        if self.omega is not None:
+            alpha = self.known_cls_num / self.total_cls_num
+            for name, old_omega in self.omega.items():
+                if name not in new_omega:
+                    continue
+                old_slice = self._prefix_slice(old_omega)
+                new_omega[name][old_slice] = (
+                    alpha * old_omega + (1 - alpha) * new_omega[name][old_slice]
+                )
+
+        self.omega = new_omega
+        self.ref_param = {
+            name: param.clone().detach()
+            for name, param in self.network.named_parameters()
+            if param.requires_grad
+        }
+
+    def get_importance(self, train_loader):
+        omega = {
+            name: torch.zeros_like(param, device=self.device)
+            for name, param in self.network.named_parameters()
+            if param.requires_grad
+        }
+
+        was_training = self.network.training
+        self.network.train()
+        for data in train_loader:
+            x = data["image"].to(self.device)
+            y = data["label"].to(self.device)
+            logits = -self.network(x)
+            loss = F.cross_entropy(logits, y)
+
+            self.network.zero_grad(set_to_none=True)
+            loss.backward()
+
+            for name, param in self.network.named_parameters():
+                if param.grad is not None and name in omega:
+                    omega[name] += param.grad.pow(2).detach()
+
+        for name, value in omega.items():
+            value = value / len(train_loader)
+            omega[name] = torch.clamp(value, max=self.omega_max)
+
+        self.network.zero_grad(set_to_none=True)
+        self.network.train(was_training)
+        return omega
+
+    def compute_ewc(self):
+        if self.omega is None:
+            return torch.zeros((), device=self.device)
+
+        loss = torch.zeros((), device=self.device)
+        for name, param in self.network.named_parameters():
+            if name not in self.omega:
+                continue
+            ref = self.ref_param[name]
+            old_slice = self._prefix_slice(ref)
+            loss += torch.sum(self.omega[name] * (param[old_slice] - ref).pow(2)) / 2
+        return loss
+
+    @staticmethod
+    def _prefix_slice(tensor):
+        return (slice(0, tensor.shape[0]),) + (slice(None),) * (tensor.dim() - 1)
+
+    def forward(self, x):
+        return self.network(x)
+
+    def get_parameters(self, config):
+        return [{"params": self.network.parameters()}]

core/model/finetune.py

@@ -35,6 +35,11 @@ class Finetune(nn.Module):
         acc = torch.sum(pred == y).item()
         return pred, acc / x.size(0)
 
+    def predict_logits(self, data):
+        """Return logits for calibration metrics during evaluation."""
+        x = data['image'].to(self.device)
+        return self.classifier(self.backbone(x)['features'])
+
     def forward(self, x):
         return self.classifier(self.backbone(x)['features'])  
     

core/model/fly_cl.py

@@ -0,0 +1,242 @@
+import time
+
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from tqdm import tqdm
+
+
+class FlyCL(nn.Module):
+    def __init__(self, backbone, device, **kwargs):
+        super().__init__()
+        self.backbone = backbone
+        self.device = device
+
+        self.embedding_dim = int(kwargs.get("embedding_dim", backbone.feat_dim))
+        self.expand_dim = int(kwargs.get("expand_dim", 10000))
+        self.synaptic_degree = int(kwargs.get("synaptic_degree", 300))
+        self.coding_level = float(kwargs.get("coding_level", 0.3))
+        self.ridge_lower = float(kwargs.get("ridge_lower", 6))
+        self.ridge_upper = float(kwargs.get("ridge_upper", 10))
+        self.gcv_backend = kwargs.get("gcv_backend", "dual_eigh")
+        self.disable_cudnn = bool(kwargs.get("disable_cudnn", False))
+        self.use_projection = bool(kwargs.get("use_projection", True))
+        self.use_ridge = bool(kwargs.get("use_ridge", True))
+        self.total_cls_num = int(kwargs["total_cls_num"])
+        self.init_cls_num = int(kwargs["init_cls_num"])
+        self.inc_cls_num = int(kwargs["inc_cls_num"])
+        self.mask_unseen_classes = bool(kwargs.get("mask_unseen_classes", False))
+
+        if self.embedding_dim != int(backbone.feat_dim):
+            raise ValueError(
+                f"embedding_dim={self.embedding_dim} does not match "
+                f"backbone.feat_dim={backbone.feat_dim}"
+            )
+        if (
+            self.use_projection
+            and not 1 <= self.synaptic_degree <= self.embedding_dim
+        ):
+            raise ValueError("synaptic_degree must be in [1, embedding_dim]")
+        if self.use_projection and not 0.0 < self.coding_level <= 1.0:
+            raise ValueError("coding_level must be in (0, 1]")
+        if self.ridge_upper <= self.ridge_lower:
+            raise ValueError("ridge_upper must be greater than ridge_lower")
+        if self.gcv_backend not in {"dual_eigh", "svd"}:
+            raise ValueError("gcv_backend must be 'dual_eigh' or 'svd'")
+
+        for parameter in self.backbone.parameters():
+            parameter.requires_grad = False
+        self.backbone.eval()
+
+        if self.disable_cudnn:
+            torch.backends.cudnn.deterministic = True
+            torch.backends.cudnn.enabled = False
+
+        self.analytic_dim = (
+            self.expand_dim if self.use_projection else self.embedding_dim
+        )
+        projection = self._build_sparse_projection() if self.use_projection else None
+        self.register_buffer("projection", projection)
+        self.register_buffer("Q", torch.zeros(self.analytic_dim, self.total_cls_num))
+        gram = (
+            torch.zeros(self.analytic_dim, self.analytic_dim)
+            if self.use_ridge
+            else None
+        )
+        self.register_buffer("G", gram)
+        self.register_buffer(
+            "classifier_weight", torch.zeros(self.analytic_dim, self.total_cls_num)
+        )
+        self.register_buffer("class_counts", torch.zeros(self.total_cls_num))
+
+        self._optimizer_anchor = nn.Parameter(torch.zeros(()))
+
+        self._classes_seen_so_far = 0
+        self._is_fitted = False
+        self.last_ridge = None
+        self.last_feature_time = 0.0
+        self.last_post_time = 0.0
+
+    def _build_sparse_projection(self):
+        """Create the reference implementation's sparse Gaussian projection."""
+        projection = torch.zeros(self.expand_dim, self.embedding_dim)
+        for row in range(self.expand_dim):
+            selected = torch.randperm(self.embedding_dim)[: self.synaptic_degree]
+            projection[row, selected] = torch.randn(self.synaptic_degree)
+        return projection.to_sparse_csc()
+
+    def train(self, mode=True):
+        # Trainer calls model.train(); the pre-trained feature extractor must
+        # nevertheless stay deterministic and frozen, as in the official code.
+        super().train(mode)
+        self.backbone.eval()
+        return self
+
+    def before_task(self, task_idx, buffer, train_loader, test_loaders):
+        increment = self.init_cls_num if task_idx == 0 else self.inc_cls_num
+        self._classes_seen_so_far += increment
+        if self._classes_seen_so_far > self.total_cls_num:
+            raise ValueError("Seen classes exceed total_cls_num")
+        print(
+            f"[FlyCL] Task {task_idx}: frozen feature extraction; "
+            "analytic classifier will be updated after the task."
+        )
+
+    def observe(self, data):
+        loss = self._optimizer_anchor * 0.0
+        return None, 0.0, loss
+
+    @torch.no_grad()
+    def _extract_features(self, data_loader):
+        features = []
+        labels = []
+        self.backbone.eval()
+        for batch in tqdm(data_loader, desc="Fly-CL feature extraction"):
+            images = batch["image"].to(self.device, non_blocking=True)
+            feature = self.backbone(images)
+            features.append(feature)
+            labels.append(batch["label"].to(self.device, non_blocking=True))
+        if not features:
+            raise ValueError("Cannot fit Fly-CL on an empty dataloader")
+        return torch.cat(features, dim=0), torch.cat(labels, dim=0)
+
+    def _fly_encode(self, features):
+        if not self.use_projection:
+            return features.T
+        expanded = torch.sparse.mm(self.projection, features.T)
+        topk = max(1, int(self.expand_dim * self.coding_level))
+        values, indices = expanded.topk(topk, dim=0, largest=True)
+        encoded = torch.zeros_like(expanded)
+        encoded.scatter_(0, indices, values)
+        return encoded
+
+    @torch.no_grad()
+    def _select_ridge_parameter(self, features, targets):
+        """Generalized cross-validation used by the official implementation."""
+        if self.gcv_backend == "svd":
+            u, singular_values, _ = torch.linalg.svd(features, full_matrices=False)
+            singular_sq = singular_values.square()
+        else:
+            sample_gram = features @ features.T
+            singular_sq, u = torch.linalg.eigh(sample_gram)
+            singular_sq.clamp_(min=0.0)
+        uty = u.T @ targets
+        exponents = np.arange(self.ridge_lower, self.ridge_upper)
+        ridges = torch.as_tensor(
+            10.0 ** exponents,
+            dtype=features.dtype,
+            device=features.device,
+        )
+        n_samples = features.shape[0]
+        scores = []
+        for ridge in ridges:
+            diagonal = singular_sq / (singular_sq + ridge)
+            degrees_of_freedom = diagonal.sum()
+            prediction = u @ (diagonal[:, None] * uty)
+            residual = torch.linalg.vector_norm(targets - prediction).square()
+            denominator = (1.0 - degrees_of_freedom / n_samples).square()
+            scores.append((residual / n_samples) / denominator)
+        return ridges[torch.argmin(torch.stack(scores))]
+
+    @torch.no_grad()
+    def fit_features(self, features, labels):
+        """Update the analytic head from already-extracted backbone features."""
+        encoded = self._fly_encode(features)
+        del features
+        targets = F.one_hot(labels.long(), self.total_cls_num).to(encoded.dtype)
+        self.class_counts.add_(targets.sum(dim=0))
+        del labels
+        self.Q.add_(encoded @ targets)
+        if self.use_ridge:
+            self.G.add_(encoded @ encoded.T)
+            ridge = self._select_ridge_parameter(encoded.T, targets)
+            del encoded, targets
+            regularized = self.G.clone()
+            regularized.diagonal().add_(ridge)
+            cholesky = torch.linalg.cholesky(regularized)
+            del regularized
+            self.classifier_weight.copy_(torch.cholesky_solve(self.Q, cholesky))
+            self.last_ridge = float(ridge.item())
+        else:
+            del encoded, targets
+            counts = self.class_counts.clamp_min(1.0).unsqueeze(0)
+            self.classifier_weight.copy_(self.Q / counts)
+            self.last_ridge = None
+
+        self._is_fitted = True
+
+    @torch.no_grad()
+    def after_task(self, task_idx, buffer, train_loader, test_loaders):
+        total_start = time.perf_counter()
+        feature_start = time.perf_counter()
+        features, labels = self._extract_features(train_loader)
+        self.last_feature_time = time.perf_counter() - feature_start
+        self.fit_features(features, labels)
+        self.last_post_time = (
+            time.perf_counter() - total_start - self.last_feature_time
+        )
+        print(
+            f"[FlyCL] Task {task_idx} fitted: "
+            f"ridge={self.last_ridge if self.last_ridge is not None else 'disabled'}, "
+            f"feature_time={self.last_feature_time:.2f}s, "
+            f"post_time={self.last_post_time:.2f}s"
+        )
+
+    @torch.no_grad()
+    def predict_features(self, features, targets=None):
+        """Predict from cached backbone features using the fitted analytic head."""
+        if not self._is_fitted:
+            raise RuntimeError("Fly-CL classifier is not fitted; call after_task first")
+        encoded = self._fly_encode(features)
+        if self.use_ridge:
+            if self.use_projection:
+                logits = torch.sparse.mm(
+                    encoded.T.to_sparse_csc(), self.classifier_weight
+                )
+            else:
+                logits = encoded.T @ self.classifier_weight
+        else:
+            encoded = F.normalize(encoded, dim=0)
+            prototypes = F.normalize(self.classifier_weight, dim=0)
+            if self.use_projection:
+                logits = torch.sparse.mm(encoded.T.to_sparse_csc(), prototypes)
+            else:
+                logits = encoded.T @ prototypes
+        if self.mask_unseen_classes:
+            logits = logits[:, : self._classes_seen_so_far]
+
+        accuracy = None
+        if targets is not None:
+            predictions = logits.argmax(dim=1).cpu()
+            accuracy = predictions.eq(targets.cpu()).float().mean().item()
+        return logits, accuracy
+
+    @torch.no_grad()
+    def inference(self, data):
+        images = data["image"].to(self.device, non_blocking=True)
+        features = self.backbone(images)
+        return self.predict_features(features, data["label"])
+
+    def get_parameters(self, config):
+        return [self._optimizer_anchor]

core/model/ider.py

@@ -0,0 +1,328 @@
+import copy
+import os
+from collections import Counter
+
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from PIL import Image
+from torchvision import transforms
+
+from .finetune import Finetune
+
+
+class IDER(Finetune):
+    """Idempotent Experience Replay for LibContinual.
+
+    This follows the official implementation: the current batch receives the
+    standard idempotent CE loss, while replay samples are drawn inside
+    ``observe`` for supervised ER and old-checkpoint idempotent distillation.
+    """
+
+    def __init__(self, backbone, feat_dim, num_class, **kwargs):
+        super().__init__(backbone, feat_dim, num_class, **kwargs)
+        self.classifier = nn.Identity()
+        self.weighta = kwargs.get("weighta", 0.5)
+        self.weightb = kwargs.get("weightb", 0.4)
+        self.weightc = kwargs.get("weightc", 0.5)
+        self.empty_prob = kwargs.get("weightmask", kwargs.get("empty_prob", 1.0))
+        self.refine_inference = kwargs.get("refine_inference", False)
+        self.mask_unseen = kwargs.get("mask_unseen", False)
+        self.old_backbone = None
+        self.old_seen_cls_num = 0
+        self.seen_cls_num = num_class
+        self.task_idx = 0
+        self.buffer = None
+        self.buffer_dataset = None
+        self.minibatch_size = kwargs.get("minibatch_size", kwargs.get("buffer_batch_size", 32))
+        self.online_buffer_update = kwargs.get("online_buffer_update", True)
+        self.class_balance = kwargs.get("class_balance", True)
+        self.print_buffer_stats = kwargs.get("print_buffer_stats", False)
+        self.buffer_update_after_step = kwargs.get("buffer_update_after_step", False)
+        self.buffer_storage = kwargs.get("buffer_storage", "raw")
+        self.ce_detach_signal = kwargs.get("ce_detach_signal", True)
+        self.distill_detach_signal = kwargs.get("distill_detach_signal", True)
+        self.old_model_eval = kwargs.get("old_model_eval", True)
+        self.freeze_old_model = kwargs.get("freeze_old_model", True)
+        self._pending_buffer_data = None
+        self._pending_buffer_labels = None
+        self._to_tensor = transforms.ToTensor()
+        self._to_pil = transforms.ToPILImage()
+
+    def _empty_signal(self, batch_size, upto=None):
+        upto = self.num_class if upto is None else upto
+        if hasattr(self.backbone, "empty_signal"):
+            return self.backbone.empty_signal(batch_size, self.device, upto=upto)
+        signal = torch.zeros(batch_size, self.num_class, device=self.device)
+        signal[:, :upto] = 1.0 / max(upto, 1)
+        return signal
+
+    def _one_hot(self, labels):
+        return F.one_hot(labels, num_classes=self.num_class).float()
+
+    def _sample_second_input(self, labels):
+        empty = self._empty_signal(labels.size(0))
+        one_hot = self._one_hot(labels)
+        if self.empty_prob >= 1:
+            return empty
+        if self.empty_prob <= 0:
+            return one_hot
+        use_empty = (torch.rand(1, device=self.device) <= self.empty_prob).item()
+        return empty if use_empty else one_hot
+
+    def _mask_logits(self, logits, upto=None):
+        if not self.mask_unseen:
+            return logits
+        upto = self.seen_cls_num if upto is None else upto
+        if upto >= logits.size(1):
+            return logits
+        masked = logits.clone()
+        masked[:, upto:] = -1e9
+        return masked
+
+    def _backbone_logits(self, x, y_signal=None, upto=None, backbone=None):
+        backbone = self.backbone if backbone is None else backbone
+        try:
+            output = backbone(x, y_signal, upto=upto)
+        except TypeError:
+            output = backbone(x, y_signal)
+
+        if isinstance(output, dict):
+            return output["logits"]
+        if isinstance(output, tuple):
+            return output[0]
+        return output
+
+    def _refined_logits(self, x):
+        empty = self._empty_signal(x.size(0))
+        logits0 = self._backbone_logits(x, empty)
+        logits0 = self._mask_logits(logits0)
+        if not self.refine_inference:
+            return logits0
+
+        probs0 = F.softmax(logits0, dim=1).detach()
+        logits1 = self._backbone_logits(x, probs0)
+        return self._mask_logits(logits1)
+
+    def before_task(self, task_idx, buffer, train_loader, test_loaders):
+        self.task_idx = task_idx
+        self.seen_cls_num = self.kwargs["init_cls_num"] + task_idx * self.kwargs["inc_cls_num"]
+        self.seen_cls_num = min(self.seen_cls_num, self.num_class)
+        self.buffer = buffer
+        self.buffer_dataset = train_loader.dataset if train_loader is not None else None
+        if buffer is not None and hasattr(buffer, "batch_size"):
+            self.minibatch_size = buffer.batch_size
+        if self.old_backbone is not None:
+            self.old_backbone = self.old_backbone.to(self.device)
+            if self.old_model_eval:
+                self.old_backbone.eval()
+            else:
+                self.old_backbone.train()
+
+    def _load_buffer_image(self, image_ref):
+        dataset = self.buffer_dataset
+        if dataset is None:
+            raise RuntimeError("IDER needs the train dataset to sample replay images.")
+
+        if torch.is_tensor(image_ref):
+            image = self._to_pil(image_ref.cpu())
+        elif dataset.dataset in ["binary_cifar10", "binary_cifar100"]:
+            image = Image.fromarray(np.uint8(image_ref))
+        elif dataset.dataset == "tiny-imagenet":
+            image = Image.open(image_ref).convert("RGB")
+        elif dataset.dataset == "processed_tinyimg":
+            if np.max(image_ref) <= 1.0:
+                image_ref = np.uint8(255 * image_ref)
+            image = Image.fromarray(np.uint8(image_ref)).convert("RGB")
+        else:
+            image = Image.open(os.path.join(dataset.data_root, dataset.mode, image_ref)).convert("RGB")
+        return dataset.trfms(image)
+
+    def _make_buffer_image_ref(self, image_ref):
+        if self.buffer_storage != "tensor":
+            return image_ref
+
+        if torch.is_tensor(image_ref):
+            return image_ref.detach().cpu()
+
+        dataset = self.buffer_dataset
+        if dataset.dataset in ["binary_cifar10", "binary_cifar100"]:
+            image = Image.fromarray(np.uint8(image_ref))
+        elif dataset.dataset == "tiny-imagenet":
+            image = Image.open(image_ref).convert("RGB")
+        elif dataset.dataset == "processed_tinyimg":
+            if np.max(image_ref) <= 1.0:
+                image_ref = np.uint8(255 * image_ref)
+            image = Image.fromarray(np.uint8(image_ref)).convert("RGB")
+        else:
+            image = Image.open(os.path.join(dataset.data_root, dataset.mode, image_ref)).convert("RGB")
+        return self._to_tensor(image).cpu()
+
+    def _sample_buffer_batch(self):
+        if self.buffer is None or self.buffer.is_empty():
+            return None
+
+        n_items = len(self.buffer.labels)
+        batch_size = min(self.minibatch_size, n_items)
+        indices = np.random.choice(n_items, size=batch_size, replace=False)
+        images = [self._load_buffer_image(self.buffer.images[idx]) for idx in indices]
+        labels = [self.buffer.labels[idx] for idx in indices]
+        return torch.stack(images).to(self.device), torch.tensor(labels, dtype=torch.long, device=self.device)
+
+    def _online_update_buffer(self, data, labels):
+        if (
+            not self.online_buffer_update
+            or self.buffer is None
+            or self.buffer.buffer_size <= 0
+            or self.buffer_dataset is None
+            or "index" not in data
+        ):
+            return
+
+        if not hasattr(self.buffer, "num_seen_examples"):
+            self.buffer.num_seen_examples = len(self.buffer.labels)
+
+        indices = data["index"].detach().cpu().numpy().tolist()
+        labels = labels.detach().cpu().numpy().tolist()
+
+        for dataset_idx, label in zip(indices, labels):
+            image_ref = self._make_buffer_image_ref(self.buffer_dataset.images[int(dataset_idx)])
+            seen = self.buffer.num_seen_examples
+            if len(self.buffer.labels) < self.buffer.buffer_size:
+                self.buffer.images.append(image_ref)
+                self.buffer.labels.append(int(label))
+            else:
+                replace_idx = np.random.randint(0, seen + 1)
+                if replace_idx < self.buffer.buffer_size:
+                    if self.class_balance and len(self.buffer.labels) > 0:
+                        counts = Counter(self.buffer.labels)
+                        max_count = max(counts.values())
+                        majority_classes = {cls for cls, cnt in counts.items() if cnt == max_count}
+                        majority_indices = [
+                            idx for idx, old_label in enumerate(self.buffer.labels)
+                            if old_label in majority_classes
+                        ]
+                        replace_idx = int(np.random.choice(majority_indices))
+                    self.buffer.images[replace_idx] = image_ref
+                    self.buffer.labels[replace_idx] = int(label)
+            self.buffer.num_seen_examples += 1
+
+    def _idempotent_ce_loss(self, x, y):
+        y_signal = self._sample_second_input(y)
+        logits1 = self._backbone_logits(x, y_signal)
+        signal_logits = logits1.detach() if self.ce_detach_signal else logits1
+        logits2 = self._backbone_logits(x, F.softmax(signal_logits, dim=1))
+        loss1 = self.loss_fn(self._mask_logits(logits1), y)
+        loss2 = self.loss_fn(self._mask_logits(logits2), y)
+        return 0.5 * (loss1 + loss2), logits1, logits2, y_signal
+
+    def _distill_loss(self, x):
+        uniform = self._empty_signal(x.size(0))
+        logits1 = self._backbone_logits(x, uniform)
+        signal_logits = logits1.detach() if self.distill_detach_signal else logits1
+        if self.old_model_eval:
+            self.old_backbone.eval()
+        else:
+            self.old_backbone.train()
+        logits2 = self._backbone_logits(
+            x,
+            F.softmax(signal_logits, dim=1),
+            backbone=self.old_backbone,
+        )
+        return F.mse_loss(logits1, logits2)
+
+    def observe(self, data):
+        x, y = data["image"], data["label"]
+        x = x.to(self.device)
+        y = y.to(self.device)
+
+        loss, logits1, logits2, _ = self._idempotent_ce_loss(x, y)
+
+        if self.old_backbone is not None and self.old_seen_cls_num > 0 and self.weightb > 0:
+            loss = loss + self.weightb * self._distill_loss(x)
+
+        if self.weightc != 0:
+            buffer_batch = self._sample_buffer_batch()
+            if buffer_batch is not None:
+                buffer_loss, _, _, _ = self._idempotent_ce_loss(*buffer_batch)
+                loss = loss + self.weightc * 2.0 * buffer_loss
+
+        if self.old_backbone is not None and self.old_seen_cls_num > 0 and self.weighta != 0:
+            buffer_batch = self._sample_buffer_batch()
+            if buffer_batch is not None:
+                loss = loss + self.weighta * self._distill_loss(buffer_batch[0])
+
+        pred = torch.argmax(self._mask_logits(logits2), dim=1)
+        acc = torch.sum(pred == y).item()
+        if self.buffer_update_after_step:
+            self._pending_buffer_data = data
+            self._pending_buffer_labels = y.detach()
+        else:
+            self._online_update_buffer(data, y)
+        return pred, acc / x.size(0), loss
+
+    def after_observe(self):
+        if self._pending_buffer_data is None or self._pending_buffer_labels is None:
+            return
+        self._online_update_buffer(self._pending_buffer_data, self._pending_buffer_labels)
+        self._pending_buffer_data = None
+        self._pending_buffer_labels = None
+
+    def _task_class_range(self, task_id):
+        if task_id < 0:
+            return 0, self.num_class
+        if task_id == 0:
+            start = 0
+            end = self.kwargs["init_cls_num"]
+        else:
+            start = self.kwargs["init_cls_num"] + (task_id - 1) * self.kwargs["inc_cls_num"]
+            end = start + self.kwargs["inc_cls_num"]
+        return start, min(end, self.num_class)
+
+    def inference(self, data, task_id=-1):
+        x, y = data["image"], data["label"]
+        x = x.to(self.device)
+        y = y.to(self.device)
+
+        logits = self._refined_logits(x)
+        if task_id > -1:
+            start, end = self._task_class_range(task_id)
+            task_logits = torch.full_like(logits, -1e9)
+            task_logits[:, start:end] = logits[:, start:end]
+            logits = task_logits
+        pred = torch.argmax(logits, dim=1)
+        acc = torch.sum(pred == y).item()
+        return pred, acc / x.size(0)
+
+    def predict_logits(self, data):
+        x = data["image"].to(self.device)
+        return self._refined_logits(x)
+
+    def forward(self, x):
+        return self._refined_logits(x)
+
+    def after_task(self, task_idx, buffer, train_loader, test_loaders):
+        if self.print_buffer_stats and buffer is not None and hasattr(buffer, "labels") and len(buffer.labels) > 0:
+            counts = Counter(buffer.labels)
+            seen_classes = self.kwargs["init_cls_num"] + task_idx * self.kwargs["inc_cls_num"]
+            seen_classes = min(seen_classes, self.num_class)
+            values = [counts.get(cls, 0) for cls in range(seen_classes)]
+            zero_classes = sum(1 for value in values if value == 0)
+            print(
+                "[IDER] Buffer class counts after task "
+                f"{task_idx}: size={len(buffer.labels)}, classes={len(counts)}, "
+                f"min={min(values)}, max={max(values)}, zero={zero_classes}, "
+                f"head={values[:min(20, len(values))]}"
+            )
+        self.old_backbone = copy.deepcopy(self.backbone).to(self.device)
+        if self.old_model_eval:
+            self.old_backbone.eval()
+        else:
+            self.old_backbone.train()
+        for param in self.old_backbone.parameters():
+            param.requires_grad = not self.freeze_old_model
+        self.old_seen_cls_num = self.seen_cls_num
+
+    def get_parameters(self, config):
+        return [{"params": self.backbone.parameters()}]

core/model/paper_er.py

@@ -0,0 +1,170 @@
+import os
+
+import numpy as np
+import torch
+from PIL import Image
+from torchvision import transforms
+
+from .finetune import Finetune
+
+
+class PaperER(Finetune):
+    """Online ER baseline used by the original IDER code.
+
+    Each training step optimizes CE on the current mini-batch concatenated with
+    a random replay mini-batch. The current mini-batch is inserted into the
+    replay memory by reservoir sampling after the optimizer step.
+    """
+
+    def __init__(self, backbone, feat_dim, num_class, **kwargs):
+        super().__init__(backbone, feat_dim, num_class, **kwargs)
+        self.buffer = None
+        self.buffer_dataset = None
+        self.minibatch_size = kwargs.get("minibatch_size", kwargs.get("buffer_batch_size", 32))
+        self.buffer_storage = kwargs.get("buffer_storage", "tensor")
+        self._pending_buffer_data = None
+        self._pending_buffer_labels = None
+        self._to_tensor = transforms.ToTensor()
+        self._to_pil = transforms.ToPILImage()
+
+    def before_task(self, task_idx, buffer, train_loader, test_loaders):
+        self.buffer = buffer
+        self.buffer_dataset = train_loader.dataset if train_loader is not None else None
+        if buffer is not None and hasattr(buffer, "batch_size"):
+            self.minibatch_size = buffer.batch_size
+
+    def _logits(self, x):
+        return self.classifier(self.backbone(x)["features"])
+
+    def _load_buffer_image(self, image_ref):
+        dataset = self.buffer_dataset
+        if dataset is None:
+            raise RuntimeError("PaperER needs the train dataset to sample replay images.")
+
+        if torch.is_tensor(image_ref):
+            image = self._to_pil(image_ref.cpu())
+        elif dataset.dataset in ["binary_cifar10", "binary_cifar100"]:
+            image = Image.fromarray(np.uint8(image_ref))
+        elif dataset.dataset == "tiny-imagenet":
+            image = Image.open(image_ref).convert("RGB")
+        elif dataset.dataset == "processed_tinyimg":
+            if np.max(image_ref) <= 1.0:
+                image_ref = np.uint8(255 * image_ref)
+            image = Image.fromarray(np.uint8(image_ref)).convert("RGB")
+        else:
+            image = Image.open(os.path.join(dataset.data_root, dataset.mode, image_ref)).convert("RGB")
+        return dataset.trfms(image)
+
+    def _make_buffer_image_ref(self, image_ref):
+        if self.buffer_storage != "tensor":
+            return image_ref
+
+        dataset = self.buffer_dataset
+        if torch.is_tensor(image_ref):
+            return image_ref.detach().cpu()
+        if dataset.dataset in ["binary_cifar10", "binary_cifar100"]:
+            image = Image.fromarray(np.uint8(image_ref))
+        elif dataset.dataset == "tiny-imagenet":
+            image = Image.open(image_ref).convert("RGB")
+        elif dataset.dataset == "processed_tinyimg":
+            if np.max(image_ref) <= 1.0:
+                image_ref = np.uint8(255 * image_ref)
+            image = Image.fromarray(np.uint8(image_ref)).convert("RGB")
+        else:
+            image = Image.open(os.path.join(dataset.data_root, dataset.mode, image_ref)).convert("RGB")
+        return self._to_tensor(image).cpu()
+
+    def _sample_buffer_batch(self):
+        if self.buffer is None or self.buffer.is_empty():
+            return None
+
+        batch_size = min(self.minibatch_size, len(self.buffer.labels))
+        indices = np.random.choice(len(self.buffer.labels), size=batch_size, replace=False)
+        images = [self._load_buffer_image(self.buffer.images[idx]) for idx in indices]
+        labels = [self.buffer.labels[idx] for idx in indices]
+        return torch.stack(images).to(self.device), torch.tensor(labels, dtype=torch.long, device=self.device)
+
+    def _online_update_buffer(self, data, labels):
+        if (
+            self.buffer is None
+            or self.buffer.buffer_size <= 0
+            or self.buffer_dataset is None
+            or "index" not in data
+        ):
+            return
+
+        if not hasattr(self.buffer, "num_seen_examples"):
+            self.buffer.num_seen_examples = len(self.buffer.labels)
+
+        indices = data["index"].detach().cpu().numpy().tolist()
+        labels = labels.detach().cpu().numpy().tolist()
+
+        for dataset_idx, label in zip(indices, labels):
+            image_ref = self._make_buffer_image_ref(self.buffer_dataset.images[int(dataset_idx)])
+            seen = self.buffer.num_seen_examples
+            if len(self.buffer.labels) < self.buffer.buffer_size:
+                self.buffer.images.append(image_ref)
+                self.buffer.labels.append(int(label))
+            else:
+                replace_idx = np.random.randint(0, seen + 1)
+                if replace_idx < self.buffer.buffer_size:
+                    self.buffer.images[replace_idx] = image_ref
+                    self.buffer.labels[replace_idx] = int(label)
+            self.buffer.num_seen_examples += 1
+
+    def observe(self, data):
+        x, y = data["image"].to(self.device), data["label"].to(self.device)
+        real_batch_size = x.size(0)
+
+        replay_batch = self._sample_buffer_batch()
+        if replay_batch is not None:
+            buf_x, buf_y = replay_batch
+            train_x = torch.cat((x, buf_x), dim=0)
+            train_y = torch.cat((y, buf_y), dim=0)
+        else:
+            train_x, train_y = x, y
+
+        logits = self._logits(train_x)
+        loss = self.loss_fn(logits, train_y)
+
+        current_logits = logits[:real_batch_size]
+        pred = torch.argmax(current_logits, dim=1)
+        acc = torch.sum(pred == y).item()
+
+        self._pending_buffer_data = data
+        self._pending_buffer_labels = y.detach()
+        return pred, acc / real_batch_size, loss
+
+    def after_observe(self):
+        if self._pending_buffer_data is None or self._pending_buffer_labels is None:
+            return
+        self._online_update_buffer(self._pending_buffer_data, self._pending_buffer_labels)
+        self._pending_buffer_data = None
+        self._pending_buffer_labels = None
+
+    def inference(self, data, task_id=-1):
+        x, y = data["image"].to(self.device), data["label"].to(self.device)
+        logits = self._logits(x)
+        if task_id > -1:
+            start, end = self._task_class_range(task_id)
+            task_logits = torch.full_like(logits, -1e9)
+            task_logits[:, start:end] = logits[:, start:end]
+            logits = task_logits
+        pred = torch.argmax(logits, dim=1)
+        acc = torch.sum(pred == y).item()
+        return pred, acc / x.size(0)
+
+    def _task_class_range(self, task_id):
+        if task_id < 0:
+            return 0, self.num_class
+        if task_id == 0:
+            start = 0
+            end = self.kwargs["init_cls_num"]
+        else:
+            start = self.kwargs["init_cls_num"] + (task_id - 1) * self.kwargs["inc_cls_num"]
+            end = start + self.kwargs["inc_cls_num"]
+        return start, min(end, self.num_class)
+
+    def predict_logits(self, data):
+        x = data["image"].to(self.device)
+        return self._logits(x)

core/trainer.py

@@ -165,19 +165,25 @@ class Trainer(object):
                 torch.optim, "optimizer", config, params=model.get_parameters(config)
             )
 
+        scheduler_config = config["lr_scheduler"]
+        if self.task_idx == 0 and "init_lr_scheduler" in config.keys():
+            scheduler_config = config["init_lr_scheduler"]
+
         # Check if the learning rate scheduler specified in the configuration is "CosineSchedule"
-        if config['lr_scheduler']['name'] == "CosineSchedule":
-            scheduler = CosineSchedule(optimizer, K=config['lr_scheduler']['kwargs']['K'])
-        elif config['lr_scheduler']['name'] == "PatienceSchedule":
-            scheduler = PatienceSchedule(optimizer, patience = config['lr_scheduler']['kwargs']['patience'], factor = config['lr_scheduler']['kwargs']['factor'])
-        elif config['lr_scheduler']['name'] == "Constant":
+        if scheduler_config['name'] == "CosineSchedule":
+            scheduler = CosineSchedule(optimizer, K=scheduler_config['kwargs']['K'])
+        elif scheduler_config['name'] == "PatienceSchedule":
+            scheduler = PatienceSchedule(optimizer, patience = scheduler_config['kwargs']['patience'], factor = scheduler_config['kwargs']['factor'])
+        elif scheduler_config['name'] == "Constant":
             scheduler = optim.lr_scheduler.LambdaLR(optimizer, lr_lambda=lambda e: 1)
-        elif config['lr_scheduler']['name'] == "CosineAnnealingWarmUp":
+        elif scheduler_config['name'] == "CosineAnnealingWarmUp":
             T_max = len(self.train_loader.get_loader(self.task_idx))
             T_max *= init_epoch if self.task_idx == 0 else config['epoch']
-            scheduler = CosineAnnealingWarmUp(optimizer, config['lr_scheduler']['kwargs']['warmup_length'], T_max)
+            scheduler = CosineAnnealingWarmUp(optimizer, scheduler_config['kwargs']['warmup_length'], T_max)
         else:
-            scheduler = get_instance(torch.optim.lr_scheduler, "lr_scheduler", config, optimizer=optimizer)
+            scheduler = getattr(torch.optim.lr_scheduler, scheduler_config["name"])(
+                optimizer=optimizer, **scheduler_config["kwargs"]
+            )
 
         return init_epoch, config['epoch'], optimizer, scheduler
 
@@ -302,7 +308,7 @@ class Trainer(object):
 
                 dataloader, val_bias_dataloader = self.model.spilt_and_update(dataloader, self.buffer, task_idx, self.config)
 
-            elif isinstance(self.buffer, (LinearBuffer, LinearHerdingBuffer)) and self.buffer.buffer_size > 0 and task_idx > 0:
+            elif method_name not in ["IDER", "PaperER"] and isinstance(self.buffer, (LinearBuffer, LinearHerdingBuffer)) and self.buffer.buffer_size > 0 and task_idx > 0:
                 datasets = dataloader.dataset
                 if isinstance(datasets.images, list):
                     datasets.images.extend(self.buffer.images)
@@ -371,7 +377,8 @@ class Trainer(object):
                                        'PRAKA', 
                                        'TRGP_CLIP', 
                                        'LoRAsub_DRS',
-                                       'CL_LoRA'
+                                       'CL_LoRA',
+                                       'FlyCL'
                         ]:
                         if self.rank == 0:
                             print(f" * Disabled validation for this method")
@@ -407,7 +414,7 @@ class Trainer(object):
                 model.after_task(task_idx, self.buffer, self.train_loader.get_loader(task_idx), self.test_loader.get_loader(task_idx))
 
             # Update Buffer
-            if method_name not in ['bic', 'ERACE', 'ERAML']:
+            if method_name not in ['bic', 'ERACE', 'ERAML', 'IDER', 'PaperER']:
                 self.buffer.total_classes += self.init_cls_num if task_idx == 0 else self.inc_cls_num
                 if self.buffer.buffer_size > 0:
                     if self.buffer.strategy == 'herding':
@@ -454,6 +461,7 @@ class Trainer(object):
 
                     #bias_scheduler.step()
 
+            paper_ece_sum, paper_ece_count = 0., 0
             for test_idx in range(testing_times):
                 if self.rank == 0:
                     print(f"================Test {test_idx+1}/{testing_times} of Task {task_idx}!================")
@@ -461,6 +469,9 @@ class Trainer(object):
                 test_acc = self._validate(task_idx)
 
                 batch_last_acc, per_task_acc = test_acc['avg_acc'], test_acc['per_task_acc']
+                if test_acc.get('paper_ece') is not None:
+                    paper_ece_sum += test_acc['paper_ece']
+                    paper_ece_count += 1
                 best_batch_last_acc = max(batch_last_acc, best_batch_last_acc)
 
                 task_last_acc = np.mean(per_task_acc)
@@ -474,6 +485,8 @@ class Trainer(object):
                     print(f" * [Task] Last Average Acc: {task_last_acc:.2f} (Best: {best_task_last_acc:.2f})")
                     print(f" * Forgetting: {frgt:.3f} (Best: {best_frgt:.3f})")
                     print(f" * Backward Transfer: {bwt:.2f} (Best: {best_bwt:.2f})")
+                    if test_acc.get('paper_ece') is not None:
+                        print(f" * [Paper] ECE: {test_acc['paper_ece']:.3f}")
                     print(f" * Per-Task Acc: {per_task_acc}")
 
                 batch_last_acc_list[task_idx] += batch_last_acc # avg_acc_list[task_idx] += avg_acc
@@ -492,6 +505,9 @@ class Trainer(object):
             task_last_acc = task_last_acc_list[task_idx]
 
             frgt, bwt = compute_frgt(acc_table, acc_table[task_idx], task_idx), compute_bwt(acc_table, acc_table[task_idx], task_idx)
+            paper_faa = compute_paper_faa(acc_table, task_idx)
+            paper_ff = compute_paper_ff(acc_table, task_idx)
+            paper_ece = paper_ece_sum / paper_ece_count if paper_ece_count > 0 else None
             best_frgt, best_bwt = min(frgt, best_frgt), max(bwt, best_bwt)
             if task_idx > 1:
                 frgt_list.append(frgt)
@@ -503,6 +519,10 @@ class Trainer(object):
                 print(f" * [Task] Last Average Acc: {task_last_acc:.2f} (Best: {best_task_last_acc:.2f})")
                 print(f" * Forgetting: {frgt:.3f} (Best: {best_frgt:.3f})")
                 print(f" * Backward Transfer: {bwt:.2f} (Best: {best_bwt:.2f})")
+                print(f" * [Paper] FAA/CIL: {paper_faa:.2f}")
+                print(f" * [Paper] FF: {paper_ff:.3f}")
+                if paper_ece is not None:
+                    print(f" * [Paper] ECE: {paper_ece:.3f}")
                 print(f" * Per-Task Acc: {acc_table[task_idx][:task_idx + 1]}")
 
         batch_ovr_avg_acc = np.mean(batch_last_acc_list) #batch_ovr_avg_acc = np.mean(avg_acc_list)
@@ -512,6 +532,8 @@ class Trainer(object):
         
         ovr_bwt = np.mean(bwt_list) if len(bwt_list) > 0 else float('-inf')
         ovr_frgt = np.mean(frgt_list) if len(frgt_list) > 0 else float('inf')
+        paper_faa = compute_paper_faa(acc_table, task_idx)
+        paper_ff = compute_paper_ff(acc_table, task_idx)
 
         if self.rank == 0:
             print(f"================Overall Result of {self.task_num} Tasks!================")
@@ -523,6 +545,10 @@ class Trainer(object):
             print(f" * [Task] Overall Avg Acc : {task_ovr_avg_acc:.2f}")
             print(f" * Overall Frgt : {ovr_frgt:.3f}")
             print(f" * Overall BwT : {ovr_bwt:.2f}")
+            print(f" * [Paper] Final Average Accuracy (FAA/CIL) : {paper_faa:.2f}")
+            print(f" * [Paper] Final Forgetting (FF) : {paper_ff:.3f}")
+            if 'paper_ece' in locals() and paper_ece is not None:
+                print(f" * [Paper] Expected Calibration Error (ECE) : {paper_ece:.3f}")
             print(f" * Average Acc Table : \n{acc_table}")
 
             print(f"================Model Performance Analysis================")
@@ -605,6 +631,9 @@ class Trainer(object):
 
             self.optimizer.step()
 
+            if hasattr(model, 'after_observe'):
+                model.after_observe()
+
             if self.config["classifier"]["name"] in ['ERACE', 'ERAML']:
                 model.add_reservoir()
 
@@ -626,6 +655,24 @@ class Trainer(object):
 
         per_task_acc = []
         count_all, correct_all = 0, 0
+        ece_confidences, ece_correctness = [], []
+
+        def _infer_batch(batch, task_id=None):
+            if self.config['setting'] == 'task-agnostic' and hasattr(model, 'predict_logits'):
+                logits = model.predict_logits(batch)
+                probs = torch.softmax(logits, dim=1)
+                confidences, preds = torch.max(probs, dim=1)
+                labels = batch['label'].to(self.device)
+                correct = preds == labels
+                ece_confidences.extend(confidences.detach().cpu().numpy().tolist())
+                ece_correctness.extend(correct.detach().cpu().numpy().tolist())
+                return preds, correct.float().mean().item()
+
+            if self.config['setting'] == 'task-aware':
+                return model.inference(batch, task_id=task_id)
+            if self.config['setting'] == 'task-agnostic':
+                return model.inference(batch)
+            raise ValueError(f"Unsupported setting: {self.config['setting']}")
 
         if self.config['testing_per_task']:
 
@@ -636,13 +683,9 @@ class Trainer(object):
                     correct_task, count_task = 0, 0
 
                     for b, batch in tqdm(enumerate(dataloader), total=len(dataloader), desc = f"Testing on Task {t} data", disable=self.rank != 0):  # Disable tqdm for non-master processes
+                        output, acc = _infer_batch(batch, task_id=t)
                         
-                        if self.config['setting'] == 'task-aware':
-                            output, acc = model.inference(batch, task_id=t)
-                        elif self.config['setting'] == 'task-agnostic':
-                            output, acc = model.inference(batch)
-                        
-                        correct_task += int(acc * batch['label'].shape[0])
+                        correct_task += int(round(acc * batch['label'].shape[0]))
                         count_task += batch['label'].shape[0]
 
                     correct_all += correct_task
@@ -693,9 +736,7 @@ class Trainer(object):
                     if self.config['setting'] == 'task-aware':
                         print('Mostly methods dont support this, set testing_per_task to False')
                         raise NotImplementedError
-                        output, acc = model.inference(batch, task_id=None)
-                    elif self.config['setting'] == 'task-agnostic':
-                        output, acc = model.inference(batch)
+                    output, acc = _infer_batch(batch)
                     preds = output.cpu().numpy()
 
                     labels = batch['label'].cpu().numpy()
@@ -714,7 +755,10 @@ class Trainer(object):
 
         avg_acc = round(correct_all * 100 / count_all, 2)
 
+        paper_ece = compute_ece(ece_confidences, ece_correctness)
+
         return {
             "avg_acc": avg_acc,
-            "per_task_acc": per_task_acc
+            "per_task_acc": per_task_acc,
+            "paper_ece": paper_ece,
         }

core/utils/utils.py

@@ -22,8 +22,7 @@ class AverageMeter(object):
         self.reset()
 
     def reset(self):
-        for col in self._data.columns:
-            self._data[col].values[:] = 0
+        self._data.loc[:, :] = 0
 
     def update(self, key, value, n=1):
         if self.writer is not None:
@@ -200,7 +199,7 @@ def fmt_date_str(date=None, fmt="%y-%m-%d-%H-%M-%S"):
     return date.strftime(fmt)
 
 def compute_bwt(acc_table, curr_acc, task_idx):
-    '''
+    r'''
     After training T tasks, $BWT = \frac{\sum_{i=3}^T\sum_{j=1}^{i-2}R_{i,j}-R{j,j}}{T(T-1)/2}$
     Equivalent to Positive BwT of Continuum
     https://arxiv.org/pdf/1810.13166
@@ -222,7 +221,7 @@ def compute_bwt(acc_table, curr_acc, task_idx):
 
 
 def compute_frgt(acc_table, curr_acc, task_idx):
-    '''
+    r'''
     After training T tasks, $Frgt = \frac{\sum_{j=1}^{T-2}R_{T-1,j}-R_{j,j}}{T-1}$
     Equivalent to Forgetting of Continuum
     '''
@@ -232,6 +231,47 @@ def compute_frgt(acc_table, curr_acc, task_idx):
     return 0.
 
 
+def compute_paper_faa(acc_table, task_idx):
+    """
+    Final Average Accuracy (FAA) used by IDER: average accuracy over all
+    tasks observed so far after finishing the current task.
+    """
+    return float(np.mean(acc_table[task_idx, :task_idx + 1]))
+
+
+def compute_paper_ff(acc_table, task_idx):
+    """
+    Final Forgetting (FF) used by IDER:
+    mean over old tasks of historical best accuracy minus final accuracy.
+    """
+    if task_idx <= 0:
+        return 0.
+
+    old_task_acc = acc_table[:task_idx + 1, :task_idx]
+    best_old_acc = np.max(old_task_acc, axis=0)
+    final_old_acc = acc_table[task_idx, :task_idx]
+    return float(np.mean(best_old_acc - final_old_acc))
+
+
+def compute_ece(confidences, correctness, n_bins=15):
+    """Expected Calibration Error in the IDER paper."""
+    if len(confidences) == 0:
+        return None
+
+    confidences = np.asarray(confidences)
+    correctness = np.asarray(correctness).astype(float)
+    ece = 0.
+    bin_boundaries = np.linspace(0., 1., n_bins + 1)
+    for bin_lower, bin_upper in zip(bin_boundaries[:-1], bin_boundaries[1:]):
+        in_bin = (confidences > bin_lower) & (confidences <= bin_upper)
+        prop_in_bin = np.mean(in_bin)
+        if prop_in_bin > 0:
+            acc_in_bin = np.mean(correctness[in_bin])
+            conf_in_bin = np.mean(confidences[in_bin])
+            ece += prop_in_bin * abs(conf_in_bin - acc_in_bin)
+    return float(ece * 100)
+
+
 def compute_fps(model, config):
     model.eval()
 
@@ -254,4 +294,4 @@ def compute_fps(model, config):
         t_all.append(t2 - t1)
 
     return {'avg_fps' : 1 / np.mean(t_all),
-            'best_fps' : 1 / min(t_all)}
+            'best_fps' : 1 / min(t_all)}

reproduce/ewcdr/README.md

@@ -0,0 +1,160 @@
+# EWC-DR: Elastic Weight Consolidation Done Right for Continual Learning [(CVPR'2026)](https://arxiv.org/abs/2603.18596)
+
+## Abstract
+
+Weight-regularization methods alleviate catastrophic forgetting by estimating
+which parameters are important for previous tasks and penalizing their changes.
+Elastic Weight Consolidation (EWC) is a classic method in this family, but its
+Fisher-based importance estimation can suffer from vanishing gradients when the
+model is already confident on old-task samples. EWC-DR addresses this issue with
+a simple Logits Reversal operation during importance estimation:
+
+```python
+loss = cross_entropy(-logits, labels)
+```
+
+This migrated implementation follows the official EWC-DR logic in the
+LibContinual training interface. It keeps the expanding classifier, new-class
+cross entropy for incremental tasks, clipped importance weights, and
+exemplar-free class-incremental learning setting.
+
+
+## Citation
+
+```bibtex
+@inproceedings{liu2026elastic,
+  title={Elastic Weight Consolidation Done Right for Continual Learning},
+  author={Liu, Xuan and Chang, Xiaobin},
+  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+  year={2026}
+}
+```
+
+## How to Reproduce
+
+- **Step 1: Install dependencies**
+
+    Use the repository dependency file:
+
+    ```bash
+    conda create -n libcontinual-ewcdr python=3.8 -y
+    conda activate libcontinual-ewcdr
+    pip install -r requirements.txt
+    ```
+
+- **Step 2: Prepare CIFAR-100**
+
+    The provided configs use `binary_cifar100`. CIFAR-100 will be downloaded to:
+
+    ```text
+    ./data/binary_cifar100/cifar-100-python
+    ```
+
+- **Step 3: Run one setting**
+
+    ```bash
+    python run_trainer.py --config ewcdr-resnet18-cifar100-b10-10-10.yaml --device 0
+    python run_trainer.py --config ewcdr-resnet18-cifar100-b50-5-11.yaml --device 0
+    ```
+
+- **Step 4: Run all CIFAR-100 Table 1 EWC-DR settings**
+
+    ```bash
+    python reproduce/ewcdr/run_cifar100_table1.py --device 0
+    ```
+
+    To reproduce the paper-style mean over three independent trials:
+
+    ```bash
+    python reproduce/ewcdr/run_cifar100_table1.py --device 0 --seeds 1993 1994 1995
+    ```
+
+- **Step 5: Run ImageNet-Subset Table 2 or Tiny-ImageNet Table 3**
+
+    Prepare ImageNet-Subset in ImageFolder layout under
+    `./data/ImageNet-100/imagenet-100/{train,val}/<class>/`. Tiny-ImageNet is
+    downloaded by the existing `tiny-imagenet` data backend under
+    `./data/tiny-imagenet-200` (`data_root` itself is `./data`, because the
+    backend appends the dataset directory name).
+
+    ```bash
+    python reproduce/ewcdr/run_imagenet_subset_table2.py --device 0
+    python reproduce/ewcdr/run_tiny_imagenet_table3.py --device 0
+    ```
+
+    Use `--seeds 1993 1994 1995` for the paper-style three-run mean, or
+    `--dry-run` to inspect all generated commands. The runners execute two jobs
+    concurrently per GPU by default. Pass multiple IDs, for example
+    `--device 0 1 2 --max-parallel 2`, to run at most two jobs on each GPU
+    (six total). They archive stable logs plus `summary.md` under
+    `reproduce/ewcdr/logs/<dataset_table>/`.
+
+## Settings
+
+| Paper Setting | LibContinual Config | Class Split |
+| :---: | :--- | :--- |
+| Big T=5 | `ewcdr-resnet18-cifar100-b50-10-6.yaml` | 50 initial + 5 increments x 10 classes |
+| Big T=10 | `ewcdr-resnet18-cifar100-b50-5-11.yaml` | 50 initial + 10 increments x 5 classes |
+| Big T=20 | `ewcdr-resnet18-cifar100-b40-3-21.yaml` | 40 initial + 20 increments x 3 classes |
+| Eq T=5 | `ewcdr-resnet18-cifar100-b20-20-5.yaml` | 5 tasks x 20 classes |
+| Eq T=10 | `ewcdr-resnet18-cifar100-b10-10-10.yaml` | 10 tasks x 10 classes |
+| Eq T=20 | `ewcdr-resnet18-cifar100-b5-5-20.yaml` | 20 tasks x 5 classes |
+
+ImageNet-Subset uses the same six 100-class splits as CIFAR-100. Its configs
+are named `ewcdr-resnet18-imagenet100-*.yaml`. Tiny-ImageNet uses:
+
+| Paper Setting | LibContinual Config | Class Split |
+| :---: | :--- | :--- |
+| Big T=5 | `ewcdr-resnet18-tinyimagenet-b100-20-6.yaml` | 100 initial + 5 increments x 20 classes |
+| Big T=10 | `ewcdr-resnet18-tinyimagenet-b100-10-11.yaml` | 100 initial + 10 increments x 10 classes |
+| Big T=20 | `ewcdr-resnet18-tinyimagenet-b100-5-21.yaml` | 100 initial + 20 increments x 5 classes |
+| Eq T=5 | `ewcdr-resnet18-tinyimagenet-b40-40-5.yaml` | 5 tasks x 40 classes |
+| Eq T=10 | `ewcdr-resnet18-tinyimagenet-b20-20-10.yaml` | 10 tasks x 20 classes |
+| Eq T=20 | `ewcdr-resnet18-tinyimagenet-b10-10-20.yaml` | 20 tasks x 10 classes |
+
+## Results
+
+The reproduced results below are single-run results with `seed=1993`. The paper
+reports means over three independent trials.
+
+### CIFAR-100
+
+| Setting | Seed | LibContinual A_last | Paper A_last | Diff | LibContinual A_avg | Paper A_avg | Diff |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| Big start / T=5 | 1993 | 50.87 | 50.23 | **+0.64** | 64.48 | 63.75 | **+0.73** |
+| Big start / T=10 | 1993 | 46.91 | 44.88 | **+2.03** | 61.09 | 60.94 | **+0.15** |
+| Big start / T=20 | 1993 | 36.02 | 35.86 | **+0.16** | 52.80 | 53.45 | **-0.65** |
+| Equally split / T=5 | 1993 | 46.45 | 46.89 | **-0.44** | 61.42 | 61.47 | **-0.05** |
+| Equally split / T=10 | 1993 | 30.34 | 29.41 | **+0.93** | 47.45 | 46.01 | **+1.44** |
+| Equally split / T=20 | 1993 | 19.15 | 18.00 | **+1.15** | 34.44 | 33.52 | **+0.92** |
+
+### Tiny-ImageNet
+
+| Setting | Seed | LibContinual A_last | Paper A_last | Diff | LibContinual A_avg | Paper A_avg | Diff |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| Big start / T=5 | 1993 | 39.59 | 38.24 | **+1.35** | 47.47 | 47.00 | **+0.47** |
+| Big start / T=10 | 1993 | 35.48 | 31.43 | **+4.05** | 45.13 | 42.88 | **+2.25** |
+| Big start / T=20 | 1993 | 30.69 | 23.64 | **+7.05** | 41.32 | 37.56 | **+3.76** |
+| Equally split / T=5 | 1993 | 26.46 | 28.67 | **-2.21** | 38.34 | 39.52 | **-1.18** |
+| Equally split / T=10 | 1993 | 21.46 | 21.46 | **+0.00** | 34.30 | 32.79 | **+1.51** |
+| Equally split / T=20 | 1993 | 13.60 | 12.09 | **+1.51** | 24.50 | 22.62 | **+1.88** |
+
+
+The LibContinual metrics correspond to the final log fields:
+
+| Paper Metric | LibContinual Log Field |
+| :---: | :--- |
+| `A_last` | `[Batch] Last Average Acc` |
+| `A_avg` | `[Batch] Overall Avg Acc` |
+
+## Notes
+
+- The official big-start `T` counts incremental phases, while LibContinual
+  `task_num` includes the initial phase. For example, Big T=10 is configured as
+  `task_num=11`.
+- `binary_cifar100` must follow the same shuffled class order and label remap as
+  the official implementation; this is handled in the migrated data pipeline.
+- Initial and incremental stages use different schedulers, matching the official
+  setup.
+- The Table 1 runner can launch multiple jobs on the same GPU for convenience.
+  For strict reporting, prefer three seeds and avoid overloading one GPU.

reproduce/fly_cl/run_cifar100_ablations.py

@@ -0,0 +1,135 @@
+"""Run the five Fly-CL component ablations on CIFAR-100.
+
+The script caches the frozen ViT features once for each input-normalization
+setting and delegates every analytic update and prediction to ``FlyCL``.
+"""
+
+import argparse
+import json
+import math
+import os
+import sys
+from pathlib import Path
+
+import torch
+
+ROOT = Path(__file__).resolve().parents[2]
+if str(ROOT) not in sys.path:
+    sys.path.insert(0, str(ROOT))
+
+from core.config import Config
+from reproduce.fly_cl.run_cub200_ablations import (
+    COMPONENT_EXPERIMENTS,
+    load_or_build_cache,
+    run_experiment,
+    save_results,
+)
+
+
+EXPECTED_EXPERIMENTS = {spec["name"] for spec in COMPONENT_EXPERIMENTS}
+
+
+def validate_complete_results(results):
+    by_name = {item["name"]: item for item in results}
+    if set(by_name) != EXPECTED_EXPERIMENTS:
+        missing = EXPECTED_EXPERIMENTS - set(by_name)
+        extra = set(by_name) - EXPECTED_EXPERIMENTS
+        raise ValueError(
+            f"Incomplete CIFAR sweep; missing={sorted(missing)}, "
+            f"extra={sorted(extra)}"
+        )
+    if len(by_name) != len(results):
+        raise ValueError("Duplicate CIFAR experiment names")
+    for name, item in by_name.items():
+        if len(item["stage_average_accuracy"]) != 10:
+            raise ValueError(f"{name}: expected 10 stage accuracies")
+        if [len(row) for row in item["accuracy_matrix"]] != list(range(1, 11)):
+            raise ValueError(f"{name}: malformed triangular accuracy matrix")
+        metrics = (
+            item["overall_accuracy"],
+            item["last_stage_accuracy"],
+            item["bwt"],
+            item["wall_seconds"],
+        )
+        if not all(math.isfinite(value) for value in metrics):
+            raise ValueError(f"{name}: non-finite metric")
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--config", default="config/fly_cl_cifar100.yaml")
+    parser.add_argument("--device", type=int, default=0)
+    parser.add_argument("--rebuild-cache", action="store_true")
+    parser.add_argument(
+        "--output", default="outputs/fly_cl_cifar100_ablations.json"
+    )
+    parser.add_argument("--only", nargs="*")
+    parser.add_argument("--rerun", nargs="*", default=())
+    args = parser.parse_args()
+
+    os.chdir(ROOT)
+    config = Config(args.config).get_config_dict()
+    device = torch.device(f"cuda:{args.device}")
+    cache_dir = ROOT / "outputs" / "fly_cl_cifar100_feature_cache"
+    normalized_cache = load_or_build_cache(
+        config,
+        True,
+        cache_dir / f"normalized_seed{config['seed']}.pt",
+        device,
+        args.rebuild_cache,
+    )
+    no_norm_cache = load_or_build_cache(
+        config,
+        False,
+        cache_dir / f"no_normalization_seed{config['seed']}.pt",
+        device,
+        args.rebuild_cache,
+    )
+
+    specs = list(COMPONENT_EXPERIMENTS)
+    if args.only:
+        requested = set(args.only)
+        unknown = requested - EXPECTED_EXPERIMENTS
+        if unknown:
+            raise ValueError(f"Unknown CIFAR experiments: {sorted(unknown)}")
+        specs = [spec for spec in specs if spec["name"] in requested]
+    rerun = set(args.rerun)
+    unknown_reruns = rerun - {spec["name"] for spec in specs}
+    if unknown_reruns:
+        raise ValueError(f"Unknown CIFAR reruns: {sorted(unknown_reruns)}")
+
+    output = ROOT / args.output
+    previous = {}
+    if output.exists():
+        with output.open("r", encoding="utf-8") as handle:
+            previous = {
+                item["name"]: item for item in json.load(handle)["experiments"]
+            }
+    completed = dict(previous)
+    metadata = {
+        "dataset": config["dataset"],
+        "seed": config["seed"],
+        "train_images": sum(len(x["labels"]) for x in normalized_cache["train"]),
+        "test_images": sum(len(x["labels"]) for x in normalized_cache["test"]),
+        "normalized_feature_seconds": normalized_cache["feature_seconds"],
+        "no_normalization_feature_seconds": no_norm_cache["feature_seconds"],
+    }
+
+    for spec in specs:
+        if spec["name"] in previous and spec["name"] not in rerun:
+            print(f"Skipping completed experiment: {spec['name']}")
+            continue
+        cache = normalized_cache if spec.get("normalization", True) else no_norm_cache
+        print(f"Running CIFAR-100 experiment: {spec['name']}")
+        completed[spec["name"]] = run_experiment(config, spec, cache, device)
+        save_results(output, metadata, list(completed.values()))
+
+    results = list(completed.values())
+    if set(completed) == EXPECTED_EXPERIMENTS:
+        validate_complete_results(results)
+    save_results(output, metadata, results)
+    print(f"Saved {len(results)} CIFAR-100 experiments to {output}")
+
+
+if __name__ == "__main__":
+    main()

reproduce/fly_cl/run_cub200_ablations.py

@@ -0,0 +1,380 @@
+"""Run the Fly-CL CUB-200 component ablations and sensitivity sweeps.
+
+The frozen ViT features are cached once per input-normalization setting.  Every
+analytic experiment still uses ``core.model.fly_cl.FlyCL`` for projection,
+streaming statistics, GCV, Cholesky solving, prototype construction, and
+inference.  This removes repeated backbone inference without changing the
+algorithm under study.
+"""
+
+import argparse
+import copy
+import gc
+import json
+import math
+import os
+import sys
+import time
+from pathlib import Path
+
+import torch
+import torch.nn as nn
+
+ROOT = Path(__file__).resolve().parents[2]
+if str(ROOT) not in sys.path:
+    sys.path.insert(0, str(ROOT))
+
+from core.config import Config
+from core.data import get_dataloader
+from core.model.backbone.timm_backbone import timm_backbone
+from core.model.fly_cl import FlyCL
+from core.utils import init_seed
+
+
+COMPONENT_EXPERIMENTS = [
+    {"name": "full", "normalization": True},
+    {
+        "name": "wo_projection",
+        "normalization": True,
+        "use_projection": False,
+        # Removing the expansion changes the feature scale by orders of
+        # magnitude.  A broad GCV grid is required for a meaningful ablation.
+        "ridge_lower": 0,
+        "ridge_upper": 10,
+    },
+    {"name": "wo_ridge", "normalization": True, "use_ridge": False},
+    {"name": "wo_normalization", "normalization": False},
+    {
+        "name": "wo_all",
+        "normalization": False,
+        "use_projection": False,
+        "use_ridge": False,
+    },
+]
+
+SENSITIVITY_EXPERIMENTS = (
+    [
+        {"name": f"m_{m}", "group": "expand_dim", "expand_dim": m}
+        for m in (500, 1000, 2000, 5000, 10000, 20000)
+    ]
+    + [
+        {"name": f"p_{p}", "group": "synaptic_degree", "synaptic_degree": p}
+        for p in (50, 100, 200, 300, 500, 700, 768)
+    ]
+    + [
+        {
+            "name": f"k_{k}",
+            "group": "activation_sparsity",
+            "coding_level": k / 10000,
+            "coding_k": k,
+        }
+        for k in (500, 1000, 2000, 3000, 4000, 5000, 7000, 8000, 9000, 10000)
+    ]
+)
+
+EXPECTED_EXPERIMENTS = {
+    spec["name"] for spec in COMPONENT_EXPERIMENTS + SENSITIVITY_EXPERIMENTS
+}
+
+
+class IdentityBackbone(nn.Module):
+    def __init__(self, feat_dim):
+        super().__init__()
+        self.feat_dim = feat_dim
+
+    def forward(self, features):
+        return features
+
+
+def _without_normalization(config):
+    config = copy.deepcopy(config)
+    for key in ("train_trfms", "test_trfms"):
+        config[key] = [item for item in config[key] if "Normalize" not in item]
+    return config
+
+
+@torch.no_grad()
+def _extract_split(backbone, loaders, device):
+    tasks = []
+    for task_idx, loader in enumerate(loaders.dataloaders):
+        features, labels = [], []
+        for batch in loader:
+            images = batch["image"].to(device, non_blocking=True)
+            features.append(backbone(images).cpu())
+            labels.append(batch["label"].cpu())
+        tasks.append(
+            {
+                "features": torch.cat(features),
+                "labels": torch.cat(labels),
+                "task": task_idx,
+            }
+        )
+    return tasks
+
+
+def build_feature_cache(config, normalized, cache_path, device):
+    variant = config if normalized else _without_normalization(config)
+    init_seed(variant["seed"], variant["deterministic"])
+    backbone = timm_backbone(**variant["backbone"]["kwargs"]).to(device).eval()
+
+    # FlyCL constructs its random projection immediately after the backbone.
+    # Saving this state lets cached-feature experiments reproduce that ordering.
+    projection_rng_state = torch.get_rng_state().clone()
+    train_loaders = get_dataloader(variant, "train")
+    test_loaders = get_dataloader(
+        variant, "test", cls_map=train_loaders.cls_map
+    )
+    started = time.perf_counter()
+    train = _extract_split(backbone, train_loaders, device)
+    test = _extract_split(backbone, test_loaders, device)
+    cache = {
+        "normalized": normalized,
+        "seed": variant["seed"],
+        "class_map": train_loaders.cls_map,
+        "projection_rng_state": projection_rng_state,
+        "train": train,
+        "test": test,
+        "feature_seconds": time.perf_counter() - started,
+    }
+    cache_path.parent.mkdir(parents=True, exist_ok=True)
+    torch.save(cache, cache_path)
+    del backbone, train_loaders, test_loaders
+    torch.cuda.empty_cache()
+    return cache
+
+
+def load_or_build_cache(config, normalized, cache_path, device, rebuild=False):
+    if cache_path.exists() and not rebuild:
+        cache = torch.load(cache_path, map_location="cpu")
+        if cache.get("seed") != config["seed"]:
+            raise ValueError(f"Stale cache seed in {cache_path}")
+        return cache
+    return build_feature_cache(config, normalized, cache_path, device)
+
+
+@torch.no_grad()
+def evaluate_task(model, task, device, batch_size=512):
+    correct = 0
+    total = len(task["labels"])
+    for start in range(0, total, batch_size):
+        stop = min(start + batch_size, total)
+        features = task["features"][start:stop].to(device)
+        labels = task["labels"][start:stop]
+        logits, _ = model.predict_features(features)
+        correct += logits.argmax(dim=1).cpu().eq(labels).sum().item()
+    return 100.0 * correct / total
+
+
+def run_experiment(base_config, spec, cache, device):
+    kwargs = copy.deepcopy(base_config["classifier"]["kwargs"])
+    kwargs.update(
+        {
+            key: value
+            for key, value in spec.items()
+            if key
+            in {
+                "expand_dim",
+                "synaptic_degree",
+                "coding_level",
+                "ridge_lower",
+                "ridge_upper",
+                "use_projection",
+                "use_ridge",
+            }
+        }
+    )
+    torch.set_rng_state(cache["projection_rng_state"].clone())
+    model = FlyCL(
+        IdentityBackbone(kwargs["embedding_dim"]), device, **kwargs
+    ).to(device)
+
+    accuracy_matrix = []
+    stage_average = []
+    ridge_values = []
+    post_seconds = []
+    started = time.perf_counter()
+    for task_idx, train_task in enumerate(cache["train"]):
+        model.before_task(task_idx, None, None, None)
+        task_started = time.perf_counter()
+        model.fit_features(
+            train_task["features"].to(device), train_task["labels"].to(device)
+        )
+        post_seconds.append(time.perf_counter() - task_started)
+        ridge_values.append(model.last_ridge)
+        row = [
+            evaluate_task(model, cache["test"][test_idx], device)
+            for test_idx in range(task_idx + 1)
+        ]
+        accuracy_matrix.append(row)
+        stage_average.append(sum(row) / len(row))
+        print(
+            f"[{spec['name']}] task={task_idx + 1}/10 "
+            f"A_t={stage_average[-1]:.4f}"
+        )
+
+    final_row = accuracy_matrix[-1]
+    bwt = sum(
+        final_row[idx] - accuracy_matrix[idx][idx]
+        for idx in range(len(final_row) - 1)
+    ) / (len(final_row) - 1)
+    result = {
+        **spec,
+        "stage_average_accuracy": stage_average,
+        "accuracy_matrix": accuracy_matrix,
+        "overall_accuracy": sum(stage_average) / len(stage_average),
+        "last_stage_accuracy": stage_average[-1],
+        "bwt": bwt,
+        "ridge_values": ridge_values,
+        "post_seconds_per_task": post_seconds,
+        "wall_seconds": time.perf_counter() - started,
+    }
+    del model
+    gc.collect()
+    torch.cuda.empty_cache()
+    return result
+
+
+def save_results(path, metadata, results):
+    path.parent.mkdir(parents=True, exist_ok=True)
+    payload = {"metadata": metadata, "experiments": results}
+    with path.open("w", encoding="utf-8") as handle:
+        json.dump(payload, handle, ensure_ascii=False, indent=2)
+
+
+def validate_complete_results(results):
+    """Fail loudly if a supposedly complete sweep is missing or malformed."""
+    by_name = {item["name"]: item for item in results}
+    if set(by_name) != EXPECTED_EXPERIMENTS:
+        missing = EXPECTED_EXPERIMENTS - set(by_name)
+        extra = set(by_name) - EXPECTED_EXPERIMENTS
+        raise ValueError(f"Incomplete sweep; missing={sorted(missing)}, extra={sorted(extra)}")
+    if len(by_name) != len(results):
+        raise ValueError("Duplicate experiment names in result file")
+    for name, item in by_name.items():
+        if len(item["stage_average_accuracy"]) != 10:
+            raise ValueError(f"{name}: expected 10 stage accuracies")
+        if [len(row) for row in item["accuracy_matrix"]] != list(range(1, 11)):
+            raise ValueError(f"{name}: malformed triangular accuracy matrix")
+        numeric = [
+            item["overall_accuracy"],
+            item["last_stage_accuracy"],
+            item["bwt"],
+            item["wall_seconds"],
+        ]
+        if not all(math.isfinite(value) for value in numeric):
+            raise ValueError(f"{name}: non-finite metric")
+
+    reference = by_name["full"]["accuracy_matrix"]
+    for duplicate in ("m_10000", "p_300", "k_3000"):
+        if by_name[duplicate]["accuracy_matrix"] != reference:
+            raise ValueError(f"{duplicate} does not reproduce the default full model")
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--config", default="config/fly_cl_cub200.yaml")
+    parser.add_argument("--device", type=int, default=0)
+    parser.add_argument("--rebuild-cache", action="store_true")
+    parser.add_argument(
+        "--scope", choices=("all", "components", "sensitivity"), default="all"
+    )
+    parser.add_argument(
+        "--output", default="outputs/fly_cl_cub200_ablations.json"
+    )
+    parser.add_argument(
+        "--only",
+        nargs="*",
+        help="Optional experiment names, for example: --only full wo_ridge m_500",
+    )
+    parser.add_argument(
+        "--rerun",
+        nargs="*",
+        default=(),
+        help="Recompute named experiments even when they exist in the output JSON",
+    )
+    args = parser.parse_args()
+
+    root = ROOT
+    os.chdir(root)
+    config = Config(args.config).get_config_dict()
+    device = torch.device(f"cuda:{args.device}")
+    cache_dir = root / "outputs" / "fly_cl_cub200_feature_cache"
+    normalized_cache = load_or_build_cache(
+        config,
+        True,
+        cache_dir / f"normalized_seed{config['seed']}.pt",
+        device,
+        args.rebuild_cache,
+    )
+    no_norm_cache = None
+    if args.scope in ("all", "components"):
+        no_norm_cache = load_or_build_cache(
+            config,
+            False,
+            cache_dir / f"no_normalization_seed{config['seed']}.pt",
+            device,
+            args.rebuild_cache,
+        )
+
+    specs = []
+    if args.scope in ("all", "components"):
+        specs.extend(COMPONENT_EXPERIMENTS)
+    if args.scope in ("all", "sensitivity"):
+        specs.extend(SENSITIVITY_EXPERIMENTS)
+    if args.only:
+        requested = set(args.only)
+        known = {spec["name"] for spec in specs}
+        unknown = requested - known
+        if unknown:
+            raise ValueError(f"Unknown experiments: {sorted(unknown)}")
+        specs = [spec for spec in specs if spec["name"] in requested]
+    rerun = set(args.rerun)
+    unknown_reruns = rerun - {spec["name"] for spec in specs}
+    if unknown_reruns:
+        raise ValueError(f"Unknown rerun experiments: {sorted(unknown_reruns)}")
+
+    output = root / args.output
+    previous = {}
+    previous_metadata = {}
+    if output.exists():
+        with output.open("r", encoding="utf-8") as handle:
+            payload = json.load(handle)
+            previous = {
+                item["name"]: item for item in payload["experiments"]
+            }
+            previous_metadata = payload.get("metadata", {})
+    completed = dict(previous)
+    metadata = {
+        "dataset": config["dataset"],
+        "seed": config["seed"],
+        "train_images": sum(len(x["labels"]) for x in normalized_cache["train"]),
+        "test_images": sum(len(x["labels"]) for x in normalized_cache["test"]),
+        "normalized_feature_seconds": normalized_cache["feature_seconds"],
+        "no_normalization_feature_seconds": (
+            no_norm_cache["feature_seconds"]
+            if no_norm_cache
+            else previous_metadata.get("no_normalization_feature_seconds")
+        ),
+    }
+    for spec in specs:
+        if spec["name"] in previous and spec["name"] not in rerun:
+            print(f"Skipping completed experiment: {spec['name']}")
+            continue
+        cache = (
+            normalized_cache
+            if spec.get("normalization", True)
+            else no_norm_cache
+        )
+        print(f"Running experiment: {spec['name']}")
+        completed[spec["name"]] = run_experiment(config, spec, cache, device)
+        save_results(output, metadata, list(completed.values()))
+
+    complete_results = list(completed.values())
+    if set(completed) == EXPECTED_EXPERIMENTS:
+        validate_complete_results(complete_results)
+    save_results(output, metadata, complete_results)
+    print(f"Saved {len(completed)} experiments to {output}")
+
+
+if __name__ == "__main__":
+    main()

reproduce/ider/README.md

@@ -0,0 +1,64 @@
+# IDER: Idempotent Experience Replay for Reliable Continual Learning [(ICLR 2026)](https://arxiv.org/abs/2603.00624)
+
+## Abstract
+
+Catastrophic forgetting is a central challenge in continual learning: when a neural network learns new tasks, it often loses performance on previously learned tasks. The IDER paper further points out that continual learning models should not only maintain accuracy, but also produce reliable confidence estimates, especially in mission-critical scenarios.
+
+IDER proposes **Idempotent Experience Replay**, a replay-based continual learning method built on the idempotent property. The method encourages repeated prediction with the model's own output signal to remain stable, and introduces an idempotence distillation objective between the current model and the old checkpoint. As a result, IDER can be integrated with mainstream replay methods and improves prediction reliability while also improving accuracy and reducing forgetting.
+
+In this LibContinual reproduction, ordinary Experience Replay is reported as `baseline`, and IDER corresponds to the paper's `ER+ID`.
+
+![IDER](../../resources/imgs/ider_method.jpg)
+
+## Citation
+
+```bibtex
+@article{liu2026ider,
+  title={IDER: IDempotent Experience Replay for Reliable Continual Learning},
+  author={Liu, Zhanwang and Li, Yuting and Gao, Haoyuan and Li, Yexin and Kong, Linghe and Sun, Lichao and Huang, Weiran},
+  journal={arXiv preprint arXiv:2603.00624},
+  year={2026}
+}
+```
+
+## How to Reproduce
+
+The IDER and ER baseline configs are in [config/zz_IDER](../../config/zz_IDER/). Run an experiment by passing the corresponding config name to `run_trainer.py`:
+
+```shell
+python run_trainer.py --config ider_cifar100_buf500 --device 0 --seed 0
+```
+
+Choose another IDER or ER config from `config/zz_IDER/` as needed. To reproduce multi-seed results, repeat the command with seeds `0`, `1`, `2`, `3`, and `4`.
+
+## Results
+
+The following table compares our LibContinual reproduction with the original paper. `FAA/CIL` is the final class-incremental average accuracy, and `FF` is final forgetting. Higher FAA/CIL is better, lower FF is better.
+
+| Dataset | Buffer | Method | Paper FAA/CIL | Ours FAA/CIL | Paper FF | Ours FF |
+|---|---:|---|---:|---:|---:|---:|
+| CIFAR-10 | 200 | ER / baseline | 44.46 +/- 2.87 | 48.55 +/- 0.78 | 71.35 +/- 7.77 | 59.84 +/- 1.11 |
+| CIFAR-10 | 200 | ER+ID / IDER | 71.02 +/- 1.98 | 70.57 +/- 0.60 | 15.28 +/- 2.41 | 17.06 +/- 1.73 |
+| CIFAR-10 | 500 | ER / baseline | 58.84 +/- 3.85 | 61.82 +/- 1.60 | 52.12 +/- 7.56 | 42.84 +/- 2.02 |
+| CIFAR-10 | 500 | ER+ID / IDER | 74.74 +/- 0.42 | 75.14 +/- 1.06 | 11.93 +/- 0.49 | 12.28 +/- 0.90 |
+| CIFAR-100 | 500 | ER / baseline | 23.41 +/- 1.15 | 20.37 +/- 0.58 | 71.92 +/- 0.74 | 74.23 +/- 0.66 |
+| CIFAR-100 | 500 | ER+ID / IDER | 44.82 +/- 0.85 | 44.09 +/- 0.80 | 29.98 +/- 2.52 | 34.86 +/- 1.34 |
+| CIFAR-100 | 2000 | ER / baseline | 40.47 +/- 0.95 | 36.53 +/- 0.37 | 51.82 +/- 0.75 | 55.33 +/- 0.47 |
+| CIFAR-100 | 2000 | ER+ID / IDER | 56.59 +/- 0.35 | 55.99 +/- 0.39 | 17.46 +/- 1.04 | 20.03 +/- 0.54 |
+| TinyImageNet | 500 | ER / baseline | 10.13 +/- 0.39 | 9.252 +/- 0.130 | 74.79 +/- 0.67 | 71.567 +/- 0.140 |
+| TinyImageNet | 500 | ER+ID / IDER | 29.88 +/- 1.15 | 30.734 +/- 1.756 | 36.63 +/- 3.37 | 36.682 +/- 4.177 |
+
+For CIFAR experiments, the paper also reports Expected Calibration Error (ECE). Lower ECE is better.
+
+| Dataset | Buffer | Method | Paper ECE | Ours ECE |
+|---|---:|---|---:|---:|
+| CIFAR-10 | 200 | ER / baseline | 45.53 | 45.57 +/- 0.78 |
+| CIFAR-10 | 200 | ER+ID / IDER | 12.36 | 12.96 +/- 0.82 |
+| CIFAR-10 | 500 | ER / baseline | 32.69 | 32.61 +/- 1.57 |
+| CIFAR-10 | 500 | ER+ID / IDER | 11.73 | 12.33 +/- 0.84 |
+| CIFAR-100 | 500 | ER / baseline | 64.59 | 65.53 +/- 0.58 |
+| CIFAR-100 | 500 | ER+ID / IDER | 13.65 | 10.94 +/- 1.03 |
+| CIFAR-100 | 2000 | ER / baseline | 45.64 | 48.58 +/- 0.30 |
+| CIFAR-100 | 2000 | ER+ID / IDER | 12.87 | 10.61 +/- 0.63 |
+
+Overall, the reproduced IDER results are close to the original paper across CIFAR-10, CIFAR-100, and TinyImageNet. The reproduced experiments preserve the main conclusions of the paper: IDER improves final class-incremental accuracy, reduces forgetting, and substantially improves calibration on CIFAR settings.