| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - scene-graph-generation |
| - object-detection |
| - visual-relationship-detection |
| - pytorch |
| - yolo |
| pipeline_tag: object-detection |
| library_name: sgg-benchmark |
| model-index: |
| - name: REACT++ yolo12l |
| results: |
| - task: |
| type: object-detection |
| name: Scene Graph Detection |
| dataset: |
| name: PSG |
| type: psg |
| metrics: |
| - type: mR@20 |
| value: 23.2 |
| name: mR@20 |
| - type: R@20 |
| value: 30.99 |
| name: R@20 |
| - type: F1@20 |
| value: 26.53 |
| name: F1@20 |
| - type: mR@50 |
| value: 25.49 |
| name: mR@50 |
| - type: R@50 |
| value: 35.3 |
| name: R@50 |
| - type: F1@50 |
| value: 29.6 |
| name: F1@50 |
| - type: mR@100 |
| value: 26.45 |
| name: mR@100 |
| - type: R@100 |
| value: 36.68 |
| name: R@100 |
| - type: F1@100 |
| value: 30.74 |
| name: F1@100 |
| - type: e2e_latency_ms |
| value: 19.6 |
| name: e2e_latency_ms |
| - name: REACT++ yolo12m |
| results: |
| - task: |
| type: object-detection |
| name: Scene Graph Detection |
| dataset: |
| name: PSG |
| type: psg |
| metrics: |
| - type: mR@20 |
| value: 22.74 |
| name: mR@20 |
| - type: R@20 |
| value: 32.69 |
| name: R@20 |
| - type: F1@20 |
| value: 26.82 |
| name: F1@20 |
| - type: mR@50 |
| value: 25.21 |
| name: mR@50 |
| - type: R@50 |
| value: 37.2 |
| name: R@50 |
| - type: F1@50 |
| value: 30.05 |
| name: F1@50 |
| - type: mR@100 |
| value: 26.08 |
| name: mR@100 |
| - type: R@100 |
| value: 38.58 |
| name: R@100 |
| - type: F1@100 |
| value: 31.12 |
| name: F1@100 |
| - type: e2e_latency_ms |
| value: 15.7 |
| name: e2e_latency_ms |
| - name: REACT++ yolo12s |
| results: |
| - task: |
| type: object-detection |
| name: Scene Graph Detection |
| dataset: |
| name: PSG |
| type: psg |
| metrics: |
| - type: mR@20 |
| value: 21.12 |
| name: mR@20 |
| - type: R@20 |
| value: 29.28 |
| name: R@20 |
| - type: F1@20 |
| value: 24.54 |
| name: F1@20 |
| - type: mR@50 |
| value: 23.21 |
| name: mR@50 |
| - type: R@50 |
| value: 33.48 |
| name: R@50 |
| - type: F1@50 |
| value: 27.41 |
| name: F1@50 |
| - type: mR@100 |
| value: 23.77 |
| name: mR@100 |
| - type: R@100 |
| value: 34.74 |
| name: R@100 |
| - type: F1@100 |
| value: 28.23 |
| name: F1@100 |
| - type: e2e_latency_ms |
| value: 12.2 |
| name: e2e_latency_ms |
| - name: REACT++ yolo12n |
| results: |
| - task: |
| type: object-detection |
| name: Scene Graph Detection |
| dataset: |
| name: PSG |
| type: psg |
| metrics: |
| - type: mR@20 |
| value: 16.88 |
| name: mR@20 |
| - type: R@20 |
| value: 26.88 |
| name: R@20 |
| - type: F1@20 |
| value: 20.74 |
| name: F1@20 |
| - type: mR@50 |
| value: 18.65 |
| name: mR@50 |
| - type: R@50 |
| value: 30.61 |
| name: R@50 |
| - type: F1@50 |
| value: 23.17 |
| name: F1@50 |
| - type: mR@100 |
| value: 19.5 |
| name: mR@100 |
| - type: R@100 |
| value: 31.8 |
| name: R@100 |
| - type: F1@100 |
| value: 24.17 |
| name: F1@100 |
| - type: e2e_latency_ms |
| value: 11.4 |
| name: e2e_latency_ms |
| - name: REACT++ yolov8m |
| results: |
| - task: |
| type: object-detection |
| name: Scene Graph Detection |
| dataset: |
| name: PSG |
| type: psg |
| metrics: |
| - type: mR@20 |
| value: 22.75 |
| name: mR@20 |
| - type: R@20 |
| value: 30.69 |
| name: R@20 |
| - type: F1@20 |
| value: 26.13 |
| name: F1@20 |
| - type: mR@50 |
| value: 25.46 |
| name: mR@50 |
| - type: R@50 |
| value: 35.68 |
| name: R@50 |
| - type: F1@50 |
| value: 29.72 |
| name: F1@50 |
| - type: mR@100 |
| value: 26.4 |
| name: mR@100 |
| - type: R@100 |
| value: 37.43 |
| name: R@100 |
| - type: F1@100 |
| value: 30.96 |
| name: F1@100 |
| - type: e2e_latency_ms |
| value: 15.3 |
| name: e2e_latency_ms |
| --- |
| |
| # REACT++ Scene Graph Generation — PSG (yolo12l, yolo12m, yolo12s, yolo12n, yolov8m) |
|
|
| This repository contains **REACT++** model checkpoints for scene graph generation (SGG) |
| on the **PSG** benchmark, across 5 backbone sizes. |
|
|
| REACT++ is a parameter-efficient, attention-augmented relation predictor built on top of |
| a YOLO12 backbone. It uses: |
|
|
| - **DAMP** (Detection-Anchored Multi-Scale Pooling), a new simple pooling algorithm for one-stage object detectors such as YOLO |
| - **SwiGLU gated MLP** for all feed-forward blocks (½ the params of ReLU-MLP at equal capacity) |
| - **Visual x Semantic cross-attention** — visual tokens attend to GloVe prototype embeddings |
| - **Geometry RoPE** — box-position encoded as a rotary frequency bias on the Q matrix |
| - **Prototype Momentum Buffer** — per-class EMA prototype bank |
| - **P5 Scene Context** — AIFI-enhanced P5 tokens provide global context via cross-attention |
|
|
| The models were trained with the |
| [SGG-Benchmark](https://github.com/Maelic/SGG-Benchmark) framework and described in the |
| [REACT++ paper (Neau et al., 2026)](https://arxiv.org/abs/2603.06386). |
|
|
| --- |
|
|
| ## Results — SGDet on PSG test split (ONNX, CUDA) |
|
|
| > Metrics from end-to-end ONNX evaluation (`tools/eval_onnx_psg.py`). E2E Latency = image load + pre-process + ONNX forward. |
|
|
| | Backbone | Params | R@20 | R@50 | R@100 | mR@20 | mR@50 | mR@100 | F1@20 | F1@50 | F1@100 | E2E Lat. (ms) | |
| |----------|:------:|-----:|-----:|------:|------:|------:|-------:|------:|------:|-------:|--------------:| |
| | yolo12l | ~26.5M | 30.99 | 35.3 | 36.68 | 23.2 | 25.49 | 26.45 | 26.53 | 29.6 | 30.74 | 19.6 | |
| | yolo12m | ~20.2M | 32.69 | 37.2 | 38.58 | 22.74 | 25.21 | 26.08 | 26.82 | 30.05 | 31.12 | 15.7 | |
| | yolo12s | ~9.2M | 29.28 | 33.48 | 34.74 | 21.12 | 23.21 | 23.77 | 24.54 | 27.41 | 28.23 | 12.2 | |
| | yolo12n | ~2.6M | 26.88 | 30.61 | 31.8 | 16.88 | 18.65 | 19.5 | 20.74 | 23.17 | 24.17 | 11.4 | |
| | yolov8m | ~25.9M | 30.69 | 35.68 | 37.43 | 22.75 | 25.46 | 26.4 | 26.13 | 29.72 | 30.96 | 15.3 | |
|
|
| --- |
|
|
| ## Checkpoints |
|
|
| | Variant | Sub-folder | Checkpoint files | |
| |---------|------------|-----------------| |
| | yolo12l | `yolo12l/` | `yolo12l/model.onnx` (ONNX) · `yolo12l/best_model_epoch_9.pth` (PyTorch) | |
| | yolo12m | `yolo12m/` | `yolo12m/model.onnx` (ONNX) · `yolo12m/best_model_epoch_9.pth` (PyTorch) | |
| | yolo12s | `yolo12s/` | `yolo12s/model.onnx` (ONNX) · `yolo12s/best_model_epoch_6.pth` (PyTorch) | |
| | yolo12n | `yolo12n/` | `yolo12n/model.onnx` (ONNX) · `yolo12n/best_model_epoch_5.pth` (PyTorch) | |
| | yolov8m | `yolov8m/` | `yolov8m/model.onnx` (ONNX) · `yolov8m/best_model_epoch_6.pth` (PyTorch) | |
|
|
| --- |
|
|
| ## Usage |
|
|
| ### ONNX (recommended — no Python dependencies beyond onnxruntime) |
|
|
| ```python |
| from huggingface_hub import hf_hub_download |
| |
| onnx_path = hf_hub_download( |
| repo_id="maelic/REACTPlusPlus_PSG", |
| filename="yolo12l/react_pp_yolo12m.onnx", |
| repo_type="model", |
| ) |
| # Run with tools/eval_onnx_psg.py or load directly via onnxruntime |
| ``` |
|
|
| ### PyTorch |
|
|
| ```python |
| # 1. Clone the repository |
| # git clone https://github.com/Maelic/SGG-Benchmark |
| |
| # 2. Install dependencies |
| # pip install -e . |
| |
| # 3. Download checkpoint + config |
| from huggingface_hub import hf_hub_download |
| |
| ckpt_path = hf_hub_download( |
| repo_id="maelic/REACTPlusPlus_PSG", |
| filename="yolo12l/best_model.pth", |
| repo_type="model", |
| ) |
| cfg_path = hf_hub_download( |
| repo_id="maelic/REACTPlusPlus_PSG", |
| filename="yolo12l/config.yml", |
| repo_type="model", |
| ) |
| |
| # 4. Run evaluation |
| import subprocess |
| subprocess.run([ |
| "python", "tools/relation_eval_hydra.py", |
| "--config-path", str(cfg_path), |
| "--task", "sgdet", |
| "--eval-only", |
| "--checkpoint", str(ckpt_path), |
| ]) |
| ``` |
|
|
| --- |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{neau2026reactpp, |
| title = {REACT++: Efficient Cross-Attention for Real-Time Scene Graph Generation |
| }, |
| author = {Neau, Maëlic and Falomir, Zoe}, |
| year = {2026}, |
| url = {https://arxiv.org/abs/2603.06386}, |
| } |
| ``` |
|
|
