Update task checkpoint and evaluation summary

README.md

@@ -14,17 +14,25 @@ pipeline_tag: text-generation
 
 # Multi-3DLLM Checkpoints
 
-This repository hosts the released BeyondSingleObject checkpoints:
+This repository hosts the BeyondSingleObject checkpoints:
 
 - `multi-3dllm/`: MO3D, Shape Mating, and Change Captioning
 - `multi-3dllm-classification/`: ModelNet40 zero-shot classification
 
-Use the code and scripts from:
+The task checkpoint uses the Patch-Interaction Transformer implementation and
+was jointly fine-tuned on the three released task datasets. Code and training
+scripts are available at:
 
 ```text
 https://github.com/KohsukeIde/BeyondSingleObject
 ```
 
+Dataset annotations and point-cloud assets are available at:
+
+```text
+https://huggingface.co/datasets/idekoh/BeyondSingleObject
+```
+
 ## Download
 
 ```bash
@@ -37,18 +45,37 @@ Expected local layout:
 
 ```text
 checkpoints/
-├── multi-3dllm/
-└── multi-3dllm-classification/
+|-- multi-3dllm/
+`-- multi-3dllm-classification/
 data/
 ```
 
+## Evaluation Snapshot
+
+The following deterministic results use the compact evaluation subsets and the
+public inference format. They are regression checks rather than replacements
+for the paper's LLM-judged reasoning metrics.
+
+| Task | Metric | Result |
+| --- | --- | ---: |
+| MO3D holistic QA | Binary accuracy | 84.0% (42/50) |
+| Shape Mating | Selection accuracy | 73.0% (146/200) |
+| Change Captioning verify | Binary accuracy | 67.0% (67/100) |
+| Change Captioning verify, no point cloud | Binary accuracy | 50.0% (50/100) |
+
+The separately released classification checkpoint follows the PointLLM
+ModelNet40 evaluation protocol. See the repository README for its six
+one-, two-, and three-object commands.
+
 ## Usage
 
-Example inference and LLM-based evaluation:
+Example inference:
 
 ```bash
 MODEL_PATH=checkpoints/multi-3dllm \
-OUTPUT_DIR=outputs/infer \
+ANNO_PATH=data/mo3d/test.json \
+DATA_PATH=data/point_clouds \
+OUTPUT_DIR=outputs/mo3d_eval \
 scripts/eval/infer.sh
 ```
 
@@ -64,15 +91,12 @@ TARGET_POSITION=1 \
 scripts/eval/eval_modelnet.sh
 ```
 
-Repeat `(NUM_OBJECTS, TARGET_POSITION) = (1,1), (2,1), (2,2), (3,1), (3,2),
-(3,3)` for the full table.
-
 ## Notes
 
-The LLM-judged metrics for reasoning and delta-caption quality depend on the
-judge model and prompt configuration. Use the released evaluation scripts for
-reproducible comparisons, and report the exact judge configuration together
-with the checkpoint.
+LLM-judged reasoning and delta-caption scores depend on the judge model and
+prompt configuration. Use the released evaluators and report the exact judge
+configuration when comparing those metrics. The task and classification
+checkpoints have different roles and should not be interchanged.
 
 ## License
 

multi-3dllm/config.json

@@ -2,7 +2,7 @@
   "DEFAULT_POINT_END_TOKEN": "<point_end>",
   "DEFAULT_POINT_PATCH_TOKEN": "<point_patch>",
   "DEFAULT_POINT_START_TOKEN": "<point_start>",
-  "_name_or_path": "/groups/gag51404/ide/BeyondSIngleObject/outputs/finetune/joint_release_uniform3_32gpu_20260526_2142_uniform3",
+  "_name_or_path": "/groups/gag51404/ide/PointLLM/outputs/PointLLM_train_stage1/PointLLM_train_stage1_original",
   "architectures": [
     "PointLLMCVPRLlamaForCausalLM"
   ],

multi-3dllm/point_proj.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6be00bbe048381a1311358b314ef1ae399eade6cfedffee521f2b961ef01e797
+oid sha256:6d26f93ca9978468f6a6a7f188497a09fe1cd2a86f60c527186d529fada4950f
 size 567885048

multi-3dllm/pytorch_model-00001-of-00003.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c516359ab4995a23404cba2a886c8d28215e0a9e0d75d671a8f97a126adb69cf
+oid sha256:d64e5adb20cea7673e9f929aacdc37243a32cd72cefb08939e19f39efb9a6172
 size 9878039688

multi-3dllm/pytorch_model-00002-of-00003.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:33bdb8bb1ab1092add7bf6a6c26ef94bfbb59a5882bfac977a16e70483d16922
+oid sha256:b4ee892feff102a765e9d723cf92beaeb20705f2ae1994dadae55838787afafb
 size 9894801940

multi-3dllm/pytorch_model-00003-of-00003.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dd82457d8e8344f58ff2c9acf07dd8cfddcab0c92972316c05b7a33f6f61a859
+oid sha256:31a6cc7fbbdbbbcd79a8170cc3100b86d5e1daa90acbfe749711b282dd9331e4
 size 8117611116