Graspmax — GeoMatch v3 · GeoMatch v2 · GeoMatch++ · GeoMatch v1 · RobotFingerPrint

Graspmax contains geometry-aware contact/coordinate prediction models for dexterous robotic grasping, trained on the CMapDataset / GenDexGrasp dataset across 5 robot end-effectors (EZGripper, Barrett, Robotiq 3-Finger, Allegro, ShadowHand).

⚠️ Version notice: v1, GeoMatch++, and v2 were all trained on end-effector keypoint data that was wrong in some way — see Keypoint bug history. GeoMatch v3 is the first run trained against the real, hand-verified robot_keypoints.json (the paper describes these as chosen visually on each end-effector's mesh surface, but never released them — see google-deepmind/geomatch#1, open since Feb 2024). It is also the only checkpoint in this repository that has been run through the actual Isaac Gym grasp-stability test the paper uses — every number below it in this table is val-loss/val-accuracy only, not simulated grasp success. Use GeoMatch v3 for any new work.

---

Models at a Glance

Model	Status	Folder	Keypoints	Isaac Gym success rate measured?
GeoMatch v3	✅ Recommended (default)	geomatch_v3/	✅ real, surface-verified	✅ yes — 3-seed results below
GeoMatch v2	⚠️ Superseded	geomatch_v2/	⚠️ unverified against the paper's real keypoints	❌ not measured
GeoMatch++	⚠️ Deprecated (built on v1 encoders)	geomatch_pp/	❌ corrupted (inherits v1 bugs)	❌ not measured
GeoMatch v1	⚠️ Deprecated	geomatch_v1/	❌ corrupted (2× scale + axis-swap bugs)	❌ not measured here; a prior corrupted-keypoint run scored 0%
RobotFingerPrint	✅ Paper reproduction	robotfingerprint/	n/a — different method (continuous UV coords)	—

Read the numbers below honestly, not as a marketing claim: GeoMatch v3's measured Isaac Gym success rates (30–68% depending on gripper/checkpoint/seed) are well under the original paper's published results (75–90%). v3 is "recommended" because it is the only checkpoint trained on keypoints that are actually correct and the only one with real simulated grasp-success evidence attached — not because it matches the paper. If you need paper-matching performance, none of the checkpoints in this repository currently deliver it; v3 is the closest verified starting point for further work.

---

GeoMatch v3 — Training and Evaluation

Training configuration

Faithful to arXiv:2312.03864 Appendix C, with batch size taken from the upstream DeepMind train.py default (the paper does not state one):

Setting	Value
Dataset	GenDexGrasp MultiDex subset — 41,871 train + 8,931 validate = 50,802 grasps (matches the paper's total); split 48 train / 10 validate objects
End-effectors	EZGripper, Barrett, Robotiq 3-Finger, Allegro, ShadowHand
Batch size	64
Optimizer	Adam (β=(0.9, 0.99)), weight_decay=0
Learning rate	1e-4, constant
Epochs	200 (0–199)
Precision	FP32 (no AMP)
λ robot / matchnet	500 / 200
α / β loss weights	0.5 / 0.5
N keypoints / KNN	6 / 8
Object / EE point cloud	2048 / 1000 (ezgripper 768)
Seed	42
Hardware	1× AMD MI300X (ROCm 6.2.4), torch 2.5.1
Training time	37.97 h, zero errors, all 400 progress bars (train + val, per epoch) ran their full step count

Two checkpoints are published — neither training-loss curve alone resolves which is better (train accuracy improves monotonically 0.689→0.728, but validation loss and accuracy disagree on direction), so both were evaluated:

File	Epoch	Train loss / acc	Val loss / acc	SHA-256
geomatch_v3/checkpoint_epoch100.pth	100 (halfway)	1.6752 / 0.689	1.7518 / 0.642	c6bd5e0e...c2fbd
geomatch_v3/checkpoint_final.pth	199 (end of schedule)	1.6164 / 0.728	1.7846 / 0.692	8f72bde4...66a915

Both are raw state_dict (collections.OrderedDict, 72 tensors, 1,890,309 params), load cleanly into models.geomatch.GeoMatch, and are verified free of NaN/Inf.

robot_keypoints.json (md5 3bbd2df2a990a505f31883ac64f9cb60) and robot_centroids.json (md5 d3305eafbb691c1f87890dd98e2db506) in this folder are the exact files used to train both checkpoints and must be used unmodified at evaluation time — IK solves for these exact keypoint positions, and a mismatch silently produces wrong grasps with no error message. This is the failure mode that invalidated geomatch_v1 and earlier geomatch_v2_repro attempts.

Contact-precision sanity check

Cheap pre-check (evaluate.py, not part of the upstream google-deepmind/geomatch release — a standalone eval script written for this repo) on the 10-object validation split, computing the same point-level contact-map accuracy/precision/recall train.py reports every epoch:

gripper	epoch 100 acc / prec / recall	final (199) acc / prec / recall
allegro	0.588 / 0.009 / 0.503	0.666 / 0.009 / 0.430
barrett	0.689 / 0.011 / 0.464	0.696 / 0.012 / 0.507
ezgripper	0.576 / 0.019 / 0.844	0.656 / 0.023 / 0.864
robotiq_3finger	0.711 / 0.012 / 0.632	0.709 / 0.012 / 0.635
shadowhand	0.533 / 0.011 / 0.691	0.571 / 0.012 / 0.712
overall	0.621 / 0.012 / 0.640	0.658 / 0.014 / 0.648

Precision looks numerically tiny in absolute terms — this is an artifact of severe class imbalance in the point-level contact metric (only a small fraction of the 2048 object points are true positives per keypoint), not a defect. Compare relative to corrupted-keypoint runs, where precision sat near ~0.02 or below; both v3 checkpoints clear that bar comfortably, and overall accuracy tracks each checkpoint's own reported validation accuracy closely — a good sanity signal that evaluation is using matched keypoints/data, not a broken pipeline.

Isaac Gym grasp-stability evaluation (3 seeds)

Full physics evaluation, run identically to the original paper's protocol (arXiv:2312.03864 §4, confirmed against the paper text): 10 held-out objects × 4 grasps per object-gripper (top-[0, 20, 50, 100] predicted keypoint configurations, matching the paper's sampling), tested with the authors' own Isaac Gym harness (GenDexGrasp/run_grasp_test.py) — 0.5 m/s² acceleration applied sequentially along ±x, ±y, ±z (6 directions, 50 sim-steps each), object displaced by a pre-grasp offset of 5 mm along the surface normal, then SciPy TRF inverse kinematics; a grasp counts as successful only if displacement stays under 2 cm in all six directions. Only EZGripper, Barrett, and ShadowHand have an Isaac Gym task implementation (envs/tasks/grasp_test_force_*.py) — the same three grippers the paper evaluates; Robotiq-3F and Allegro are training-only and are not part of the physics test in either the paper or here.

Grasp generation (which keypoints/IK solution get tested) is fully deterministic — empirically verified bit-identical across --seed 0/1/2 (no dropout is actually applied in the model's forward() despite being configured, no sampling anywhere in inference, IK uses a deterministic finite-difference optimizer). The only source of real run-to-run variance is the GPU physics rollout itself, so 3 seeds below vary the Isaac Gym simulation seed (42/43/44) on identical input grasps, not the grasps themselves — this isolates simulation noise from genuine checkpoint differences.

GeoMatch v3, epoch 100:

gripper	seed 42	seed 43	seed 44	mean	range	paper (in-domain)
Barrett	55.0% (22/40)	60.0% (24/40)	60.0% (24/40)	58.3%	5.0pp	90.0%
EZGripper	50.0% (20/40)	57.5% (23/40)	47.5% (19/40)	51.7%	10.0pp	75.0%
ShadowHand	60.0% (24/40)	52.5% (21/40)	52.5% (21/40)	55.0%	7.5pp	72.5%
pooled overall				55.0% (66/120)		79.2% (mean)

GeoMatch v3, final (epoch 199):

gripper	seed 42	seed 43	seed 44	mean	range	paper (in-domain)
Barrett	30.0% (12/40)	30.0% (12/40)	30.0% (12/40)	30.0%	0.0pp	90.0%
EZGripper	65.0% (26/40)	72.5% (29/40)	65.0% (26/40)	67.5%	7.5pp	75.0%
ShadowHand	57.5% (23/40)	65.0% (26/40)	52.5% (21/40)	58.3%	12.5pp	72.5%
pooled overall				51.9% (62/120)		79.2% (mean)

Takeaways:

• Neither checkpoint is close to the paper's numbers. Both sit 20–60 percentage points below target across the board.
• More training did not uniformly help, and this holds up under seed variance, not just single-seed noise. EZGripper improves from epoch 100 → final on every one of 3 seeds (+16pp mean). Barrett gets worse — and does so with zero variance across all 3 seeds (30.0% exactly, every time), which rules out simulation noise as the explanation; the final checkpoint is genuinely, deterministically worse at Barrett grasps. ShadowHand is a wash (55.0% vs 58.3% mean, within each other's seed-to-seed spread) — the earlier single-seed gap there was noise.
• Simulation seed-to-seed variance is real but modest (5–12.5pp swings on a 40-trial sample, consistent with expected binomial noise at this sample size) — far too small to explain the Barrett regression or EZGripper improvement.
• Both checkpoints, both bridging scripts (bridge_to_gendexgrasp.py), the physics harness itself, and all keypoint/checksum matching were independently verified during this evaluation — see geomatch_v3/INFO.md for the full audit trail (checkpoint SHA-256s, keypoint md5s, generated-vs-reused pose diffs, file timelines).

---

Architecture

GeoMatch (v1, v2, v3 share the same architecture)

Dual GCN encoder (object + robot surface) → L2-normalised embeddings → linear projection heads (512→64) × 2 → 5 autoregressive MLP modules → per-keypoint BCE contact map prediction.

Based on: Geometry Matching for Multi-Embodiment Grasping (Attarian, Asif, Liu, Hari, Garg, Gilitschenski, Tompson — CoRL 2023)

GeoMatch++

Extends GeoMatch with a morphology encoder (GCN over the robot kinematic-tree graph, 9D node features, 32 nodes) and a DCP-style cross-attention transformer that fuses object geometry with robot morphology before contact prediction. Pretrained GeoMatch v1 encoders are frozen.

Based on: GeoMatch++: Morphology-Aware Grasping via Correspondence Learning

RobotFingerPrint

A conditional VAE (GcsCVAE) that predicts a per-point Unified Gripper Coordinate Space (UGCS) 2D coordinate map over an object's point cloud, conditioned on the object geometry. Unlike GeoMatch's discrete per-keypoint contact classification, RobotFingerPrint regresses a continuous (u, v) coordinate for every object point, which is what allows a single trained model to transfer grasps across grippers with a different number of fingers without any manual re-targeting.

• Encoder: PointNet-style per-point Conv1d stack over (object_pc, gt_uv) → max-pool global feature → linear heads to VAE latent mean/logvar (encoder_layers_size=[5, 64, 128, 512, 512], latent_size=128)
• Decoder: per-point features + global object feature + latent code → Conv1d stack (decoder_decoder_layers_size=[64+512+128, 512, 64]) → two parallel U/V prediction heads (uv_layers_size=[64, 32, 1])
• Loss: reconstruction (weighted L2 on predicted vs. ground-truth UV coordinates) + annealed KL divergence (weight increased every ann_per_epochs epochs following a temperature schedule)

Based on: RobotFingerPrint: Unified Gripper Coordinate Space for Multi-Gripper Grasp Synthesis (Khargonkar, Casas, Prabhakaran, Xiang)

Component Comparison (GeoMatch family)

Component	GeoMatch v1 / v2 / v3	GeoMatch++
Object GCN encoder	3 layers × 256 → 512, trainable	Same, frozen (from GeoMatch v1)
Robot surface GCN	3 layers × 256 → 512, trainable	Same, frozen (from GeoMatch v1)
Morphology encoder	—	NEW GCN(9 → 256×3 → 512), trainable
Cross-attention	—	NEW DCP transformer (512-dim, 4 heads, 1 layer)
Projection heads	Linear(512→64) × 2	Same, re-initialised
AR keypoint modules	5× MLP	Same, re-initialised
Total params	~1.9M	6.4M (5.8M trainable)

---

Keypoint bug history

robot_keypoints.json and robot_centroids.json define each end-effector's canonical 3D contact-point positions in rest-pose space. The paper states these were chosen by hand, on the mesh surface:

"The canonical contact points kᵢ ∈ V_G on the surface of each end-effector are selected visually." — arXiv:2312.03864 §3.1

but never released the actual files (open issue since Feb 2024, see link above). Every run before v3 used a stand-in that was wrong in some way:

run	keypoint data	outcome
geomatch_v1	2× scale factor + ShadowHand axis-swap applied at the wrong pipeline stage (before FK instead of in link-local space)	0% Isaac Gym success
geomatch_v2	scale/axis-swap bugs fixed, but not verified against the paper's real (never-released) keypoints	not Isaac Gym validated
geomatch_v2_repro (internal repro runs)	keypoints substituted with link mesh centroids — interior points, not hand-picked surface points	5–10% Isaac Gym success at epoch 50
geomatch_v3	the real robot_keypoints.json, verified: all 30 link names resolve against the GenDexGrasp URDFs, every keypoint lies within ≤9.2 mm of its link's mesh surface (most ≤2 mm), repeated links occupy contiguous keypoint indices	30–68% Isaac Gym success, measured above

Bug 1 — 2× scale factor (v1): The generation script applied world_pos *= 2.0, citing HandModel's hand_scale=2.0 class default. Every actual call site passes hand_scale=1.0, overriding that default. Because the scale was applied before the inverse-FK projection HandModel.get_canonical_keypoints() uses (T⁻¹[2p;1] ≠ 2·T⁻¹[p;1]), the distortion was not uniform — it grew with each link's distance from the kinematic root.

Bug 2 — ShadowHand axis-swap at the wrong stage (v1): The [x, -z, y] axis permutation for ShadowHand was applied to the final world-space world_pos (after FK) instead of to raw mesh points in link-local space before the visual-origin transform (as the reference gripper_utils.py does). Rotation and axis permutation do not commute, so this produced scrambled keypoint positions for any link with a non-zero visual-origin rotation — confirmed by v1 ShadowHand tip keypoints landing at y ≈ −0.84 m (outside any physical hand envelope) versus the corrected ≈ 0.01 m.

Switching from centroid-derived to the real keypoints improved contact-label supervision on every evaluated gripper (average keypoints-in-contact-per-grasp, out of 6): ezgripper 3.74→5.66 (+51%), barrett 4.08→4.47 (+10%), shadowhand 4.17→4.26 (+2%). Full verification detail in geomatch_v3/INFO.md.

---

Training Details

GeoMatch v3 ✅ (Recommended — see full config above)

See GeoMatch v3 — Training and Evaluation above.

GeoMatch v2 ⚠️ (Superseded by v3)

Setting	Value
Dataset	CMapDataset (ContactDB + YCB)
End-effectors	EZGripper, Barrett, Robotiq 3-Finger, Allegro, ShadowHand
Batch size	256
Optimizer	Adam (β₁=0.9, β₂=0.99)
Learning rate	1e-4
Epochs	200
Hardware	AMD Instinct MI300X (192 GB HBM3), ROCm 6.2.4
Training time	8.58 hours
Precision	FP32
Final val loss	1.594
Final val accuracy	0.695

Not run through Isaac Gym, and its keypoints were never checked against the paper's real (unreleased) file — treat val loss/accuracy here as informative but unvalidated by physics.

GeoMatch v2 Training Curves

Epoch	Val Loss	Val Accuracy
0	1.935	0.205
25	1.731	0.563
50	1.675	0.580
100	1.649	0.632
150	1.603	0.656
199	1.594	0.695

---

GeoMatch++ ⚠️ (Deprecated — built on GeoMatch v1 encoders)

Setting	Value
Initialisation	Pretrained GeoMatch v1 encoders (frozen)
Trainable params	~5.8M
Batch size	32 per GPU × 8 GPUs = 256 effective
Optimizer	Adam (β₁=0.9, β₂=0.99)
Learning rate	5e-5
Epochs	150
Hardware	8× AMD Instinct MI300X, ROCm 6.2.4 (DDP)
Training time	~2.8 hours
Precision	FP32
Final val loss	0.350 (artefact of corrupted training data)
Final val accuracy	0.940 (artefact of corrupted training data)

GeoMatch++ Training Curves

Epoch	Val Loss	Val Accuracy
0	0.465	0.999
25	0.370	0.880
89	0.362	0.902
149	0.350	0.940

---

GeoMatch v1 ⚠️ (Deprecated — corrupted keypoints)

Setting	Value
Dataset	CMapDataset (ContactDB + YCB), corrupted keypoints
Batch size	256
Optimizer	Adam (β₁=0.9, β₂=0.99)
Learning rate	1e-4
Epochs	200
Hardware	AMD Instinct MI300X (192 GB HBM3), ROCm 6.2.4
Training time	22.18 hours
Precision	FP32
Final val loss	0.435 (artefact of corrupted training data)
Final val accuracy	0.959 (artefact of corrupted training data)

---

RobotFingerPrint ✅ (Paper reproduction — 4 experiments)

All 4 experiments use the exact recipe published by the paper's authors (this repo's own README): --n_epochs 16 --ann_temp 1.5 --ann_per_epochs 2, plus code defaults lr=1e-4, batch_size=64, lw_recon=1000.0, lw_kld=0.01, attn_alpha=3, Adam(β₁=0.9, β₂=0.999), seed=42.

Setting	Value
Dataset	GenDexGrasp CMapDataset-sqrt_align + RobotFingerPrint UGCS coordinates
Batch size	64
Optimizer	Adam (β₁=0.9, β₂=0.999)
Learning rate	1e-4 (StepLR decay disabled — decay_lr_freq=1000 > total epochs)
Epochs	16
KL annealing	temperature 1.5, weight increased every 2 epochs
Hardware	AMD Instinct MI300X (192 GB HBM3), ROCm 6.2.4, single GPU per run
Precision	FP32

Experiment	Seen grippers	Held-out (unseen)	Training time	Final val recon loss	Final val KLD	Final val overall
fullrobots/	ezgripper, barrett, robotiq_3finger, allegro, shadowhand	none (main paper result)	449 s	0.2495	112.57	268.72
unseen_barrett/	ezgripper, robotiq_3finger, allegro, shadowhand	barrett	366 s	0.2568	105.99	274.90
unseen_ezgripper/	barrett, robotiq_3finger, allegro, shadowhand	ezgripper	383 s	0.2791	109.36	297.77
unseen_shadowhand/	ezgripper, barrett, robotiq_3finger, allegro	shadowhand	366 s	0.2590	111.11	278.02

"Val recon loss" is the reconstruction term of the CVAE loss (weighted L2 between predicted and ground-truth UGCS coordinates) — lower is better. The 3 unseen-gripper runs measure how well the model's learned coordinate space generalizes to a gripper never seen during training (evaluated only on the 4 remaining seen grippers' validation split, same as the paper's ablation setup — a full cross-gripper zero-shot transfer evaluation requires the downstream grasp generation + IsaacGym stability test pipeline, not covered by this checkpoint alone).

---

Checkpoints

GeoMatch v3 ✅ (Use these)

File	Epoch	Val Loss / Acc	Isaac Gym pooled success	Notes
geomatch_v3/checkpoint_epoch100.pth	100	1.752 / 0.642	55.0% (66/120, 3-seed)	Mid-training
geomatch_v3/checkpoint_final.pth	199	1.785 / 0.692	51.9% (62/120, 3-seed)	End of 200-epoch schedule — not uniformly better; see per-gripper breakdown above
geomatch_v3/robot_keypoints.json	—	—	—	Required — must match training exactly (md5 3bbd2df2...)
geomatch_v3/robot_centroids.json	—	—	—	Loaded by HandModel, not used in the core math (md5 d3305eaf...)

Pick per gripper if you care about a specific end-effector (epoch 100 for Barrett, final for EZGripper — see the full table above); there is no single checkpoint that's best everywhere.

GeoMatch v2 ⚠️ (Superseded)

File	Epoch	Val Loss	Notes
geomatch_v2/checkpoint_epoch50.pth	50	1.675	Early convergence
geomatch_v2/checkpoint_epoch100.pth	100	1.649	Mid-training
geomatch_v2/checkpoint_epoch150.pth	150	1.603	Near-converged
geomatch_v2/final.pth	199	1.594	Final — not Isaac Gym validated

GeoMatch++ ⚠️ (Deprecated)

File	Epoch	Notes
geomatch_pp/checkpoint_epoch50.pth	50	Early convergence
geomatch_pp/checkpoint_epoch100.pth	100	Mid-training
geomatch_pp/checkpoint_epoch140.pth	140	Near-converged
geomatch_pp/final.pth	149	Final (deprecated)

GeoMatch v1 ⚠️ (Deprecated)

File	Epoch	Notes
geomatch_v1/checkpoint_epoch50.pth	50	Early convergence
geomatch_v1/checkpoint_epoch100.pth	100	Mid-training
geomatch_v1/checkpoint_epoch150.pth	150	Near-converged
geomatch_v1/final.pth	200	Final (deprecated)

RobotFingerPrint ✅ (final checkpoint only per experiment)

File	Experiment	Notes
robotfingerprint/fullrobots/final.ckpt	All 5 grippers seen	Main paper result — recommended
robotfingerprint/unseen_barrett/final.ckpt	Barrett held out	Generalization ablation
robotfingerprint/unseen_ezgripper/final.ckpt	EZGripper held out	Generalization ablation
robotfingerprint/unseen_shadowhand/final.ckpt	ShadowHand held out	Generalization ablation

These are full PyTorch Lightning checkpoints (model weights + hyperparameters, no optimizer/epoch intermediates — only the last training epoch of each run is kept). Only the final epoch is published; intermediate per-epoch checkpoints are not included here.

---

Usage

GeoMatch v3 (Recommended)

import torch, sys
sys.path.append(".")            # repo root containing config.py and models/
import config
from models.geomatch import GeoMatch

model = GeoMatch(config).cuda()
model.load_state_dict(torch.load("geomatch_v3/checkpoint_final.pth", map_location="cuda"))
model.eval()

with torch.no_grad():
    contact_map, keypoint_probs = model(
        obj_pc,               # [B, 2048, 3]   object point cloud
        robot_pc,             # [B, 6, 3]      robot surface points (6 keypoints)
        robot_key_point_idx,  # [B, 6]         keypoint indices into robot_pc
        obj_adj,              # [B, 2048, 2048] object adjacency (sparse COO)
        robot_adj,            # [B, 6, 6]      robot adjacency
        xyz_prev,             # [B, 6, 3]      previous keypoint positions
    )
# contact_map:    [B, 2048, 6, 1]  — per-object-point × per-keypoint contact probability
# keypoint_probs: [B, 2048, 5, 1]  — autoregressive keypoint contact probabilities

geomatch_v3/robot_keypoints.json and robot_centroids.json must be placed in whatever data_dir your HandModel/dataset loader points at — do not substitute the keypoint files from any other folder in this repository; see Checkpoints above.

Architecture is identical to geomatch_v1 / geomatch_v2 — same models/geomatch.py.

GeoMatch v2 (Superseded)

model = GeoMatch(config).cuda()
model.load_state_dict(torch.load("geomatch_v2/final.pth", map_location="cuda"))
model.eval()

Same forward signature as v3, above.

GeoMatch++ (Deprecated — kept for reproducibility)

import torch, sys
sys.path.append(".")
import config
from models.geomatch_pp import GeoMatchPP

model = GeoMatchPP(config).cuda()
model.load_state_dict(torch.load("geomatch_pp/final.pth", map_location="cuda"))
model.eval()

with torch.no_grad():
    contact_map, keypoint_probs = model(
        obj_pc,               # [B, 2048, 3]
        robot_pc,             # [B, 6, 3]
        robot_key_point_idx,  # [B, 6]
        obj_adj,              # [B, 2048, 2048]
        robot_adj,            # [B, 6, 6]
        xyz_prev,             # [B, 6, 3]
        morph_features,       # [B, 32, 9]     morphology node features
        morph_adj,            # [B, 32, 32]    morphology adjacency
    )

Morphology graphs are pre-built per robot using preprocess_morphology.py → gnn_morphology_new.pt.

RobotFingerPrint

Lightning checkpoints store hyperparameters alongside weights, so the model reconstructs itself directly from the .ckpt file — no separate config needed:

import sys
sys.path.append(".")
from robotfingerprint.model.grasp_network import GcsGraspModel

model = GcsGraspModel.load_from_checkpoint("robotfingerprint/fullrobots/final.ckpt")
model.eval().cuda()

with torch.no_grad():
    # input_pc: [B, N, 3] object point cloud; gt_gcs only used to establish shape during
    # training — at inference time use model.model.predict(input_pc) for sampling-based prediction
    pred_uv = model.model.predict(input_pc.cuda())
# pred_uv: [B, N, 2] predicted Unified Gripper Coordinate Space (u, v) per object point

For the full downstream pipeline (coordinate inference on held-out objects → grasp generation → IsaacGym stability testing), see the RobotFingerPrint repository gcs_gdx_inf_cvae.py and gcs_gdx_grasp_gen.py scripts, using these checkpoints via --logdir/--ckpt.

---

Reproducing the Isaac Gym evaluation

Full step-by-step instructions (environment setup for both the py3.9 grasp-generation stage and the py3.8 Isaac Gym stage, dataset/keypoint staging, known pitfalls, and exact commands) are in geomatch_v3/README.md and geomatch_v3/INFO.md. Three traps worth knowing before you start: generate_grasps_for_obj.py hardcodes the checkpoint filename to weights/grasp_gnn.pth (stage whichever checkpoint you mean to evaluate under that exact name); the keypoint JSONs must be byte-identical between training and evaluation; and Isaac Gym Preview 4 requires its own Python 3.8 environment, separate from grasp generation, and only runs on NVIDIA GPUs whose architecture its bundled PhysX build actually supports (very new GPU architectures released after Isaac Gym Preview 4 may not work — run on an older/more established card if you hit no kernel image is available for execution on the device).

---

Repository Structure

geomatch_v3/          # GeoMatch v3 checkpoints (recommended — real keypoints, Isaac Gym validated)
  checkpoint_epoch100.pth
  checkpoint_final.pth
  robot_keypoints.json
  robot_centroids.json
  README.md            # Full reproduction guide (env setup, staging, commands)
  INFO.md              # Training record, keypoint verification, contact-label analysis
geomatch_v1/          # GeoMatch v1 checkpoints (deprecated, corrupted keypoints)
  checkpoint_epoch50.pth
  checkpoint_epoch100.pth
  checkpoint_epoch150.pth
  final.pth
geomatch_pp/          # GeoMatch++ checkpoints (deprecated, built on v1 encoders)
  checkpoint_epoch50.pth
  checkpoint_epoch100.pth
  checkpoint_epoch140.pth
  final.pth
geomatch_v2/          # GeoMatch v2 checkpoints (superseded by v3)
  checkpoint_epoch50.pth
  checkpoint_epoch100.pth
  checkpoint_epoch150.pth
  final.pth
robotfingerprint/     # RobotFingerPrint (arXiv:2409.14519) paper reproduction
  fullrobots/final.ckpt
  unseen_barrett/final.ckpt
  unseen_ezgripper/final.ckpt
  unseen_shadowhand/final.ckpt
  model/
    grasp_network.py  # GcsGraspModel (Lightning module)
    modules.py         # GcsCVAE, PointNetCmapEncoder/Decoder
    loss.py            # GcsLoss (recon + annealed KLD)
models/                # Shared GeoMatch source (v1/v2/v3/++)
  geomatch.py
  geomatch_pp.py
  gnn.py
  mlp.py
config.py              # Hyperparameters for GeoMatch models
generate_keypoints_json.py  # Keypoint generator script

---

Citation

@inproceedings{attarian2023geometry,
  title     = {Geometry Matching for Multi-Embodiment Grasping},
  author    = {Attarian, Maria and Asif, Muhammad Adil and Liu, Jingzhou and Hari, Ruthrash
               and Garg, Animesh and Gilitschenski, Igor and Tompson, Jonathan},
  booktitle = {Proceedings of the 7th Conference on Robot Learning (CoRL)},
  year      = {2023}
}

@article{geomatch_pp2024,
  title   = {GeoMatch++: Morphology-Aware Grasping via Correspondence Learning},
  journal = {arXiv preprint arXiv:2412.18998},
  year    = {2024},
}

@article{khargonkar2024robotfingerprint,
  title   = {RobotFingerPrint: Unified Gripper Coordinate Space for Multi-Gripper Grasp Synthesis},
  author  = {Khargonkar, Ninad and Casas, Luis Felipe and Prabhakaran, Balakrishnan and Xiang, Yu},
  journal = {arXiv preprint arXiv:2409.14519},
  year    = {2024},
}

---

License

Original GeoMatch code © 2023 DeepMind Technologies Limited, licensed under the Apache License 2.0. GeoMatch++ extension, v1/v2/v3 training, and all GeoMatch checkpoints produced by Dimios45 as part of the Graspmax project.

RobotFingerPrint model source (robotfingerprint/model/) is from the original authors' repository (MIT-style license, see their repo for exact terms); checkpoints in robotfingerprint/ were trained by Dimios45 reproducing the paper's published recipe.