| --- |
| license: apache-2.0 |
| library_name: pytorch |
| pipeline_tag: feature-extraction |
| tags: |
| - tactile-sensing |
| - feature-extraction |
| - robotics |
| - pytorch |
| - convnextv2 |
| --- |
| |
| # SharpaWave Deform Encoder |
|
|
| `DeformEncoder` converts a preprocessed single-channel scalar deformation image |
| into a compact learned tactile feature with shape `[B, 512, 1, 1]`. The feature |
| can be flattened to `[B, 512]` for downstream tasks. |
|
|
| The unified checkpoint also includes `DeformDecoder` parameters. The decoder |
| and `DeformAutoencoder` are provided only to demonstrate deformation |
| reconstruction from the compact feature. |
|
|
| ## Tensor Shapes |
|
|
| | Operation | Input | Output | |
| | --- | --- | --- | |
| | Encoder | `[B, 1, 240, 240]` | `[B, 512, 1, 1]` | |
| | Flatten feature | `[B, 512, 1, 1]` | `[B, 512]` | |
| | Autoencoder | `[B, 1, 240, 240]` | `[B, 1, 240, 240]` | |
|
|
| The input is a preprocessed scalar deformation image, not a raw RGB camera |
| image. |
|
|
| ## Usage |
|
|
| Download the checkpoint and use `load_encoder()` from the source repository: |
|
|
| ```python |
| import torch |
| from huggingface_hub import hf_hub_download |
| |
| from sharpawave_deform_encoder import load_encoder |
| |
| checkpoint = hf_hub_download( |
| repo_id="Sharpa-Robotics/sharpawave-deform-encoder", |
| filename="sharpawave_deform_autoencoder.safetensors", |
| ) |
| encoder = load_encoder(checkpoint, "cpu") |
| |
| deform = torch.zeros(1, 1, 240, 240) |
| with torch.inference_mode(): |
| feature = encoder(deform) # [1, 512, 1, 1] |
| ``` |
|
|
| Source code: <https://github.com/sharpa-robotics/sharpawave-deform-encoder> |
|
|
| ## Checkpoint |
|
|
| The single SafeTensors file contains both encoder and reconstruction-only |
| decoder parameters. `load_encoder()` reads only the encoder tensors; |
| `load_autoencoder()` loads the complete demonstration model. |
|
|
| The published checkpoint was trained from random initialization without |
| upstream pretrained weights. |
|
|
| The SHA-256 digest is recorded in `SHA256SUMS`. |
|
|
| ## Limitations |
|
|
| The encoder expects the documented 240-by-240 scalar input representation. |
|
|
| ## License |
|
|
| Developed by Sharpa Group. Licensed under Apache License 2.0. See `LICENSE` and |
| `THIRD_PARTY_NOTICES.md`. |
|
|
