NeoteAI commited on
Commit
b89b4ba
verified
1 Parent(s): 319fbfd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +89 -0
README.md CHANGED
@@ -1,3 +1,92 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ tags:
4
+ - robotics
5
+ - tactile
6
+ - visuo-tactile
7
+ - representation-learning
8
+ library_name: pytorch
9
  ---
10
+
11
+ # NeoForce: A Unified Force Tactile Representation Model
12
+
13
+ [Technical Report](https://research.neoteai.com/assets/n0-foundation-paper.pdf) 路 [Project Website](http://research.neoteai.com/n0-foundation) 路 [Code](https://github.com/SparkleXFantasy/N0-Foundation-Preview) 路 [OpenNeoData](https://huggingface.co/datasets/NeoteAIEmbodied/OpenNeoData)
14
+
15
+ Tactile hardware is fragmented, so a model built on one signal format is bound to the device that produced it. NeoForce instead describes every tactile observation as a **dense three-axis force field** over the sensing surface, capturing shear and pressure in a form that is physically grounded and shared across sensors, and learns a temporally structured representation on top of it.
16
+
17
+ The model is a joint visual-tactile ViT: RGB frames and dual-sensor force fields are patched onto one shared token grid and run through a single trunk, so the modalities attend to each other directly instead of being fused after the fact. A temporal transformer over per-frame CLS pairs supplies context, and a progressive decoder reconstructs the force field and contact mask at the full input resolution, supervised at every scale it passes through.
18
+
19
+ This repository holds the weights. The code, together with the data preparation, training and evaluation guides, lives in the [code repository](https://github.com/SparkleXFantasy/N0-Foundation-Preview).
20
+
21
+ ## Files
22
+
23
+ | File | Size | What it is |
24
+ |---|---|---|
25
+ | `neoforce/neoforce.pt` | 1.25 GB | the NeoForce model |
26
+ | `visuo_tactile_conversion/visuo_tactile_conversion.pt` | 49 MB | the tactile conversion model |
27
+
28
+ ### `neoforce/neoforce.pt`
29
+
30
+ The trained NeoForce model at step 100,000, kept as a full training state rather than inference weights alone:
31
+
32
+ | Key | Contents |
33
+ |---|---|
34
+ | `model` | the student: the visual-tactile encoder plus the JEPA predictor, 156.14 M parameters |
35
+ | `teacher` | the EMA copy the JEPA and DINO targets are drawn from |
36
+ | `model_cfg` | the model configuration, so nothing has to be restated to rebuild it |
37
+ | `data_cfg`, `force_norm` | the per-channel scale the force targets were divided by |
38
+ | `step` | the training step the weights come from |
39
+
40
+ ### `visuo_tactile_conversion/visuo_tactile_conversion.pt`
41
+
42
+ The frozen tactile conversion model, which turns raw tactile camera frames into the force field NeoForce consumes:
43
+
44
+ ```
45
+ gel image -> bird-view correction -> optical flow based on the reference frame
46
+ -> conversion network -> masked, scaled force field
47
+ ```
48
+
49
+ ## Setup
50
+
51
+ Each file has a fixed path the code looks in:
52
+
53
+ ```bash
54
+ git clone https://github.com/SparkleXFantasy/N0-Foundation-Preview
55
+ cd N0-Foundation-Preview/neoforce
56
+
57
+ mkdir -p weights neoforce/visuo_tactile_conversion/model
58
+ curl -L -o weights/neoforce.pt \
59
+ https://huggingface.co/NeoteAI/NeoForce/resolve/main/neoforce/neoforce.pt
60
+ curl -L -o neoforce/visuo_tactile_conversion/model/visuo_tactile_conversion.pt \
61
+ https://huggingface.co/NeoteAI/NeoForce/resolve/main/visuo_tactile_conversion/visuo_tactile_conversion.pt
62
+ ```
63
+
64
+ The [Installation Guide](https://github.com/SparkleXFantasy/N0-Foundation-Preview/blob/main/neoforce/docs/Installation.md) covers the rest of the setup.
65
+
66
+ ## Model
67
+
68
+ | | |
69
+ |---|---|
70
+ | Backbone | ViT-B, 12 layers, 768-dim, patch 20 |
71
+ | Input | 4 frames @ 30 fps, RGB 360脳640 + tactile representation `(6, 360, 640)` |
72
+ | Output | tactile representation `(6, 360, 640)`, contact mask `(6, 360, 640)`, global force `(6,)` |
73
+
74
+ The six force channels are `[left fx, fy, fz, right fx, fy, fz]`, one triplet per tactile sensor.
75
+
76
+ Trained on [OpenNeoData](https://huggingface.co/datasets/NeoteAIEmbodied/OpenNeoData), the 5,000-hour open-source subset of NeoData.
77
+
78
+ ## Citation
79
+
80
+ ```bibtex
81
+ @misc{n0foundation,
82
+ title={N0-Foundation: Towards the Age of Tactile Intelligence},
83
+ author={NeoteAI Team and TEAI Team},
84
+ year={2026},
85
+ url={https://research.neoteai.com/assets/n0-foundation-paper.pdf},
86
+ note={Technical Report}
87
+ }
88
+ ```
89
+
90
+ ## License
91
+
92
+ [MIT](https://opensource.org/licenses/MIT)