rikhoffbauer2 commited on
Commit
1aa7462
ยท
verified ยท
1 Parent(s): 7dc3aac

Update README with 3D reconstruction docs and Space link

Browse files
Files changed (1) hide show
  1. README.md +56 -87
README.md CHANGED
@@ -1,6 +1,8 @@
1
- # ๐Ÿ  Floor Plan Parser
2
 
3
- Parse floor plan images into structured data โ€” walls with thickness, doors, windows, and automatically detected rooms.
 
 
4
 
5
  ## Architecture
6
 
@@ -31,15 +33,28 @@ Floor Plan Image (raster or vector)
31
  โ”‚ (VLM or human) โ”‚ outputs field-level corrections
32
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
33
  โ”‚
34
- โ–ผ converged? โ”€โ”€noโ”€โ”€โ†’ back to step 1
35
  yes
36
  โ–ผ
37
- Final structured schema (JSON)
 
 
 
 
 
38
  ```
39
 
40
- ## The Schema
41
 
42
- Walls are first-class. Everything else references them.
 
 
 
 
 
 
 
 
43
 
44
  ```json
45
  {
@@ -49,76 +64,28 @@ Walls are first-class. Everything else references them.
49
  "centerline": [{"x": 0, "y": 0}, {"x": 6, "y": 0}],
50
  "thickness": 0.24,
51
  "openings": [
52
- {
53
- "id": "win1",
54
- "type": "window",
55
- "start": 1.5,
56
- "length": 1.5
57
- }
58
  ]
59
  }
60
  ],
61
  "rooms": [
62
  {
63
- "id": "r1",
64
- "label": "bedroom",
65
- "boundary": [
66
- {"wall_id": "w1", "side": "right"},
67
- {"wall_id": "w2", "side": "left"}
68
- ],
69
  "area": 16.0
70
  }
71
  ]
72
  }
73
  ```
74
 
75
- **Key design decisions:**
76
- - **Walls have thickness** โ€” exterior (0.20-0.30m) vs interior (0.10-0.15m)
77
- - **Openings live on walls** โ€” `start` + `length` along the centerline
78
- - **Rooms are topological** โ€” defined by ordered wall-face references (left/right side), not duplicate coordinates
79
- - **Curved walls** = polyline centerlines with many sample points
80
- - **Non-rectangular rooms** fully supported (any angles)
81
-
82
- ## Modules
83
-
84
- | File | Purpose |
85
- |------|---------|
86
- | `floorplan/schema.py` | Pydantic data models โ€” `Wall`, `Opening`, `Room`, `FloorPlan`, `Correction` |
87
- | `floorplan/geometry.py` | Computational geometry โ€” wall polygons, room detection, centerline ops |
88
- | `floorplan/renderer.py` | SVG + PIL rendering for visual verification and overlay |
89
- | `floorplan/parser.py` | VLM-based parsing with iterative correction loop |
90
-
91
  ## Quick Start
92
 
93
  ### Demo (no API key needed)
94
 
95
- ```bash
96
- pip install pydantic shapely numpy pillow
97
- python example.py --demo
98
- ```
99
-
100
- Renders a 2-bedroom apartment with angled walls โ†’ `output/demo_floorplan.png` + `.svg` + `.json`
101
-
102
- ### Parse a real floor plan
103
-
104
- ```bash
105
- pip install pydantic shapely numpy pillow openai
106
- python example.py --image myfloorplan.png --api-key sk-...
107
- ```
108
-
109
- Or with a local model (vLLM, Ollama, etc.):
110
-
111
- ```bash
112
- python example.py --image myfloorplan.png --base-url http://localhost:8000/v1 --model qwen2.5-vl-72b
113
- ```
114
-
115
- ### Python API
116
-
117
  ```python
118
- from floorplan import FloorPlan, Wall, Opening, OpeningType, Point2D
119
- from floorplan import build_rooms, render_to_image, render_floorplan_svg
120
 
121
- # Build a floor plan programmatically
122
  walls = [
123
  Wall(id="w1", centerline=[Point2D(x=0, y=0), Point2D(x=5, y=0)], thickness=0.24,
124
  openings=[Opening(id="d1", type=OpeningType.DOOR, start=1.0, length=0.9)]),
@@ -130,57 +97,59 @@ walls = [
130
  rooms, room_polygons = build_rooms(walls)
131
  fp = FloorPlan(walls=walls, rooms=rooms)
132
 
133
- # Render
134
  img = render_to_image(fp, room_polygons=room_polygons)
135
  img.save("output.png")
136
 
137
- # Export SVG
138
- svg = render_floorplan_svg(fp, room_polygons=room_polygons)
139
-
140
- # Serialize to JSON
141
- print(fp.model_dump_json(indent=2))
142
  ```
143
 
144
- ### VLM parsing with correction loop
145
 
146
  ```python
147
- from floorplan import parse_floorplan
148
 
 
149
  fp, room_polygons, overlays = parse_floorplan(
150
  image="floorplan.png",
151
- api_key="sk-...",
152
- model="gpt-4o",
153
- max_iterations=4,
 
154
  )
155
 
156
- # overlays contains the render-on-original images for each iteration
157
- for i, overlay in enumerate(overlays):
158
- overlay.save(f"overlay_{i}.png")
159
  ```
160
 
161
- ## Features
162
 
163
- - โœ… **Wall thickness** โ€” not just centerlines
164
- - โœ… **Non-rectangular rooms** โ€” any angles
165
- - โœ… **Curved walls** โ€” polyline approximation
166
- - โœ… **Automatic room detection** โ€” from wall topology via Shapely
167
- - โœ… **Doors & windows** โ€” parametric on walls
168
- - โœ… **SVG + PNG rendering** โ€” dual output
169
- - โœ… **Overlay verification** โ€” rendered schema composited on original
170
- - โœ… **Iterative VLM correction** โ€” parse โ†’ render โ†’ compare โ†’ correct โ†’ repeat
171
- - โœ… **JSON serialization** โ€” full Pydantic roundtrip
172
- - โœ… **Field-level corrections** โ€” modify, add, delete walls/openings
173
 
174
  ## Dependencies
175
 
176
- **Core** (always needed):
177
  - `pydantic` โ€” schema validation
178
  - `shapely` โ€” computational geometry
179
  - `numpy` โ€” array ops
180
  - `pillow` โ€” image rendering
181
 
182
- **VLM parsing** (optional):
183
- - `openai` โ€” for VLM API calls (works with any OpenAI-compatible endpoint)
 
 
 
184
 
185
  ## License
186
 
 
1
+ # ๐Ÿ  Floor Plan Parser + 3D Reconstruction
2
 
3
+ Parse floor plan images into structured data โ€” walls with thickness, doors, windows, and automatically detected rooms. Then generate interactive 3D models.
4
+
5
+ **๐ŸŽฎ Try it live:** [rikhoffbauer2/floorplan-3d](https://huggingface.co/spaces/rikhoffbauer2/floorplan-3d)
6
 
7
  ## Architecture
8
 
 
33
  โ”‚ (VLM or human) โ”‚ outputs field-level corrections
34
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
35
  โ”‚
36
+ โ–ผ converged? โ”€โ”€noโ”€โ”€โ†’ back to step 2
37
  yes
38
  โ–ผ
39
+ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
40
+ โ”‚ 5. 3D RECONSTRUCTION โ”‚ Extrude walls, cut openings,
41
+ โ”‚ (trimesh + manifold3d) โ”‚ add floors/ceilings โ†’ GLB
42
+ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
43
+ โ–ผ
44
+ Interactive 3D model (GLB/glTF)
45
  ```
46
 
47
+ ## Modules
48
 
49
+ | File | Purpose |
50
+ |------|---------|
51
+ | `floorplan/schema.py` | Pydantic data models โ€” `Wall`, `Opening`, `Room`, `FloorPlan`, `Correction` |
52
+ | `floorplan/geometry.py` | Computational geometry โ€” wall polygons, room detection, centerline ops |
53
+ | `floorplan/renderer.py` | SVG + PIL rendering for visual verification and overlay |
54
+ | `floorplan/parser.py` | VLM-based parsing with iterative correction loop |
55
+ | `floorplan/reconstruction.py` | **NEW** โ€” 3D mesh generation, boolean ops, GLB/OBJ export |
56
+
57
+ ## The Schema
58
 
59
  ```json
60
  {
 
64
  "centerline": [{"x": 0, "y": 0}, {"x": 6, "y": 0}],
65
  "thickness": 0.24,
66
  "openings": [
67
+ {"id": "win1", "type": "window", "start": 1.5, "length": 1.5}
 
 
 
 
 
68
  ]
69
  }
70
  ],
71
  "rooms": [
72
  {
73
+ "id": "r1", "label": "bedroom",
74
+ "boundary": [{"wall_id": "w1", "side": "right"}, {"wall_id": "w2", "side": "left"}],
 
 
 
 
75
  "area": 16.0
76
  }
77
  ]
78
  }
79
  ```
80
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
81
  ## Quick Start
82
 
83
  ### Demo (no API key needed)
84
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
85
  ```python
86
+ from floorplan import *
 
87
 
88
+ # Build a floor plan
89
  walls = [
90
  Wall(id="w1", centerline=[Point2D(x=0, y=0), Point2D(x=5, y=0)], thickness=0.24,
91
  openings=[Opening(id="d1", type=OpeningType.DOOR, start=1.0, length=0.9)]),
 
97
  rooms, room_polygons = build_rooms(walls)
98
  fp = FloorPlan(walls=walls, rooms=rooms)
99
 
100
+ # 2D render
101
  img = render_to_image(fp, room_polygons=room_polygons)
102
  img.save("output.png")
103
 
104
+ # 3D model
105
+ scene = generate_3d_model(fp, room_polygons=room_polygons)
106
+ export_glb(scene, "model.glb") # Open in any 3D viewer or Three.js
 
 
107
  ```
108
 
109
+ ### Parse a real floor plan with VLM
110
 
111
  ```python
112
+ from floorplan import parse_floorplan, generate_3d_model, export_glb
113
 
114
+ # Parse image โ†’ structured data (uses HF Inference API)
115
  fp, room_polygons, overlays = parse_floorplan(
116
  image="floorplan.png",
117
+ api_key="hf_...",
118
+ base_url="https://router.huggingface.co/v1",
119
+ model="Qwen/Qwen2.5-VL-72B-Instruct",
120
+ max_iterations=3,
121
  )
122
 
123
+ # Generate 3D
124
+ scene = generate_3d_model(fp, room_polygons=room_polygons, pixels_per_meter=100)
125
+ export_glb(scene, "model.glb")
126
  ```
127
 
128
+ ## 3D Output
129
 
130
+ The 3D reconstruction generates:
131
+ - **Walls** โ€” extruded from 2D polygons, all walls boolean-unioned into a single watertight mesh
132
+ - **Door openings** โ€” cut through walls via boolean subtraction + door frame meshes
133
+ - **Window openings** โ€” cut at sill/head height + glass pane meshes
134
+ - **Floor slabs** โ€” per room, with wood material
135
+ - **Ceilings** โ€” per room, with white material
136
+ - **PBR materials** โ€” walls (warm white), floors (light wood), ceilings (white), glass (translucent blue)
137
+
138
+ Export formats: **GLB** (web/Three.js), **glTF**, **OBJ**
 
139
 
140
  ## Dependencies
141
 
142
+ **Core:**
143
  - `pydantic` โ€” schema validation
144
  - `shapely` โ€” computational geometry
145
  - `numpy` โ€” array ops
146
  - `pillow` โ€” image rendering
147
 
148
+ **3D reconstruction:**
149
+ - `trimesh[easy]` โ€” mesh generation, boolean ops, GLB export (includes manifold3d)
150
+
151
+ **VLM parsing (optional):**
152
+ - `openai` โ€” for VLM API calls
153
 
154
  ## License
155