Update README with 3D reconstruction docs and Space link
Browse files
README.md
CHANGED
|
@@ -1,6 +1,8 @@
|
|
| 1 |
-
# ๐ Floor Plan Parser
|
| 2 |
|
| 3 |
-
Parse floor plan images into structured data โ walls with thickness, doors, windows, and automatically detected rooms.
|
|
|
|
|
|
|
| 4 |
|
| 5 |
## Architecture
|
| 6 |
|
|
@@ -31,15 +33,28 @@ Floor Plan Image (raster or vector)
|
|
| 31 |
โ (VLM or human) โ outputs field-level corrections
|
| 32 |
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 33 |
โ
|
| 34 |
-
โผ converged? โโnoโโโ back to step
|
| 35 |
yes
|
| 36 |
โผ
|
| 37 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
```
|
| 39 |
|
| 40 |
-
##
|
| 41 |
|
| 42 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
|
| 44 |
```json
|
| 45 |
{
|
|
@@ -49,76 +64,28 @@ Walls are first-class. Everything else references them.
|
|
| 49 |
"centerline": [{"x": 0, "y": 0}, {"x": 6, "y": 0}],
|
| 50 |
"thickness": 0.24,
|
| 51 |
"openings": [
|
| 52 |
-
{
|
| 53 |
-
"id": "win1",
|
| 54 |
-
"type": "window",
|
| 55 |
-
"start": 1.5,
|
| 56 |
-
"length": 1.5
|
| 57 |
-
}
|
| 58 |
]
|
| 59 |
}
|
| 60 |
],
|
| 61 |
"rooms": [
|
| 62 |
{
|
| 63 |
-
"id": "r1",
|
| 64 |
-
"
|
| 65 |
-
"boundary": [
|
| 66 |
-
{"wall_id": "w1", "side": "right"},
|
| 67 |
-
{"wall_id": "w2", "side": "left"}
|
| 68 |
-
],
|
| 69 |
"area": 16.0
|
| 70 |
}
|
| 71 |
]
|
| 72 |
}
|
| 73 |
```
|
| 74 |
|
| 75 |
-
**Key design decisions:**
|
| 76 |
-
- **Walls have thickness** โ exterior (0.20-0.30m) vs interior (0.10-0.15m)
|
| 77 |
-
- **Openings live on walls** โ `start` + `length` along the centerline
|
| 78 |
-
- **Rooms are topological** โ defined by ordered wall-face references (left/right side), not duplicate coordinates
|
| 79 |
-
- **Curved walls** = polyline centerlines with many sample points
|
| 80 |
-
- **Non-rectangular rooms** fully supported (any angles)
|
| 81 |
-
|
| 82 |
-
## Modules
|
| 83 |
-
|
| 84 |
-
| File | Purpose |
|
| 85 |
-
|------|---------|
|
| 86 |
-
| `floorplan/schema.py` | Pydantic data models โ `Wall`, `Opening`, `Room`, `FloorPlan`, `Correction` |
|
| 87 |
-
| `floorplan/geometry.py` | Computational geometry โ wall polygons, room detection, centerline ops |
|
| 88 |
-
| `floorplan/renderer.py` | SVG + PIL rendering for visual verification and overlay |
|
| 89 |
-
| `floorplan/parser.py` | VLM-based parsing with iterative correction loop |
|
| 90 |
-
|
| 91 |
## Quick Start
|
| 92 |
|
| 93 |
### Demo (no API key needed)
|
| 94 |
|
| 95 |
-
```bash
|
| 96 |
-
pip install pydantic shapely numpy pillow
|
| 97 |
-
python example.py --demo
|
| 98 |
-
```
|
| 99 |
-
|
| 100 |
-
Renders a 2-bedroom apartment with angled walls โ `output/demo_floorplan.png` + `.svg` + `.json`
|
| 101 |
-
|
| 102 |
-
### Parse a real floor plan
|
| 103 |
-
|
| 104 |
-
```bash
|
| 105 |
-
pip install pydantic shapely numpy pillow openai
|
| 106 |
-
python example.py --image myfloorplan.png --api-key sk-...
|
| 107 |
-
```
|
| 108 |
-
|
| 109 |
-
Or with a local model (vLLM, Ollama, etc.):
|
| 110 |
-
|
| 111 |
-
```bash
|
| 112 |
-
python example.py --image myfloorplan.png --base-url http://localhost:8000/v1 --model qwen2.5-vl-72b
|
| 113 |
-
```
|
| 114 |
-
|
| 115 |
-
### Python API
|
| 116 |
-
|
| 117 |
```python
|
| 118 |
-
from floorplan import
|
| 119 |
-
from floorplan import build_rooms, render_to_image, render_floorplan_svg
|
| 120 |
|
| 121 |
-
# Build a floor plan
|
| 122 |
walls = [
|
| 123 |
Wall(id="w1", centerline=[Point2D(x=0, y=0), Point2D(x=5, y=0)], thickness=0.24,
|
| 124 |
openings=[Opening(id="d1", type=OpeningType.DOOR, start=1.0, length=0.9)]),
|
|
@@ -130,57 +97,59 @@ walls = [
|
|
| 130 |
rooms, room_polygons = build_rooms(walls)
|
| 131 |
fp = FloorPlan(walls=walls, rooms=rooms)
|
| 132 |
|
| 133 |
-
#
|
| 134 |
img = render_to_image(fp, room_polygons=room_polygons)
|
| 135 |
img.save("output.png")
|
| 136 |
|
| 137 |
-
#
|
| 138 |
-
|
| 139 |
-
|
| 140 |
-
# Serialize to JSON
|
| 141 |
-
print(fp.model_dump_json(indent=2))
|
| 142 |
```
|
| 143 |
|
| 144 |
-
###
|
| 145 |
|
| 146 |
```python
|
| 147 |
-
from floorplan import parse_floorplan
|
| 148 |
|
|
|
|
| 149 |
fp, room_polygons, overlays = parse_floorplan(
|
| 150 |
image="floorplan.png",
|
| 151 |
-
api_key="
|
| 152 |
-
|
| 153 |
-
|
|
|
|
| 154 |
)
|
| 155 |
|
| 156 |
-
#
|
| 157 |
-
|
| 158 |
-
|
| 159 |
```
|
| 160 |
|
| 161 |
-
##
|
| 162 |
|
| 163 |
-
|
| 164 |
-
-
|
| 165 |
-
-
|
| 166 |
-
-
|
| 167 |
-
-
|
| 168 |
-
-
|
| 169 |
-
-
|
| 170 |
-
|
| 171 |
-
|
| 172 |
-
- โ
**Field-level corrections** โ modify, add, delete walls/openings
|
| 173 |
|
| 174 |
## Dependencies
|
| 175 |
|
| 176 |
-
**Core**
|
| 177 |
- `pydantic` โ schema validation
|
| 178 |
- `shapely` โ computational geometry
|
| 179 |
- `numpy` โ array ops
|
| 180 |
- `pillow` โ image rendering
|
| 181 |
|
| 182 |
-
**
|
| 183 |
-
- `
|
|
|
|
|
|
|
|
|
|
| 184 |
|
| 185 |
## License
|
| 186 |
|
|
|
|
| 1 |
+
# ๐ Floor Plan Parser + 3D Reconstruction
|
| 2 |
|
| 3 |
+
Parse floor plan images into structured data โ walls with thickness, doors, windows, and automatically detected rooms. Then generate interactive 3D models.
|
| 4 |
+
|
| 5 |
+
**๐ฎ Try it live:** [rikhoffbauer2/floorplan-3d](https://huggingface.co/spaces/rikhoffbauer2/floorplan-3d)
|
| 6 |
|
| 7 |
## Architecture
|
| 8 |
|
|
|
|
| 33 |
โ (VLM or human) โ outputs field-level corrections
|
| 34 |
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 35 |
โ
|
| 36 |
+
โผ converged? โโnoโโโ back to step 2
|
| 37 |
yes
|
| 38 |
โผ
|
| 39 |
+
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 40 |
+
โ 5. 3D RECONSTRUCTION โ Extrude walls, cut openings,
|
| 41 |
+
โ (trimesh + manifold3d) โ add floors/ceilings โ GLB
|
| 42 |
+
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 43 |
+
โผ
|
| 44 |
+
Interactive 3D model (GLB/glTF)
|
| 45 |
```
|
| 46 |
|
| 47 |
+
## Modules
|
| 48 |
|
| 49 |
+
| File | Purpose |
|
| 50 |
+
|------|---------|
|
| 51 |
+
| `floorplan/schema.py` | Pydantic data models โ `Wall`, `Opening`, `Room`, `FloorPlan`, `Correction` |
|
| 52 |
+
| `floorplan/geometry.py` | Computational geometry โ wall polygons, room detection, centerline ops |
|
| 53 |
+
| `floorplan/renderer.py` | SVG + PIL rendering for visual verification and overlay |
|
| 54 |
+
| `floorplan/parser.py` | VLM-based parsing with iterative correction loop |
|
| 55 |
+
| `floorplan/reconstruction.py` | **NEW** โ 3D mesh generation, boolean ops, GLB/OBJ export |
|
| 56 |
+
|
| 57 |
+
## The Schema
|
| 58 |
|
| 59 |
```json
|
| 60 |
{
|
|
|
|
| 64 |
"centerline": [{"x": 0, "y": 0}, {"x": 6, "y": 0}],
|
| 65 |
"thickness": 0.24,
|
| 66 |
"openings": [
|
| 67 |
+
{"id": "win1", "type": "window", "start": 1.5, "length": 1.5}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 68 |
]
|
| 69 |
}
|
| 70 |
],
|
| 71 |
"rooms": [
|
| 72 |
{
|
| 73 |
+
"id": "r1", "label": "bedroom",
|
| 74 |
+
"boundary": [{"wall_id": "w1", "side": "right"}, {"wall_id": "w2", "side": "left"}],
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
"area": 16.0
|
| 76 |
}
|
| 77 |
]
|
| 78 |
}
|
| 79 |
```
|
| 80 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
## Quick Start
|
| 82 |
|
| 83 |
### Demo (no API key needed)
|
| 84 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
```python
|
| 86 |
+
from floorplan import *
|
|
|
|
| 87 |
|
| 88 |
+
# Build a floor plan
|
| 89 |
walls = [
|
| 90 |
Wall(id="w1", centerline=[Point2D(x=0, y=0), Point2D(x=5, y=0)], thickness=0.24,
|
| 91 |
openings=[Opening(id="d1", type=OpeningType.DOOR, start=1.0, length=0.9)]),
|
|
|
|
| 97 |
rooms, room_polygons = build_rooms(walls)
|
| 98 |
fp = FloorPlan(walls=walls, rooms=rooms)
|
| 99 |
|
| 100 |
+
# 2D render
|
| 101 |
img = render_to_image(fp, room_polygons=room_polygons)
|
| 102 |
img.save("output.png")
|
| 103 |
|
| 104 |
+
# 3D model
|
| 105 |
+
scene = generate_3d_model(fp, room_polygons=room_polygons)
|
| 106 |
+
export_glb(scene, "model.glb") # Open in any 3D viewer or Three.js
|
|
|
|
|
|
|
| 107 |
```
|
| 108 |
|
| 109 |
+
### Parse a real floor plan with VLM
|
| 110 |
|
| 111 |
```python
|
| 112 |
+
from floorplan import parse_floorplan, generate_3d_model, export_glb
|
| 113 |
|
| 114 |
+
# Parse image โ structured data (uses HF Inference API)
|
| 115 |
fp, room_polygons, overlays = parse_floorplan(
|
| 116 |
image="floorplan.png",
|
| 117 |
+
api_key="hf_...",
|
| 118 |
+
base_url="https://router.huggingface.co/v1",
|
| 119 |
+
model="Qwen/Qwen2.5-VL-72B-Instruct",
|
| 120 |
+
max_iterations=3,
|
| 121 |
)
|
| 122 |
|
| 123 |
+
# Generate 3D
|
| 124 |
+
scene = generate_3d_model(fp, room_polygons=room_polygons, pixels_per_meter=100)
|
| 125 |
+
export_glb(scene, "model.glb")
|
| 126 |
```
|
| 127 |
|
| 128 |
+
## 3D Output
|
| 129 |
|
| 130 |
+
The 3D reconstruction generates:
|
| 131 |
+
- **Walls** โ extruded from 2D polygons, all walls boolean-unioned into a single watertight mesh
|
| 132 |
+
- **Door openings** โ cut through walls via boolean subtraction + door frame meshes
|
| 133 |
+
- **Window openings** โ cut at sill/head height + glass pane meshes
|
| 134 |
+
- **Floor slabs** โ per room, with wood material
|
| 135 |
+
- **Ceilings** โ per room, with white material
|
| 136 |
+
- **PBR materials** โ walls (warm white), floors (light wood), ceilings (white), glass (translucent blue)
|
| 137 |
+
|
| 138 |
+
Export formats: **GLB** (web/Three.js), **glTF**, **OBJ**
|
|
|
|
| 139 |
|
| 140 |
## Dependencies
|
| 141 |
|
| 142 |
+
**Core:**
|
| 143 |
- `pydantic` โ schema validation
|
| 144 |
- `shapely` โ computational geometry
|
| 145 |
- `numpy` โ array ops
|
| 146 |
- `pillow` โ image rendering
|
| 147 |
|
| 148 |
+
**3D reconstruction:**
|
| 149 |
+
- `trimesh[easy]` โ mesh generation, boolean ops, GLB export (includes manifold3d)
|
| 150 |
+
|
| 151 |
+
**VLM parsing (optional):**
|
| 152 |
+
- `openai` โ for VLM API calls
|
| 153 |
|
| 154 |
## License
|
| 155 |
|