coderuday21 Cursor commited on
Commit
96454a9
·
1 Parent(s): ac7ea7c

Add zone-aware library organization for dev DDA uploads.

Browse files

Unify disk layout and DB hierarchy as zone/folder/year paths, with CRUD APIs, upload assignment, legacy migration, and a navigable library tree in the UI.

Co-authored-by: Cursor <cursoragent@cursor.com>

DEPLOYMENT.md CHANGED
@@ -133,7 +133,7 @@ Set these in each Space’s **Settings → Repository secrets / Variables** if n
133
  | Variable | Purpose |
134
  |----------|---------|
135
  | `APP_MODE` | Set to `dda` on **satdetect-dev** only (enables DDA library UI) |
136
- | `LOCAL_LIBRARY_ROOT` | Path to year folders (default: `library_sources/` in project) |
137
  | `MAX_GEOTIFF_MB` | Library GeoTIFF upload cap (default **5120** = 5 GB on dev) |
138
  | `MAX_IMAGE_MB` | PNG/JPEG library cap (default 50 MB) |
139
  | `SECRET_KEY` | Optional legacy JWT setting (login disabled) |
@@ -148,20 +148,44 @@ Dev Space can omit `SECRET_KEY` (login is disabled on both Spaces).
148
 
149
  ---
150
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
151
  ## UAT checklist (satdetect-dev)
152
 
153
  Run before promoting any DDA feature to production:
154
 
155
  1. **Health** — `GET /health` returns `status: ok`, `appMode: dda`, `dda.libraryImages` ≥ 0
156
- 2. **Library** — Upload GeoTIFF to year folder; Refresh shows image; thumb loads
157
- 3. **Compare** — Select T1/T2; Run Detection completes (async job or sync fallback)
158
- 4. **Viewer** — Slider / T1 / T2 / Overlay modes; click region to locate
159
- 5. **Review** — Confirm and False Positive; Export confirmed CSV; Submit confirmed
160
- 6. **Reports** — PDF download; `/dda/reports/{id}` page; email link (if SMTP configured)
161
- 7. **Session isolation** — Two browsers see separate history (per-session cookie)
162
- 8. **Admin** — `GET /api/dda/admin/status` (analyst+); stale job reconcile after restart
163
- 9. **Training export** — Mark false positives → `GET /api/dda/training/export` (admin or export key)
164
- 10. **Security** — Path traversal blocked (`../` in library path); upload size limit enforced
 
165
 
166
  Sign-off: DDA stakeholder approves sample runs on dev Space before any push to `satdetect` production.
167
 
 
133
  | Variable | Purpose |
134
  |----------|---------|
135
  | `APP_MODE` | Set to `dda` on **satdetect-dev** only (enables DDA library UI) |
136
+ | `LOCAL_LIBRARY_ROOT` | Path to library root (default: `library_sources/` or `data/library_sources/` on HF) |
137
  | `MAX_GEOTIFF_MB` | Library GeoTIFF upload cap (default **5120** = 5 GB on dev) |
138
  | `MAX_IMAGE_MB` | PNG/JPEG library cap (default 50 MB) |
139
  | `SECRET_KEY` | Optional legacy JWT setting (login disabled) |
 
148
 
149
  ---
150
 
151
+ ## Library folder layout (zone-aware)
152
+
153
+ Images are stored under a **zone → folder → year** hierarchy:
154
+
155
+ ```text
156
+ library_sources/
157
+ central_delhi/
158
+ site_a/
159
+ 2025/
160
+ image.tif
161
+ _unassigned/
162
+ legacy/
163
+ 2025/
164
+ old_image.tif
165
+ ```
166
+
167
+ - **Zones** and **folders** are managed in the UI (admin: **Manage** in the library sidebar).
168
+ - **Upload** requires selecting zone, folder, and year (HF dev and local upload form).
169
+ - On first startup after upgrade, flat `library_sources/YEAR/` files are moved to `_unassigned/legacy/YEAR/`.
170
+ - Slugs on disk (e.g. `central_delhi`) are stable; renaming a zone/folder in the UI changes the display label only.
171
+
172
+ ---
173
+
174
  ## UAT checklist (satdetect-dev)
175
 
176
  Run before promoting any DDA feature to production:
177
 
178
  1. **Health** — `GET /health` returns `status: ok`, `appMode: dda`, `dda.libraryImages` ≥ 0
179
+ 2. **Library** — Create zone + folder (admin); upload GeoTIFF with zone/folder/year; tree and grid show breadcrumb; Refresh works
180
+ 3. **Legacy** — Old flat year files appear under Unassigned; Assign location moves file to chosen zone/folder/year
181
+ 4. **Compare** — Select T1/T2; Run Detection completes (async job or sync fallback)
182
+ 5. **Viewer** — Slider / T1 / T2 / Overlay modes; click region to locate
183
+ 6. **Review** — Confirm and False Positive; Export confirmed CSV; Submit confirmed
184
+ 7. **Reports** — PDF download; `/dda/reports/{id}` page; email link (if SMTP configured)
185
+ 8. **Session isolation** — Two browsers see separate history (per-session cookie)
186
+ 9. **Admin** — `GET /api/dda/admin/status` (analyst+); stale job reconcile after restart
187
+ 10. **Training export** — Mark false positives `GET /api/dda/training/export` (admin or export key)
188
+ 11. **Security** — Path traversal blocked (`../` in library path); upload size limit enforced
189
 
190
  Sign-off: DDA stakeholder approves sample runs on dev Space before any push to `satdetect` production.
191
 
app/dda/bootstrap.py CHANGED
@@ -4,8 +4,9 @@ from fastapi import FastAPI
4
  from sqlalchemy import text as sa_text
5
 
6
  from ..database import engine
7
- from .config import IS_DDA_MODE, ensure_library_dirs, ensure_local_year_folders, is_hf_hosted
8
  from .admin_routes import router as admin_router
 
9
  from .jobs_routes import router as jobs_router
10
  from .library_routes import router as library_router
11
  from .local_routes import router as local_router
@@ -23,12 +24,13 @@ def init_dda_database():
23
  if not IS_DDA_MODE:
24
  return
25
  ensure_library_dirs()
26
- ensure_local_year_folders()
27
  try:
28
  with engine.connect() as conn:
29
  for stmt in (
30
  "ALTER TABLE users ADD COLUMN role VARCHAR(32) DEFAULT 'analyst'",
31
  "ALTER TABLE detection_runs ADD COLUMN after_full_path VARCHAR(512) DEFAULT ''",
 
 
32
  ):
33
  try:
34
  conn.execute(sa_text(stmt))
@@ -43,14 +45,37 @@ def init_dda_database():
43
  conn.commit()
44
  except Exception:
45
  conn.rollback()
 
 
 
 
 
 
 
46
  except Exception as exc:
47
  logger.warning("DDA schema migration skipped: %s", exc)
48
 
49
- from ..database import SessionLocal
 
 
 
 
 
 
50
  db = SessionLocal()
51
  try:
52
  seed_delhi_hierarchy(db)
53
  seed_dda_admin(db)
 
 
 
 
 
 
 
 
 
 
54
  from .job_runner import reconcile_stale_jobs
55
  reconcile_stale_jobs(db)
56
  finally:
@@ -58,11 +83,17 @@ def init_dda_database():
58
 
59
  try:
60
  from .local_library import library_debug_info, scan_images
61
- info = library_debug_info()
 
 
 
 
 
 
62
  logger.info(
63
  "DDA library ready (hosted=%s): %d images, writable=%s",
64
  is_hf_hosted(),
65
- len(scan_images()),
66
  info.get("roots", [{}])[0].get("path") if info.get("roots") else "?",
67
  )
68
  except Exception as exc:
@@ -79,5 +110,6 @@ def setup_dda(app: FastAPI) -> None:
79
  app.include_router(review_router, prefix="/api/dda", tags=["dda-review"])
80
  app.include_router(training_router, prefix="/api/dda", tags=["dda-training"])
81
  app.include_router(admin_router, prefix="/api/dda", tags=["dda-admin"])
 
82
  app.include_router(local_router, prefix="/api/dda", tags=["dda-local"])
83
- logger.info("APP_MODE=dda — DDA routes enabled (library, jobs, reports, review, training, admin, local)")
 
4
  from sqlalchemy import text as sa_text
5
 
6
  from ..database import engine
7
+ from .config import IS_DDA_MODE, ensure_library_dirs, is_hf_hosted
8
  from .admin_routes import router as admin_router
9
+ from .hierarchy_routes import router as hierarchy_router
10
  from .jobs_routes import router as jobs_router
11
  from .library_routes import router as library_router
12
  from .local_routes import router as local_router
 
24
  if not IS_DDA_MODE:
25
  return
26
  ensure_library_dirs()
 
27
  try:
28
  with engine.connect() as conn:
29
  for stmt in (
30
  "ALTER TABLE users ADD COLUMN role VARCHAR(32) DEFAULT 'analyst'",
31
  "ALTER TABLE detection_runs ADD COLUMN after_full_path VARCHAR(512) DEFAULT ''",
32
+ "ALTER TABLE dda_zones ADD COLUMN slug VARCHAR(64)",
33
+ "ALTER TABLE dda_villages ADD COLUMN slug VARCHAR(64)",
34
  ):
35
  try:
36
  conn.execute(sa_text(stmt))
 
45
  conn.commit()
46
  except Exception:
47
  conn.rollback()
48
+ try:
49
+ conn.execute(sa_text(
50
+ "CREATE UNIQUE INDEX IF NOT EXISTS ix_dda_zones_slug ON dda_zones (slug)"
51
+ ))
52
+ conn.commit()
53
+ except Exception:
54
+ conn.rollback()
55
  except Exception as exc:
56
  logger.warning("DDA schema migration skipped: %s", exc)
57
 
58
+ from ..database import Base, SessionLocal
59
+ try:
60
+ from .models import DdaLocalFileIndex # noqa: F401
61
+ Base.metadata.create_all(bind=engine, tables=[DdaLocalFileIndex.__table__])
62
+ except Exception as exc:
63
+ logger.warning("DdaLocalFileIndex table create skipped: %s", exc)
64
+
65
  db = SessionLocal()
66
  try:
67
  seed_delhi_hierarchy(db)
68
  seed_dda_admin(db)
69
+ from .library_migration import (
70
+ backfill_slugs,
71
+ ensure_legacy_zone,
72
+ ensure_zone_folder_dirs,
73
+ migrate_legacy_flat_years,
74
+ )
75
+ backfill_slugs(db)
76
+ ensure_legacy_zone(db)
77
+ migrate_legacy_flat_years(db)
78
+ ensure_zone_folder_dirs(db)
79
  from .job_runner import reconcile_stale_jobs
80
  reconcile_stale_jobs(db)
81
  finally:
 
83
 
84
  try:
85
  from .local_library import library_debug_info, scan_images
86
+ from ..database import SessionLocal as SL
87
+ sdb = SL()
88
+ try:
89
+ info = library_debug_info(db=sdb)
90
+ total = len(scan_images(db=sdb))
91
+ finally:
92
+ sdb.close()
93
  logger.info(
94
  "DDA library ready (hosted=%s): %d images, writable=%s",
95
  is_hf_hosted(),
96
+ total,
97
  info.get("roots", [{}])[0].get("path") if info.get("roots") else "?",
98
  )
99
  except Exception as exc:
 
110
  app.include_router(review_router, prefix="/api/dda", tags=["dda-review"])
111
  app.include_router(training_router, prefix="/api/dda", tags=["dda-training"])
112
  app.include_router(admin_router, prefix="/api/dda", tags=["dda-admin"])
113
+ app.include_router(hierarchy_router, prefix="/api/dda", tags=["dda-hierarchy"])
114
  app.include_router(local_router, prefix="/api/dda", tags=["dda-local"])
115
+ logger.info("APP_MODE=dda — DDA routes enabled (library, jobs, reports, review, training, admin, hierarchy, local)")
app/dda/hierarchy_routes.py ADDED
@@ -0,0 +1,253 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """CRUD for zone/folder library hierarchy."""
2
+ from __future__ import annotations
3
+
4
+ import logging
5
+ from typing import Optional
6
+
7
+ from fastapi import APIRouter, Depends, HTTPException
8
+ from pydantic import BaseModel, Field
9
+ from sqlalchemy.orm import Session
10
+
11
+ from ..database import get_db
12
+ from ..models import User
13
+ from .config import IS_DDA_MODE, get_writable_library_root
14
+ from .dda_auth import current_dda_user, get_user_role, require_min_role
15
+ from .local_library import count_files_in_folder, count_files_in_zone, scan_tree
16
+ from .models import DdaLocalFileIndex, DdaVillage, DdaZone
17
+ from .path_slugs import unique_slug, validate_path_segment
18
+
19
+ logger = logging.getLogger(__name__)
20
+ router = APIRouter()
21
+
22
+
23
+ def _require_dda():
24
+ if not IS_DDA_MODE:
25
+ raise HTTPException(status_code=404, detail="DDA mode is not enabled")
26
+
27
+
28
+ class ZoneCreateBody(BaseModel):
29
+ name: str = Field(..., min_length=1, max_length=128)
30
+
31
+
32
+ class ZonePatchBody(BaseModel):
33
+ name: str = Field(..., min_length=1, max_length=128)
34
+
35
+
36
+ class FolderCreateBody(BaseModel):
37
+ name: str = Field(..., min_length=1, max_length=128)
38
+
39
+
40
+ class FolderPatchBody(BaseModel):
41
+ name: str = Field(..., min_length=1, max_length=128)
42
+
43
+
44
+ class YearCreateBody(BaseModel):
45
+ year: int = Field(..., ge=1990, le=2100)
46
+
47
+
48
+ def _zone_dict(zone: DdaZone) -> dict:
49
+ return {"id": zone.id, "name": zone.name, "slug": zone.slug, "mode": zone.mode}
50
+
51
+
52
+ def _folder_dict(folder: DdaVillage) -> dict:
53
+ return {"id": folder.id, "zoneId": folder.zone_id, "name": folder.name, "slug": folder.slug}
54
+
55
+
56
+ def _existing_zone_slugs(db: Session) -> set:
57
+ return {z.slug for z in db.query(DdaZone).filter(DdaZone.slug.isnot(None)).all() if z.slug}
58
+
59
+
60
+ def _existing_folder_slugs(db: Session, zone_id: int) -> set:
61
+ return {
62
+ f.slug for f in db.query(DdaVillage).filter(
63
+ DdaVillage.zone_id == zone_id, DdaVillage.slug.isnot(None)
64
+ ).all() if f.slug
65
+ }
66
+
67
+
68
+ @router.get("/me")
69
+ def dda_me(user: User = Depends(current_dda_user), db: Session = Depends(get_db)):
70
+ _require_dda()
71
+ return {"userId": user.id, "role": get_user_role(db, user), "email": user.email}
72
+
73
+
74
+ @router.get("/hierarchy/tree")
75
+ def hierarchy_tree(db: Session = Depends(get_db), user: User = Depends(current_dda_user)):
76
+ _require_dda()
77
+ return scan_tree(db)
78
+
79
+
80
+ @router.post("/hierarchy/zones")
81
+ def create_zone(
82
+ body: ZoneCreateBody,
83
+ db: Session = Depends(get_db),
84
+ user: User = Depends(current_dda_user),
85
+ ):
86
+ _require_dda()
87
+ require_min_role(user, db, "admin")
88
+ name = body.name.strip()
89
+ if db.query(DdaZone).filter(DdaZone.name == name).first():
90
+ raise HTTPException(status_code=400, detail="Zone name already exists")
91
+ slug = unique_slug(name, _existing_zone_slugs(db))
92
+ if not validate_path_segment(slug):
93
+ raise HTTPException(status_code=400, detail="Could not derive a valid folder slug")
94
+ zone = DdaZone(name=name, slug=slug, mode="admin")
95
+ db.add(zone)
96
+ db.commit()
97
+ db.refresh(zone)
98
+ root = get_writable_library_root()
99
+ (root / slug).mkdir(parents=True, exist_ok=True)
100
+ logger.info("Created zone %s (%s)", name, slug)
101
+ return {"zone": _zone_dict(zone)}
102
+
103
+
104
+ @router.patch("/hierarchy/zones/{zone_id}")
105
+ def patch_zone(
106
+ zone_id: int,
107
+ body: ZonePatchBody,
108
+ db: Session = Depends(get_db),
109
+ user: User = Depends(current_dda_user),
110
+ ):
111
+ _require_dda()
112
+ require_min_role(user, db, "admin")
113
+ zone = db.query(DdaZone).filter(DdaZone.id == zone_id).first()
114
+ if not zone:
115
+ raise HTTPException(status_code=404, detail="Zone not found")
116
+ name = body.name.strip()
117
+ other = db.query(DdaZone).filter(DdaZone.name == name, DdaZone.id != zone_id).first()
118
+ if other:
119
+ raise HTTPException(status_code=400, detail="Zone name already exists")
120
+ zone.name = name
121
+ db.commit()
122
+ db.refresh(zone)
123
+ return {"zone": _zone_dict(zone)}
124
+
125
+
126
+ @router.delete("/hierarchy/zones/{zone_id}")
127
+ def delete_zone(
128
+ zone_id: int,
129
+ db: Session = Depends(get_db),
130
+ user: User = Depends(current_dda_user),
131
+ ):
132
+ _require_dda()
133
+ require_min_role(user, db, "admin")
134
+ zone = db.query(DdaZone).filter(DdaZone.id == zone_id).first()
135
+ if not zone:
136
+ raise HTTPException(status_code=404, detail="Zone not found")
137
+ if not zone.slug:
138
+ raise HTTPException(status_code=400, detail="Zone has no slug")
139
+ root = get_writable_library_root()
140
+ if count_files_in_zone(root, zone.slug) > 0:
141
+ raise HTTPException(status_code=400, detail="Zone folder is not empty")
142
+ indexed = db.query(DdaLocalFileIndex).filter(DdaLocalFileIndex.zone_id == zone_id).count()
143
+ if indexed > 0:
144
+ raise HTTPException(status_code=400, detail="Zone has indexed files")
145
+ zone_dir = root / zone.slug
146
+ db.delete(zone)
147
+ db.commit()
148
+ if zone_dir.exists() and zone_dir.is_dir():
149
+ try:
150
+ zone_dir.rmdir()
151
+ except OSError:
152
+ pass
153
+ return {"ok": True}
154
+
155
+
156
+ @router.post("/hierarchy/zones/{zone_id}/folders")
157
+ def create_folder(
158
+ zone_id: int,
159
+ body: FolderCreateBody,
160
+ db: Session = Depends(get_db),
161
+ user: User = Depends(current_dda_user),
162
+ ):
163
+ _require_dda()
164
+ require_min_role(user, db, "admin")
165
+ zone = db.query(DdaZone).filter(DdaZone.id == zone_id).first()
166
+ if not zone or not zone.slug:
167
+ raise HTTPException(status_code=404, detail="Zone not found")
168
+ name = body.name.strip()
169
+ existing = db.query(DdaVillage).filter(DdaVillage.zone_id == zone_id, DdaVillage.name == name).first()
170
+ if existing:
171
+ raise HTTPException(status_code=400, detail="Folder name already exists in this zone")
172
+ slug = unique_slug(name, _existing_folder_slugs(db, zone_id))
173
+ folder = DdaVillage(zone_id=zone_id, name=name, slug=slug)
174
+ db.add(folder)
175
+ db.commit()
176
+ db.refresh(folder)
177
+ (get_writable_library_root() / zone.slug / slug).mkdir(parents=True, exist_ok=True)
178
+ return {"folder": _folder_dict(folder)}
179
+
180
+
181
+ @router.patch("/hierarchy/folders/{folder_id}")
182
+ def patch_folder(
183
+ folder_id: int,
184
+ body: FolderPatchBody,
185
+ db: Session = Depends(get_db),
186
+ user: User = Depends(current_dda_user),
187
+ ):
188
+ _require_dda()
189
+ require_min_role(user, db, "admin")
190
+ folder = db.query(DdaVillage).filter(DdaVillage.id == folder_id).first()
191
+ if not folder:
192
+ raise HTTPException(status_code=404, detail="Folder not found")
193
+ name = body.name.strip()
194
+ other = db.query(DdaVillage).filter(
195
+ DdaVillage.zone_id == folder.zone_id, DdaVillage.name == name, DdaVillage.id != folder_id
196
+ ).first()
197
+ if other:
198
+ raise HTTPException(status_code=400, detail="Folder name already exists in this zone")
199
+ folder.name = name
200
+ db.commit()
201
+ db.refresh(folder)
202
+ return {"folder": _folder_dict(folder)}
203
+
204
+
205
+ @router.delete("/hierarchy/folders/{folder_id}")
206
+ def delete_folder(
207
+ folder_id: int,
208
+ db: Session = Depends(get_db),
209
+ user: User = Depends(current_dda_user),
210
+ ):
211
+ _require_dda()
212
+ require_min_role(user, db, "admin")
213
+ folder = db.query(DdaVillage).filter(DdaVillage.id == folder_id).first()
214
+ if not folder:
215
+ raise HTTPException(status_code=404, detail="Folder not found")
216
+ zone = db.query(DdaZone).filter(DdaZone.id == folder.zone_id).first()
217
+ if not zone or not zone.slug or not folder.slug:
218
+ raise HTTPException(status_code=400, detail="Folder has no disk path")
219
+ root = get_writable_library_root()
220
+ if count_files_in_folder(root, zone.slug, folder.slug) > 0:
221
+ raise HTTPException(status_code=400, detail="Folder is not empty")
222
+ indexed = db.query(DdaLocalFileIndex).filter(DdaLocalFileIndex.folder_id == folder_id).count()
223
+ if indexed > 0:
224
+ raise HTTPException(status_code=400, detail="Folder has indexed files")
225
+ folder_dir = root / zone.slug / folder.slug
226
+ db.delete(folder)
227
+ db.commit()
228
+ if folder_dir.exists() and folder_dir.is_dir():
229
+ try:
230
+ folder_dir.rmdir()
231
+ except OSError:
232
+ pass
233
+ return {"ok": True}
234
+
235
+
236
+ @router.post("/hierarchy/folders/{folder_id}/years")
237
+ def create_year_folder(
238
+ folder_id: int,
239
+ body: YearCreateBody,
240
+ db: Session = Depends(get_db),
241
+ user: User = Depends(current_dda_user),
242
+ ):
243
+ _require_dda()
244
+ require_min_role(user, db, "uploader")
245
+ folder = db.query(DdaVillage).filter(DdaVillage.id == folder_id).first()
246
+ if not folder:
247
+ raise HTTPException(status_code=404, detail="Folder not found")
248
+ zone = db.query(DdaZone).filter(DdaZone.id == folder.zone_id).first()
249
+ if not zone or not zone.slug or not folder.slug:
250
+ raise HTTPException(status_code=400, detail="Folder has no disk path")
251
+ year_dir = get_writable_library_root() / zone.slug / folder.slug / str(body.year)
252
+ year_dir.mkdir(parents=True, exist_ok=True)
253
+ return {"ok": True, "path": f"{zone.slug}/{folder.slug}/{body.year}"}
app/dda/library_migration.py ADDED
@@ -0,0 +1,148 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """One-time library layout migrations and slug backfill."""
2
+ from __future__ import annotations
3
+
4
+ import logging
5
+ import shutil
6
+ from pathlib import Path
7
+ from typing import Set
8
+
9
+ from sqlalchemy.orm import Session
10
+
11
+ from .config import ALLOWED_EXTENSIONS, get_library_roots, get_writable_library_root
12
+ from .models import DdaLocalFileIndex, DdaVillage, DdaZone
13
+ from .path_slugs import slugify, unique_slug
14
+
15
+ logger = logging.getLogger(__name__)
16
+
17
+ LEGACY_ZONE_NAME = "Unassigned"
18
+ LEGACY_ZONE_SLUG = "_unassigned"
19
+ LEGACY_FOLDER_NAME = "Legacy"
20
+ LEGACY_FOLDER_SLUG = "legacy"
21
+
22
+
23
+ def _is_year_dir(name: str) -> bool:
24
+ return len(name) == 4 and name.isdigit() and 1990 <= int(name) <= 2100
25
+
26
+
27
+ def backfill_slugs(db: Session) -> dict:
28
+ """Assign slugs to zones/folders missing them."""
29
+ zone_slugs: Set[str] = set()
30
+ for zone in db.query(DdaZone).filter(DdaZone.slug.isnot(None)).all():
31
+ if zone.slug:
32
+ zone_slugs.add(zone.slug)
33
+
34
+ zones_updated = 0
35
+ for zone in db.query(DdaZone).order_by(DdaZone.id).all():
36
+ if zone.slug:
37
+ continue
38
+ zone.slug = unique_slug(zone.name, zone_slugs)
39
+ zone_slugs.add(zone.slug)
40
+ zones_updated += 1
41
+
42
+ folders_updated = 0
43
+ for zone in db.query(DdaZone).all():
44
+ folder_slugs: Set[str] = set()
45
+ for folder in db.query(DdaVillage).filter(
46
+ DdaVillage.zone_id == zone.id, DdaVillage.slug.isnot(None)
47
+ ).all():
48
+ if folder.slug:
49
+ folder_slugs.add(folder.slug)
50
+ for folder in db.query(DdaVillage).filter(DdaVillage.zone_id == zone.id).order_by(DdaVillage.id).all():
51
+ if folder.slug:
52
+ continue
53
+ folder.slug = unique_slug(folder.name, folder_slugs)
54
+ folder_slugs.add(folder.slug)
55
+ folders_updated += 1
56
+
57
+ if zones_updated or folders_updated:
58
+ db.commit()
59
+ return {"zonesUpdated": zones_updated, "foldersUpdated": folders_updated}
60
+
61
+
62
+ def ensure_legacy_zone(db: Session) -> DdaZone:
63
+ zone = db.query(DdaZone).filter(DdaZone.slug == LEGACY_ZONE_SLUG).first()
64
+ if not zone:
65
+ zone = db.query(DdaZone).filter(DdaZone.name == LEGACY_ZONE_NAME).first()
66
+ if not zone:
67
+ zone = DdaZone(name=LEGACY_ZONE_NAME, slug=LEGACY_ZONE_SLUG, mode="admin")
68
+ db.add(zone)
69
+ db.flush()
70
+ elif not zone.slug:
71
+ zone.slug = LEGACY_ZONE_SLUG
72
+ folder = db.query(DdaVillage).filter(
73
+ DdaVillage.zone_id == zone.id, DdaVillage.slug == LEGACY_FOLDER_SLUG
74
+ ).first()
75
+ if not folder:
76
+ folder = db.query(DdaVillage).filter(
77
+ DdaVillage.zone_id == zone.id, DdaVillage.name == LEGACY_FOLDER_NAME
78
+ ).first()
79
+ if not folder:
80
+ folder = DdaVillage(zone_id=zone.id, name=LEGACY_FOLDER_NAME, slug=LEGACY_FOLDER_SLUG)
81
+ db.add(folder)
82
+ db.flush()
83
+ elif not folder.slug:
84
+ folder.slug = LEGACY_FOLDER_SLUG
85
+ db.commit()
86
+ return zone
87
+
88
+
89
+ def migrate_legacy_flat_years(db: Session) -> dict:
90
+ """Move top-level library_sources/YEAR/* into _unassigned/legacy/YEAR/."""
91
+ ensure_legacy_zone(db)
92
+ moved = 0
93
+ roots_touched = 0
94
+
95
+ for root in get_library_roots():
96
+ if not root.exists():
97
+ continue
98
+ root_changed = False
99
+ for entry in sorted(root.iterdir()):
100
+ if not entry.is_dir() or not _is_year_dir(entry.name):
101
+ continue
102
+ year = entry.name
103
+ dest_dir = root / LEGACY_ZONE_SLUG / LEGACY_FOLDER_SLUG / year
104
+ dest_dir.mkdir(parents=True, exist_ok=True)
105
+ for path in sorted(entry.iterdir()):
106
+ if not path.is_file():
107
+ continue
108
+ if path.suffix.lower() not in ALLOWED_EXTENSIONS:
109
+ continue
110
+ target = dest_dir / path.name
111
+ if target.exists():
112
+ stem, suffix = path.stem, path.suffix
113
+ n = 1
114
+ while target.exists():
115
+ target = dest_dir / f"{stem}_{n}{suffix}"
116
+ n += 1
117
+ shutil.move(str(path), str(target))
118
+ old_rel = f"{year}/{path.name}"
119
+ new_rel = f"{LEGACY_ZONE_SLUG}/{LEGACY_FOLDER_SLUG}/{year}/{target.name}"
120
+ idx = db.query(DdaLocalFileIndex).filter(DdaLocalFileIndex.relative_path == old_rel).first()
121
+ if idx:
122
+ idx.relative_path = new_rel
123
+ moved += 1
124
+ root_changed = True
125
+ try:
126
+ if entry.is_dir() and not any(entry.iterdir()):
127
+ entry.rmdir()
128
+ except OSError:
129
+ pass
130
+ if root_changed:
131
+ roots_touched += 1
132
+
133
+ if moved:
134
+ db.commit()
135
+ logger.info("Legacy library migration: moved %d file(s) across %d root(s)", moved, roots_touched)
136
+ return {"moved": moved, "rootsTouched": roots_touched}
137
+
138
+
139
+ def ensure_zone_folder_dirs(db: Session) -> None:
140
+ """Create on-disk folders for all zones/folders."""
141
+ root = get_writable_library_root()
142
+ root.mkdir(parents=True, exist_ok=True)
143
+ for zone in db.query(DdaZone).filter(DdaZone.slug.isnot(None)).all():
144
+ zone_dir = root / zone.slug
145
+ zone_dir.mkdir(parents=True, exist_ok=True)
146
+ for folder in db.query(DdaVillage).filter(DdaVillage.zone_id == zone.id).all():
147
+ if folder.slug:
148
+ (zone_dir / folder.slug).mkdir(parents=True, exist_ok=True)
app/dda/library_routes.py CHANGED
@@ -86,34 +86,8 @@ def dda_config():
86
  @router.get("/hierarchy")
87
  def get_hierarchy(db: Session = Depends(get_db)):
88
  _require_dda()
89
- zones = db.query(DdaZone).order_by(DdaZone.name).all()
90
- tree = []
91
- for zone in zones:
92
- villages = (
93
- db.query(DdaVillage)
94
- .filter(DdaVillage.zone_id == zone.id)
95
- .order_by(DdaVillage.name)
96
- .all()
97
- )
98
- image_counts = {}
99
- for v in villages:
100
- cnt = db.query(ImageAsset).filter(ImageAsset.village_id == v.id).count()
101
- if cnt:
102
- image_counts[v.id] = cnt
103
- tree.append({
104
- "id": zone.id,
105
- "name": zone.name,
106
- "mode": zone.mode,
107
- "villages": [
108
- {
109
- "id": v.id,
110
- "name": v.name,
111
- "imageCount": image_counts.get(v.id, 0),
112
- }
113
- for v in villages
114
- ],
115
- })
116
- return {"zones": tree}
117
 
118
 
119
  @router.get("/images")
 
86
  @router.get("/hierarchy")
87
  def get_hierarchy(db: Session = Depends(get_db)):
88
  _require_dda()
89
+ from .local_library import scan_tree
90
+ return scan_tree(db)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
91
 
92
 
93
  @router.get("/images")
app/dda/local_library.py CHANGED
@@ -1,67 +1,169 @@
1
  """
2
- Read satellite/drone images from local year-based folders.
3
 
4
- Folder layout (under library_sources/):
5
 
6
  library_sources/
7
- 2024/
8
- image_a.tif
9
- 2025/
10
- image_c.tif
 
 
 
 
11
 
12
- Copy files into library_sources/YEAR/ in the project folder (or data/library_sources/ on HF).
13
  """
14
  from __future__ import annotations
15
 
16
  import hashlib
17
  import logging
18
- from dataclasses import dataclass
19
  from pathlib import Path
20
- from typing import List, Optional
21
  from urllib.parse import quote
22
 
23
  from fastapi import HTTPException
 
24
 
25
  from .config import ALLOWED_EXTENSIONS, LOCAL_THUMB_CACHE, get_library_roots
26
  from .geotiff_io import inspect_image, raster_to_preview_png, write_placeholder_png
 
 
27
 
28
  logger = logging.getLogger(__name__)
29
 
30
 
 
 
 
 
 
 
 
 
31
  @dataclass
32
  class LocalImageEntry:
33
- path: str # relative posix path, e.g. 2025/aerial.tif
34
  root: Path
35
  year: int
36
  filename: str
37
  file_size_bytes: int
 
 
 
 
 
38
 
39
 
40
  def _is_year_dir(name: str) -> bool:
41
  return len(name) == 4 and name.isdigit() and 1990 <= int(name) <= 2100
42
 
43
 
44
- def _iter_image_files(root: Path, year: Optional[int] = None):
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
  if not root.exists():
46
  return
47
- if year is not None:
48
- year_dirs = [root / str(year)] if (root / str(year)).is_dir() else []
49
- else:
50
- year_dirs = [d for d in sorted(root.iterdir()) if d.is_dir() and _is_year_dir(d.name)]
51
- for ydir in year_dirs:
52
- y = int(ydir.name)
53
- for path in sorted(ydir.rglob("*")):
54
- if not path.is_file():
 
55
  continue
56
- ext = path.suffix.lower()
57
- if ext not in ALLOWED_EXTENSIONS:
58
- logger.debug("Skipped (extension %s): %s", ext, path)
 
 
 
 
 
 
 
 
 
 
 
 
59
  continue
60
- yield root, y, path
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
61
 
62
 
63
  def safe_resolve(relative_path: str) -> Path:
64
- """Resolve a library-relative path across all configured roots."""
65
  rel = relative_path.replace("\\", "/").lstrip("/")
66
  if not rel or ".." in rel.split("/"):
67
  raise HTTPException(status_code=400, detail="Invalid image path")
@@ -76,35 +178,35 @@ def safe_resolve(relative_path: str) -> Path:
76
  raise HTTPException(status_code=404, detail="Image file not found")
77
 
78
 
79
- def scan_years() -> List[dict]:
80
- """List year folders and image counts (merged across all roots)."""
81
- ensure_root()
82
- counts: dict[int, int] = {}
83
- for root in get_library_roots():
84
- if not root.exists():
85
- continue
86
- for entry in sorted(root.iterdir()):
87
- if not entry.is_dir() or not _is_year_dir(entry.name):
88
- continue
89
- y = int(entry.name)
90
- n = sum(1 for _ in _iter_image_files(root, year=y))
91
- counts[y] = counts.get(y, 0) + n
92
- return [{"year": y, "imageCount": counts[y]} for y in sorted(counts)]
93
-
94
-
95
- def scan_images(year: Optional[int] = None, query: Optional[str] = None) -> List[LocalImageEntry]:
96
- """Scan all library roots for images."""
97
  ensure_root()
 
98
  results: List[LocalImageEntry] = []
99
  seen_paths: set[str] = set()
100
  q = (query or "").strip().lower()
101
 
102
  for root in get_library_roots():
103
- for r, y, path in _iter_image_files(root, year=year):
104
- rel = path.relative_to(r).as_posix()
105
  if rel in seen_paths:
106
  continue
107
- if q and q not in rel.lower():
 
 
 
 
 
 
 
 
 
 
108
  continue
109
  try:
110
  size = path.stat().st_size
@@ -118,13 +220,109 @@ def scan_images(year: Optional[int] = None, query: Optional[str] = None) -> List
118
  year=y,
119
  filename=path.name,
120
  file_size_bytes=size,
 
 
 
 
 
121
  )
122
  )
123
  return results
124
 
125
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
126
  def entry_to_dict(entry: LocalImageEntry, include_meta: bool = False) -> dict:
127
  encoded_path = quote(entry.path, safe="/")
 
 
 
128
  out = {
129
  "path": entry.path,
130
  "year": entry.year,
@@ -133,6 +331,12 @@ def entry_to_dict(entry: LocalImageEntry, include_meta: bool = False) -> dict:
133
  "thumbUrl": f"/api/dda/local/thumb?path={encoded_path}",
134
  "source": "local_folder",
135
  "rootPath": str(entry.root),
 
 
 
 
 
 
136
  }
137
  if include_meta:
138
  try:
@@ -175,28 +379,42 @@ def ensure_root() -> None:
175
  LOCAL_THUMB_CACHE.mkdir(parents=True, exist_ok=True)
176
 
177
 
178
- def library_debug_info() -> dict:
179
- """Diagnostics for troubleshooting missing images."""
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
180
  roots_info = []
181
  for root in get_library_roots():
182
- info = {
183
- "path": str(root),
184
- "exists": root.exists(),
185
- "years": [],
186
- "otherFiles": [],
187
- }
188
  if root.exists():
189
  for entry in sorted(root.iterdir()):
190
  if entry.is_dir() and _is_year_dir(entry.name):
191
- files = []
192
- for _, _, p in _iter_image_files(root, year=int(entry.name)):
193
- files.append({"name": p.name, "size": p.stat().st_size})
194
- info["years"].append({"year": entry.name, "files": files})
195
- elif entry.is_file() and entry.name not in ("README.md", ".gitkeep"):
196
- info["otherFiles"].append(entry.name)
197
  roots_info.append(info)
198
  return {
199
  "roots": roots_info,
200
  "allowedExtensions": sorted(ALLOWED_EXTENSIONS),
201
- "totalImages": len(scan_images()),
202
  }
 
1
  """
2
+ Read satellite/drone images from zone/folder/year hierarchy under library_sources/.
3
 
4
+ Canonical layout:
5
 
6
  library_sources/
7
+ central_delhi/
8
+ site_a/
9
+ 2024/
10
+ image_a.tif
11
+ _unassigned/
12
+ legacy/
13
+ 2025/
14
+ old_image.tif
15
 
16
+ Legacy flat library_sources/YEAR/ is still scanned (tagged legacy=true) until migrated.
17
  """
18
  from __future__ import annotations
19
 
20
  import hashlib
21
  import logging
22
+ from dataclasses import dataclass, field
23
  from pathlib import Path
24
+ from typing import Dict, List, Optional, Tuple
25
  from urllib.parse import quote
26
 
27
  from fastapi import HTTPException
28
+ from sqlalchemy.orm import Session
29
 
30
  from .config import ALLOWED_EXTENSIONS, LOCAL_THUMB_CACHE, get_library_roots
31
  from .geotiff_io import inspect_image, raster_to_preview_png, write_placeholder_png
32
+ from .library_migration import LEGACY_FOLDER_SLUG, LEGACY_ZONE_SLUG
33
+ from .models import DdaVillage, DdaZone
34
 
35
  logger = logging.getLogger(__name__)
36
 
37
 
38
+ @dataclass
39
+ class SlugLookup:
40
+ zone_by_slug: Dict[str, DdaZone] = field(default_factory=dict)
41
+ folder_by_zone_slug: Dict[str, Dict[str, DdaVillage]] = field(default_factory=dict)
42
+ zone_by_id: Dict[int, DdaZone] = field(default_factory=dict)
43
+ folder_by_id: Dict[int, DdaVillage] = field(default_factory=dict)
44
+
45
+
46
  @dataclass
47
  class LocalImageEntry:
48
+ path: str
49
  root: Path
50
  year: int
51
  filename: str
52
  file_size_bytes: int
53
+ zone_id: Optional[int] = None
54
+ zone_name: str = ""
55
+ folder_id: Optional[int] = None
56
+ folder_name: str = ""
57
+ legacy: bool = False
58
 
59
 
60
  def _is_year_dir(name: str) -> bool:
61
  return len(name) == 4 and name.isdigit() and 1990 <= int(name) <= 2100
62
 
63
 
64
+ def build_slug_lookup(db: Session) -> SlugLookup:
65
+ lookup = SlugLookup()
66
+ for zone in db.query(DdaZone).filter(DdaZone.slug.isnot(None)).all():
67
+ lookup.zone_by_slug[zone.slug] = zone
68
+ lookup.zone_by_id[zone.id] = zone
69
+ lookup.folder_by_zone_slug[zone.slug] = {}
70
+ for folder in db.query(DdaVillage).filter(DdaVillage.slug.isnot(None)).all():
71
+ zone = lookup.zone_by_id.get(folder.zone_id)
72
+ if zone and zone.slug:
73
+ lookup.folder_by_zone_slug.setdefault(zone.slug, {})[folder.slug] = folder
74
+ lookup.folder_by_id[folder.id] = folder
75
+ return lookup
76
+
77
+
78
+ def build_upload_dest(root: Path, zone_slug: str, folder_slug: str, year: int, filename: str) -> Path:
79
+ dest_dir = root / zone_slug / folder_slug / str(year)
80
+ dest_dir.mkdir(parents=True, exist_ok=True)
81
+ dest = dest_dir / filename
82
+ if dest.exists():
83
+ stem = Path(filename).stem
84
+ suffix = Path(filename).suffix
85
+ n = 1
86
+ while dest.exists():
87
+ dest = dest_dir / f"{stem}_{n}{suffix}"
88
+ n += 1
89
+ return dest
90
+
91
+
92
+ def parse_library_path(rel_path: str, lookup: SlugLookup) -> Tuple[Optional[int], Optional[int], bool]:
93
+ """Return (zone_id, folder_id, legacy) from relative path."""
94
+ parts = rel_path.replace("\\", "/").split("/")
95
+ if len(parts) >= 4 and _is_year_dir(parts[2]):
96
+ zone_slug, folder_slug = parts[0], parts[1]
97
+ zone = lookup.zone_by_slug.get(zone_slug)
98
+ folder = lookup.folder_by_zone_slug.get(zone_slug, {}).get(folder_slug)
99
+ if zone and folder:
100
+ return zone.id, folder.id, zone_slug == LEGACY_ZONE_SLUG
101
+ if len(parts) == 2 and _is_year_dir(parts[0]):
102
+ return None, None, True
103
+ return None, None, False
104
+
105
+
106
+ def _yield_files_in_year_dir(root: Path, year_dir: Path, rel_prefix: str, year: int,
107
+ zone_id: Optional[int], zone_name: str,
108
+ folder_id: Optional[int], folder_name: str, legacy: bool):
109
+ for path in sorted(year_dir.rglob("*")):
110
+ if not path.is_file():
111
+ continue
112
+ if path.suffix.lower() not in ALLOWED_EXTENSIONS:
113
+ continue
114
+ rel = f"{rel_prefix}/{path.relative_to(year_dir).as_posix()}".lstrip("/")
115
+ yield root, year, path, rel, zone_id, zone_name, folder_id, folder_name, legacy
116
+
117
+
118
+ def _iter_all_images(root: Path, lookup: SlugLookup, year_filter: Optional[int] = None):
119
  if not root.exists():
120
  return
121
+
122
+ for zone_dir in sorted(root.iterdir()):
123
+ if not zone_dir.is_dir():
124
+ continue
125
+
126
+ # Legacy flat: library_sources/2025/file.tif
127
+ if _is_year_dir(zone_dir.name):
128
+ y = int(zone_dir.name)
129
+ if year_filter is not None and y != year_filter:
130
  continue
131
+ prefix = zone_dir.name
132
+ for item in _yield_files_in_year_dir(
133
+ root, zone_dir, prefix, y, None, "Unassigned", None, "Legacy", True
134
+ ):
135
+ yield item
136
+ continue
137
+
138
+ zone_slug = zone_dir.name
139
+ zone = lookup.zone_by_slug.get(zone_slug)
140
+ zone_name = zone.name if zone else zone_slug
141
+ zone_id = zone.id if zone else None
142
+ folders_map = lookup.folder_by_zone_slug.get(zone_slug, {})
143
+
144
+ for folder_dir in sorted(zone_dir.iterdir()):
145
+ if not folder_dir.is_dir():
146
  continue
147
+ folder_slug = folder_dir.name
148
+ folder = folders_map.get(folder_slug)
149
+ folder_name = folder.name if folder else folder_slug
150
+ folder_id = folder.id if folder else None
151
+ is_legacy = zone_slug == LEGACY_ZONE_SLUG
152
+
153
+ for year_dir in sorted(folder_dir.iterdir()):
154
+ if not year_dir.is_dir() or not _is_year_dir(year_dir.name):
155
+ continue
156
+ y = int(year_dir.name)
157
+ if year_filter is not None and y != year_filter:
158
+ continue
159
+ prefix = f"{zone_slug}/{folder_slug}/{year_dir.name}"
160
+ for item in _yield_files_in_year_dir(
161
+ root, year_dir, prefix, y, zone_id, zone_name, folder_id, folder_name, is_legacy
162
+ ):
163
+ yield item
164
 
165
 
166
  def safe_resolve(relative_path: str) -> Path:
 
167
  rel = relative_path.replace("\\", "/").lstrip("/")
168
  if not rel or ".." in rel.split("/"):
169
  raise HTTPException(status_code=400, detail="Invalid image path")
 
178
  raise HTTPException(status_code=404, detail="Image file not found")
179
 
180
 
181
+ def scan_images(
182
+ db: Optional[Session] = None,
183
+ year: Optional[int] = None,
184
+ zone_id: Optional[int] = None,
185
+ folder_id: Optional[int] = None,
186
+ query: Optional[str] = None,
187
+ legacy_only: Optional[bool] = None,
188
+ ) -> List[LocalImageEntry]:
 
 
 
 
 
 
 
 
 
 
189
  ensure_root()
190
+ lookup = build_slug_lookup(db) if db else SlugLookup()
191
  results: List[LocalImageEntry] = []
192
  seen_paths: set[str] = set()
193
  q = (query or "").strip().lower()
194
 
195
  for root in get_library_roots():
196
+ for (r, y, path, rel, zid, zname, fid, fname, legacy) in _iter_all_images(root, lookup, year_filter=year):
 
197
  if rel in seen_paths:
198
  continue
199
+ if zone_id is not None and zid != zone_id:
200
+ continue
201
+ if folder_id is not None and fid != folder_id:
202
+ continue
203
+ if legacy_only is True:
204
+ parts = rel.split("/")
205
+ if not (len(parts) == 2 and _is_year_dir(parts[0])):
206
+ continue
207
+ if legacy_only is False and legacy and len(rel.split("/")) == 2:
208
+ continue
209
+ if q and q not in rel.lower() and q not in path.name.lower():
210
  continue
211
  try:
212
  size = path.stat().st_size
 
220
  year=y,
221
  filename=path.name,
222
  file_size_bytes=size,
223
+ zone_id=zid,
224
+ zone_name=zname,
225
+ folder_id=fid,
226
+ folder_name=fname,
227
+ legacy=legacy,
228
  )
229
  )
230
  return results
231
 
232
 
233
+ def scan_years(db: Optional[Session] = None) -> List[dict]:
234
+ """Flat year list (backward compat for rescan endpoint)."""
235
+ counts: dict[int, int] = {}
236
+ for entry in scan_images(db=db):
237
+ counts[entry.year] = counts.get(entry.year, 0) + 1
238
+ return [{"year": y, "imageCount": counts[y]} for y in sorted(counts)]
239
+
240
+
241
+ def scan_tree(db: Session) -> dict:
242
+ """Nested zone → folder → year tree with image counts from disk."""
243
+ lookup = build_slug_lookup(db)
244
+ zone_nodes: dict[int, dict] = {}
245
+ legacy_years: dict[int, int] = {}
246
+
247
+ for entry in scan_images(db=db):
248
+ if entry.legacy and not entry.zone_id:
249
+ legacy_years[entry.year] = legacy_years.get(entry.year, 0) + 1
250
+ continue
251
+ if entry.zone_id is None:
252
+ legacy_years[entry.year] = legacy_years.get(entry.year, 0) + 1
253
+ continue
254
+
255
+ zone = lookup.zone_by_id.get(entry.zone_id)
256
+ if not zone:
257
+ continue
258
+ znode = zone_nodes.setdefault(entry.zone_id, {
259
+ "id": zone.id,
260
+ "name": zone.name,
261
+ "slug": zone.slug,
262
+ "folders": {},
263
+ })
264
+ fid = entry.folder_id or 0
265
+ folder = lookup.folder_by_id.get(fid) if entry.folder_id else None
266
+ fnode = znode["folders"].setdefault(fid, {
267
+ "id": entry.folder_id,
268
+ "name": entry.folder_name or (folder.name if folder else "Unknown"),
269
+ "slug": folder.slug if folder else "",
270
+ "years": {},
271
+ })
272
+ fnode["years"][entry.year] = fnode["years"].get(entry.year, 0) + 1
273
+
274
+ # Include empty zones/folders from DB
275
+ for zone in db.query(DdaZone).order_by(DdaZone.name).all():
276
+ if not zone.slug:
277
+ continue
278
+ znode = zone_nodes.setdefault(zone.id, {
279
+ "id": zone.id,
280
+ "name": zone.name,
281
+ "slug": zone.slug,
282
+ "folders": {},
283
+ })
284
+ for folder in db.query(DdaVillage).filter(DdaVillage.zone_id == zone.id).order_by(DdaVillage.name).all():
285
+ if not folder.slug:
286
+ continue
287
+ znode["folders"].setdefault(folder.id, {
288
+ "id": folder.id,
289
+ "name": folder.name,
290
+ "slug": folder.slug,
291
+ "years": {},
292
+ })
293
+
294
+ zones_out = []
295
+ for znode in sorted(zone_nodes.values(), key=lambda z: z["name"].lower()):
296
+ folders_out = []
297
+ for fnode in sorted(znode["folders"].values(), key=lambda f: f["name"].lower()):
298
+ years_out = [
299
+ {"year": y, "imageCount": cnt}
300
+ for y, cnt in sorted(fnode["years"].items())
301
+ ]
302
+ folders_out.append({
303
+ "id": fnode["id"],
304
+ "name": fnode["name"],
305
+ "slug": fnode["slug"],
306
+ "years": years_out,
307
+ })
308
+ zones_out.append({
309
+ "id": znode["id"],
310
+ "name": znode["name"],
311
+ "slug": znode["slug"],
312
+ "folders": folders_out,
313
+ })
314
+
315
+ return {
316
+ "zones": zones_out,
317
+ "legacyYears": [{"year": y, "imageCount": c} for y, c in sorted(legacy_years.items())],
318
+ }
319
+
320
+
321
  def entry_to_dict(entry: LocalImageEntry, include_meta: bool = False) -> dict:
322
  encoded_path = quote(entry.path, safe="/")
323
+ breadcrumb = " / ".join(
324
+ p for p in (entry.zone_name, entry.folder_name, str(entry.year), entry.filename) if p
325
+ )
326
  out = {
327
  "path": entry.path,
328
  "year": entry.year,
 
331
  "thumbUrl": f"/api/dda/local/thumb?path={encoded_path}",
332
  "source": "local_folder",
333
  "rootPath": str(entry.root),
334
+ "zoneId": entry.zone_id,
335
+ "zoneName": entry.zone_name,
336
+ "folderId": entry.folder_id,
337
+ "folderName": entry.folder_name,
338
+ "legacy": entry.legacy,
339
+ "breadcrumb": breadcrumb,
340
  }
341
  if include_meta:
342
  try:
 
379
  LOCAL_THUMB_CACHE.mkdir(parents=True, exist_ok=True)
380
 
381
 
382
+ def count_files_in_folder(root: Path, zone_slug: str, folder_slug: str) -> int:
383
+ folder_path = root / zone_slug / folder_slug
384
+ if not folder_path.exists():
385
+ return 0
386
+ count = 0
387
+ for path in folder_path.rglob("*"):
388
+ if path.is_file() and path.suffix.lower() in ALLOWED_EXTENSIONS:
389
+ count += 1
390
+ return count
391
+
392
+
393
+ def count_files_in_zone(root: Path, zone_slug: str) -> int:
394
+ zone_path = root / zone_slug
395
+ if not zone_path.exists():
396
+ return 0
397
+ count = 0
398
+ for path in zone_path.rglob("*"):
399
+ if path.is_file() and path.suffix.lower() in ALLOWED_EXTENSIONS:
400
+ count += 1
401
+ return count
402
+
403
+
404
+ def library_debug_info(db: Optional[Session] = None) -> dict:
405
  roots_info = []
406
  for root in get_library_roots():
407
+ info = {"path": str(root), "exists": root.exists(), "zones": [], "legacyYears": []}
 
 
 
 
 
408
  if root.exists():
409
  for entry in sorted(root.iterdir()):
410
  if entry.is_dir() and _is_year_dir(entry.name):
411
+ files = [p.name for p in entry.iterdir() if p.is_file()]
412
+ info["legacyYears"].append({"year": entry.name, "files": files})
413
+ elif entry.is_dir():
414
+ info["zones"].append(entry.name)
 
 
415
  roots_info.append(info)
416
  return {
417
  "roots": roots_info,
418
  "allowedExtensions": sorted(ALLOWED_EXTENSIONS),
419
+ "totalImages": len(scan_images(db=db)),
420
  }
app/dda/local_routes.py CHANGED
@@ -1,11 +1,13 @@
1
- """API for reading images from local library_sources/ year folders."""
2
  import logging
 
3
  from pathlib import Path
4
  from typing import Optional
5
 
6
  from fastapi import APIRouter, Depends, File, Form, HTTPException, Query, Request, UploadFile
7
  from fastapi.responses import FileResponse
8
  from PIL import Image
 
9
  from sqlalchemy.orm import Session
10
 
11
  from ..database import get_db
@@ -25,13 +27,16 @@ from .config import (
25
  max_upload_bytes_for_extension,
26
  )
27
  from .local_library import (
 
28
  entry_to_dict,
29
  get_or_build_thumb,
30
  library_debug_info,
31
  safe_resolve,
32
  scan_images,
 
33
  scan_years,
34
  )
 
35
  from .upload_io import stream_upload_to_file
36
 
37
  logger = logging.getLogger(__name__)
@@ -53,6 +58,31 @@ def _safe_basename(filename: str) -> str:
53
  return name
54
 
55
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
56
  @router.get("/local/config")
57
  def local_library_config():
58
  _require_dda()
@@ -61,13 +91,13 @@ def local_library_config():
61
  hosted = is_hf_hosted()
62
  if hosted:
63
  instructions = (
64
- "On Hugging Face, images must be uploaded below (saved to persistent storage) "
65
- "or copied into the writable folder shown. Files on your PC are not visible here."
66
  )
67
  else:
68
  instructions = (
69
- "Copy .tif images into library_sources/YEAR/ in your project folder, then Refresh. "
70
- "Or use Upload to save into data/library_sources/."
71
  )
72
  return {
73
  "source": "local_folder",
@@ -87,36 +117,49 @@ def local_library_config():
87
 
88
 
89
  @router.get("/local/debug")
90
- def local_debug():
91
  _require_dda()
92
- return library_debug_info()
93
 
94
 
95
  @router.get("/local/years")
96
- def local_years():
 
 
 
 
 
 
97
  _require_dda()
98
- return {"years": scan_years(), "rootPaths": [str(r) for r in get_library_roots()]}
99
 
100
 
101
  @router.get("/local/images")
102
  def local_images(
103
  year: Optional[int] = Query(None),
 
 
 
104
  q: Optional[str] = Query(None),
 
105
  ):
106
  _require_dda()
107
- entries = scan_images(year=year, query=q)
 
 
 
108
  return [entry_to_dict(e) for e in entries]
109
 
110
 
111
  @router.get("/local/images/detail")
112
- def local_image_detail(path: str = Query(..., description="Relative path e.g. 2025/aerial.tif")):
113
  _require_dda()
114
- entries = scan_images()
115
  norm = path.replace("\\", "/")
116
  match = next((e for e in entries if e.path == norm), None)
117
  if not match:
118
  safe_resolve(path)
119
- match = next((e for e in scan_images() if e.path == norm), None)
120
  if not match:
121
  raise HTTPException(status_code=404, detail="Image not found in library scan")
122
  return entry_to_dict(match, include_meta=True)
@@ -144,16 +187,25 @@ def local_thumb(path: str = Query(...)):
144
  async def local_upload(
145
  request: Request,
146
  file: UploadFile = File(...),
 
 
147
  year: int = Form(...),
148
  db: Session = Depends(get_db),
149
  user: User = Depends(current_dda_user),
150
  ):
151
- """Upload GeoTIFF into persistent library_sources/YEAR/ (required on HF)."""
152
  _require_dda()
153
  require_min_role(user, db, "uploader")
154
  if year < 1990 or year > 2100:
155
  raise HTTPException(status_code=400, detail="year must be between 1990 and 2100")
156
 
 
 
 
 
 
 
 
157
  original = _safe_basename(file.filename or "upload")
158
  ext = Path(original).suffix.lower()
159
  from .config import ALLOWED_EXTENSIONS
@@ -161,40 +213,79 @@ async def local_upload(
161
  raise HTTPException(status_code=400, detail=f"Allowed: {', '.join(sorted(ALLOWED_EXTENSIONS))}")
162
 
163
  root = get_writable_library_root()
164
- dest = root / str(year) / original
165
- if dest.exists():
166
- stem = Path(original).stem
167
- suffix = Path(original).suffix
168
- n = 1
169
- while dest.exists():
170
- dest = root / str(year) / f"{stem}_{n}{suffix}"
171
- n += 1
172
-
173
  size = await stream_upload_to_file(file, dest, max_upload_bytes_for_extension(ext))
174
  rel = dest.relative_to(root).as_posix()
175
  logger.info("Library upload: %s (%d bytes) -> %s", original, size, dest)
176
 
177
- entries = scan_images(year=year)
 
 
178
  match = next((e for e in entries if e.path == rel or e.filename == dest.name), None)
179
  if match:
180
  return {"status": "success", "path": match.path, "image": entry_to_dict(match)}
181
  return {
182
  "status": "success",
183
- "path": f"{year}/{dest.name}",
184
  "fileSizeBytes": size,
185
  "writablePath": str(dest),
186
  }
187
 
188
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
189
  @router.post("/local/rescan")
190
- def local_rescan():
191
  _require_dda()
192
- years = scan_years()
193
- total = sum(y["imageCount"] for y in years)
194
- info = library_debug_info()
 
195
  logger.info("Library rescan: %d images, roots=%s", total, info.get("roots"))
196
  return {
197
  "ok": True,
 
198
  "years": years,
199
  "totalImages": total,
200
  "rootPaths": [str(r) for r in get_library_roots()],
@@ -220,7 +311,7 @@ async def detect_from_library(
220
  db: Session = Depends(get_db),
221
  user: User = Depends(current_dda_user),
222
  ):
223
- """Run change detection on two library images by relative path (e.g. 2025/aerial.tif)."""
224
  _require_dda()
225
  base_norm = base_path.replace("\\", "/").strip()
226
  comp_norm = comparison_path.replace("\\", "/").strip()
@@ -245,7 +336,6 @@ async def detect_from_library(
245
  except Exception as exc:
246
  raise HTTPException(status_code=400, detail=f"Could not load images: {exc}") from exc
247
 
248
- # Match dimensions so registration and overlay align with the before image
249
  if before_pil.size != after_pil.size:
250
  after_pil = after_pil.resize(before_pil.size, Image.Resampling.LANCZOS)
251
 
 
1
+ """API for reading images from local library_sources/ zone/folder/year hierarchy."""
2
  import logging
3
+ import shutil
4
  from pathlib import Path
5
  from typing import Optional
6
 
7
  from fastapi import APIRouter, Depends, File, Form, HTTPException, Query, Request, UploadFile
8
  from fastapi.responses import FileResponse
9
  from PIL import Image
10
+ from pydantic import BaseModel, Field
11
  from sqlalchemy.orm import Session
12
 
13
  from ..database import get_db
 
27
  max_upload_bytes_for_extension,
28
  )
29
  from .local_library import (
30
+ build_upload_dest,
31
  entry_to_dict,
32
  get_or_build_thumb,
33
  library_debug_info,
34
  safe_resolve,
35
  scan_images,
36
+ scan_tree,
37
  scan_years,
38
  )
39
+ from .models import DdaLocalFileIndex, DdaVillage, DdaZone
40
  from .upload_io import stream_upload_to_file
41
 
42
  logger = logging.getLogger(__name__)
 
58
  return name
59
 
60
 
61
+ def _upsert_file_index(db: Session, rel_path: str, zone_id: int, folder_id: int, year: int, user_id: int):
62
+ idx = db.query(DdaLocalFileIndex).filter(DdaLocalFileIndex.relative_path == rel_path).first()
63
+ if idx:
64
+ idx.zone_id = zone_id
65
+ idx.folder_id = folder_id
66
+ idx.year = year
67
+ idx.uploaded_by = user_id
68
+ else:
69
+ db.add(DdaLocalFileIndex(
70
+ relative_path=rel_path,
71
+ zone_id=zone_id,
72
+ folder_id=folder_id,
73
+ year=year,
74
+ uploaded_by=user_id,
75
+ ))
76
+ db.commit()
77
+
78
+
79
+ class ReassignBody(BaseModel):
80
+ path: str
81
+ zone_id: int
82
+ folder_id: int
83
+ year: int = Field(..., ge=1990, le=2100)
84
+
85
+
86
  @router.get("/local/config")
87
  def local_library_config():
88
  _require_dda()
 
91
  hosted = is_hf_hosted()
92
  if hosted:
93
  instructions = (
94
+ "Upload images below with zone, folder, and year. Files are saved under "
95
+ "library_sources/{zone}/{folder}/{year}/ on persistent Space storage."
96
  )
97
  else:
98
  instructions = (
99
+ "Copy .tif images into library_sources/{zone}/{folder}/{year}/, or use Upload "
100
+ "with zone and folder selected. Click Refresh after adding files manually."
101
  )
102
  return {
103
  "source": "local_folder",
 
117
 
118
 
119
  @router.get("/local/debug")
120
+ def local_debug(db: Session = Depends(get_db)):
121
  _require_dda()
122
+ return library_debug_info(db=db)
123
 
124
 
125
  @router.get("/local/years")
126
+ def local_years(db: Session = Depends(get_db)):
127
+ _require_dda()
128
+ return {"years": scan_years(db=db), "rootPaths": [str(r) for r in get_library_roots()]}
129
+
130
+
131
+ @router.get("/local/tree")
132
+ def local_tree(db: Session = Depends(get_db)):
133
  _require_dda()
134
+ return scan_tree(db)
135
 
136
 
137
  @router.get("/local/images")
138
  def local_images(
139
  year: Optional[int] = Query(None),
140
+ zone_id: Optional[int] = Query(None),
141
+ folder_id: Optional[int] = Query(None),
142
+ legacy_only: Optional[bool] = Query(None),
143
  q: Optional[str] = Query(None),
144
+ db: Session = Depends(get_db),
145
  ):
146
  _require_dda()
147
+ entries = scan_images(
148
+ db=db, year=year, zone_id=zone_id, folder_id=folder_id,
149
+ query=q, legacy_only=legacy_only,
150
+ )
151
  return [entry_to_dict(e) for e in entries]
152
 
153
 
154
  @router.get("/local/images/detail")
155
+ def local_image_detail(path: str = Query(..., description="Relative path e.g. zone/folder/2025/aerial.tif"), db: Session = Depends(get_db)):
156
  _require_dda()
157
+ entries = scan_images(db=db)
158
  norm = path.replace("\\", "/")
159
  match = next((e for e in entries if e.path == norm), None)
160
  if not match:
161
  safe_resolve(path)
162
+ match = next((e for e in scan_images(db=db) if e.path == norm), None)
163
  if not match:
164
  raise HTTPException(status_code=404, detail="Image not found in library scan")
165
  return entry_to_dict(match, include_meta=True)
 
187
  async def local_upload(
188
  request: Request,
189
  file: UploadFile = File(...),
190
+ zone_id: int = Form(...),
191
+ folder_id: int = Form(...),
192
  year: int = Form(...),
193
  db: Session = Depends(get_db),
194
  user: User = Depends(current_dda_user),
195
  ):
196
+ """Upload GeoTIFF into library_sources/{zone}/{folder}/{year}/."""
197
  _require_dda()
198
  require_min_role(user, db, "uploader")
199
  if year < 1990 or year > 2100:
200
  raise HTTPException(status_code=400, detail="year must be between 1990 and 2100")
201
 
202
+ zone = db.query(DdaZone).filter(DdaZone.id == zone_id).first()
203
+ folder = db.query(DdaVillage).filter(
204
+ DdaVillage.id == folder_id, DdaVillage.zone_id == zone_id
205
+ ).first()
206
+ if not zone or not folder or not zone.slug or not folder.slug:
207
+ raise HTTPException(status_code=400, detail="Invalid zone or folder")
208
+
209
  original = _safe_basename(file.filename or "upload")
210
  ext = Path(original).suffix.lower()
211
  from .config import ALLOWED_EXTENSIONS
 
213
  raise HTTPException(status_code=400, detail=f"Allowed: {', '.join(sorted(ALLOWED_EXTENSIONS))}")
214
 
215
  root = get_writable_library_root()
216
+ dest = build_upload_dest(root, zone.slug, folder.slug, year, original)
 
 
 
 
 
 
 
 
217
  size = await stream_upload_to_file(file, dest, max_upload_bytes_for_extension(ext))
218
  rel = dest.relative_to(root).as_posix()
219
  logger.info("Library upload: %s (%d bytes) -> %s", original, size, dest)
220
 
221
+ _upsert_file_index(db, rel, zone.id, folder.id, year, user.id)
222
+
223
+ entries = scan_images(db=db, year=year, zone_id=zone_id, folder_id=folder_id)
224
  match = next((e for e in entries if e.path == rel or e.filename == dest.name), None)
225
  if match:
226
  return {"status": "success", "path": match.path, "image": entry_to_dict(match)}
227
  return {
228
  "status": "success",
229
+ "path": rel,
230
  "fileSizeBytes": size,
231
  "writablePath": str(dest),
232
  }
233
 
234
 
235
+ @router.post("/local/reassign")
236
+ def local_reassign(
237
+ body: ReassignBody,
238
+ db: Session = Depends(get_db),
239
+ user: User = Depends(current_dda_user),
240
+ ):
241
+ """Move a library file to zone/folder/year and update index."""
242
+ _require_dda()
243
+ require_min_role(user, db, "uploader")
244
+
245
+ zone = db.query(DdaZone).filter(DdaZone.id == body.zone_id).first()
246
+ folder = db.query(DdaVillage).filter(
247
+ DdaVillage.id == body.folder_id, DdaVillage.zone_id == body.zone_id
248
+ ).first()
249
+ if not zone or not folder or not zone.slug or not folder.slug:
250
+ raise HTTPException(status_code=400, detail="Invalid zone or folder")
251
+
252
+ src_rel = body.path.replace("\\", "/").strip()
253
+ src = safe_resolve(src_rel)
254
+ root = get_writable_library_root()
255
+ try:
256
+ src.relative_to(root.resolve())
257
+ except ValueError:
258
+ raise HTTPException(status_code=400, detail="Only files in writable library can be reassigned")
259
+
260
+ dest = build_upload_dest(root, zone.slug, folder.slug, body.year, src.name)
261
+ if dest.resolve() != src.resolve():
262
+ shutil.move(str(src), str(dest))
263
+ new_rel = dest.relative_to(root).as_posix()
264
+
265
+ old_idx = db.query(DdaLocalFileIndex).filter(DdaLocalFileIndex.relative_path == src_rel).first()
266
+ if old_idx:
267
+ db.delete(old_idx)
268
+ db.commit()
269
+ _upsert_file_index(db, new_rel, zone.id, folder.id, body.year, user.id)
270
+
271
+ entries = scan_images(db=db)
272
+ match = next((e for e in entries if e.path == new_rel), None)
273
+ if not match:
274
+ raise HTTPException(status_code=500, detail="File moved but not found in scan")
275
+ return {"status": "success", "path": new_rel, "image": entry_to_dict(match)}
276
+
277
+
278
  @router.post("/local/rescan")
279
+ def local_rescan(db: Session = Depends(get_db)):
280
  _require_dda()
281
+ tree = scan_tree(db)
282
+ total = len(scan_images(db=db))
283
+ years = scan_years(db=db)
284
+ info = library_debug_info(db=db)
285
  logger.info("Library rescan: %d images, roots=%s", total, info.get("roots"))
286
  return {
287
  "ok": True,
288
+ "tree": tree,
289
  "years": years,
290
  "totalImages": total,
291
  "rootPaths": [str(r) for r in get_library_roots()],
 
311
  db: Session = Depends(get_db),
312
  user: User = Depends(current_dda_user),
313
  ):
314
+ """Run change detection on two library images by relative path."""
315
  _require_dda()
316
  base_norm = base_path.replace("\\", "/").strip()
317
  comp_norm = comparison_path.replace("\\", "/").strip()
 
336
  except Exception as exc:
337
  raise HTTPException(status_code=400, detail=f"Could not load images: {exc}") from exc
338
 
 
339
  if before_pil.size != after_pil.size:
340
  after_pil = after_pil.resize(before_pil.size, Image.Resampling.LANCZOS)
341
 
app/dda/models.py CHANGED
@@ -16,6 +16,7 @@ class DdaZone(Base):
16
 
17
  id = Column(Integer, primary_key=True, index=True)
18
  name = Column(String(128), unique=True, nullable=False, index=True)
 
19
  mode = Column(String(32), default="admin") # admin | grid_parent
20
  created_at = Column(DateTime, default=_utcnow)
21
 
@@ -23,17 +24,35 @@ class DdaZone(Base):
23
 
24
 
25
  class DdaVillage(Base):
 
26
  __tablename__ = "dda_villages"
27
 
28
  id = Column(Integer, primary_key=True, index=True)
29
  zone_id = Column(Integer, ForeignKey("dda_zones.id"), nullable=False, index=True)
30
  name = Column(String(128), nullable=False, index=True)
 
31
  created_at = Column(DateTime, default=_utcnow)
32
 
33
  zone = relationship("DdaZone", back_populates="villages")
34
  images = relationship("ImageAsset", back_populates="village")
35
 
36
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
  class ImageAsset(Base):
38
  """Centralized satellite / drone image (FR-01, FR-02)."""
39
  __tablename__ = "dda_image_assets"
 
16
 
17
  id = Column(Integer, primary_key=True, index=True)
18
  name = Column(String(128), unique=True, nullable=False, index=True)
19
+ slug = Column(String(64), unique=True, nullable=True, index=True)
20
  mode = Column(String(32), default="admin") # admin | grid_parent
21
  created_at = Column(DateTime, default=_utcnow)
22
 
 
24
 
25
 
26
  class DdaVillage(Base):
27
+ """Folder under a zone (exposed as folder in API/UI)."""
28
  __tablename__ = "dda_villages"
29
 
30
  id = Column(Integer, primary_key=True, index=True)
31
  zone_id = Column(Integer, ForeignKey("dda_zones.id"), nullable=False, index=True)
32
  name = Column(String(128), nullable=False, index=True)
33
+ slug = Column(String(64), nullable=True, index=True)
34
  created_at = Column(DateTime, default=_utcnow)
35
 
36
  zone = relationship("DdaZone", back_populates="villages")
37
  images = relationship("ImageAsset", back_populates="village")
38
 
39
 
40
+ class DdaLocalFileIndex(Base):
41
+ """Maps library-relative paths to zone/folder/year for traceability."""
42
+ __tablename__ = "dda_local_file_index"
43
+
44
+ id = Column(Integer, primary_key=True, index=True)
45
+ relative_path = Column(String(512), unique=True, nullable=False, index=True)
46
+ zone_id = Column(Integer, ForeignKey("dda_zones.id"), nullable=True, index=True)
47
+ folder_id = Column(Integer, ForeignKey("dda_villages.id"), nullable=True, index=True)
48
+ year = Column(Integer, nullable=True, index=True)
49
+ uploaded_by = Column(Integer, ForeignKey("users.id"), nullable=True)
50
+ created_at = Column(DateTime, default=_utcnow)
51
+
52
+ zone = relationship("DdaZone")
53
+ folder = relationship("DdaVillage")
54
+
55
+
56
  class ImageAsset(Base):
57
  """Centralized satellite / drone image (FR-01, FR-02)."""
58
  __tablename__ = "dda_image_assets"
app/dda/path_slugs.py ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Filesystem-safe slugs for zone/folder library paths."""
2
+ from __future__ import annotations
3
+
4
+ import re
5
+ import unicodedata
6
+ from typing import Iterable, Optional, Set
7
+
8
+ RESERVED_SLUGS = frozenset({"_unassigned", "legacy", "thumbs", "cache"})
9
+
10
+
11
+ def slugify(name: str, *, fallback: str = "item") -> str:
12
+ """Convert display name to a safe path segment."""
13
+ text = unicodedata.normalize("NFKD", (name or "").strip())
14
+ text = text.encode("ascii", "ignore").decode("ascii")
15
+ text = re.sub(r"[^\w\s-]", "", text, flags=re.UNICODE)
16
+ text = re.sub(r"[\s_-]+", "_", text).strip("_").lower()
17
+ if not text:
18
+ text = fallback
19
+ if text in RESERVED_SLUGS:
20
+ text = f"{text}_1"
21
+ return text[:64]
22
+
23
+
24
+ def unique_slug(base: str, existing: Set[str]) -> str:
25
+ """Return base or base-2, base-3, … if base is taken."""
26
+ slug = slugify(base)
27
+ if slug not in existing:
28
+ return slug
29
+ n = 2
30
+ while True:
31
+ candidate = f"{slug}-{n}"[:64]
32
+ if candidate not in existing:
33
+ return candidate
34
+ n += 1
35
+
36
+
37
+ def validate_path_segment(segment: str) -> bool:
38
+ if not segment or segment in (".", ".."):
39
+ return False
40
+ if ".." in segment.split("/"):
41
+ return False
42
+ if not re.match(r"^[a-z0-9][a-z0-9_-]*$", segment, re.IGNORECASE):
43
+ return False
44
+ return True
45
+
46
+
47
+ def zone_folder_path(zone_slug: str, folder_slug: str, year: int) -> str:
48
+ return f"{zone_slug}/{folder_slug}/{year}"
app/dda/seed.py CHANGED
@@ -3,6 +3,7 @@ import logging
3
  from sqlalchemy.orm import Session
4
 
5
  from .models import DdaVillage, DdaZone
 
6
  from .seed_data import DELHI_ZONES
7
 
8
  logger = logging.getLogger(__name__)
@@ -12,17 +13,25 @@ def seed_delhi_hierarchy(db: Session) -> dict:
12
  """Insert Zone → Village hierarchy if empty. Idempotent."""
13
  existing = db.query(DdaZone).count()
14
  if existing > 0:
 
 
15
  return {"seeded": False, "zones": existing}
16
 
17
  zones_created = 0
18
  villages_created = 0
 
19
  for zone_name, villages in DELHI_ZONES.items():
20
- zone = DdaZone(name=zone_name, mode="admin")
 
 
21
  db.add(zone)
22
  db.flush()
23
  zones_created += 1
 
24
  for village_name in villages:
25
- db.add(DdaVillage(zone_id=zone.id, name=village_name))
 
 
26
  villages_created += 1
27
 
28
  db.commit()
 
3
  from sqlalchemy.orm import Session
4
 
5
  from .models import DdaVillage, DdaZone
6
+ from .path_slugs import unique_slug
7
  from .seed_data import DELHI_ZONES
8
 
9
  logger = logging.getLogger(__name__)
 
13
  """Insert Zone → Village hierarchy if empty. Idempotent."""
14
  existing = db.query(DdaZone).count()
15
  if existing > 0:
16
+ from .library_migration import backfill_slugs
17
+ backfill_slugs(db)
18
  return {"seeded": False, "zones": existing}
19
 
20
  zones_created = 0
21
  villages_created = 0
22
+ zone_slugs: set = set()
23
  for zone_name, villages in DELHI_ZONES.items():
24
+ slug = unique_slug(zone_name, zone_slugs)
25
+ zone_slugs.add(slug)
26
+ zone = DdaZone(name=zone_name, slug=slug, mode="admin")
27
  db.add(zone)
28
  db.flush()
29
  zones_created += 1
30
+ folder_slugs: set = set()
31
  for village_name in villages:
32
+ fslug = unique_slug(village_name, folder_slugs)
33
+ folder_slugs.add(fslug)
34
+ db.add(DdaVillage(zone_id=zone.id, name=village_name, slug=fslug))
35
  villages_created += 1
36
 
37
  db.commit()
app/main.py CHANGED
@@ -28,7 +28,7 @@ from .auth import (
28
  from .database import Base, engine, get_db, DATA_DIR
29
  from .models import User, DetectionRun
30
  from . import dda as _dda_pkg # noqa: F401 — register DDA tables
31
- from .dda.models import DdaZone, DdaVillage, ImageAsset, DetectionJob, RegionReview # noqa: F401
32
  from .dda.config import IS_DDA_MODE
33
  from .dda.bootstrap import init_dda_database, setup_dda
34
  from .notifier import send_notification, send_test_email
 
28
  from .database import Base, engine, get_db, DATA_DIR
29
  from .models import User, DetectionRun
30
  from . import dda as _dda_pkg # noqa: F401 — register DDA tables
31
+ from .dda.models import DdaZone, DdaVillage, ImageAsset, DetectionJob, RegionReview, DdaLocalFileIndex # noqa: F401
32
  from .dda.config import IS_DDA_MODE
33
  from .dda.bootstrap import init_dda_database, setup_dda
34
  from .notifier import send_notification, send_test_email
static/css/dda.css CHANGED
@@ -83,6 +83,43 @@
83
  background: var(--bg-hover);
84
  color: var(--grad-start);
85
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
86
 
87
  .dda-folder-path {
88
  font-size: 0.78rem;
 
83
  background: var(--bg-hover);
84
  color: var(--grad-start);
85
  }
86
+ .dda-sidebar-actions { display: flex; gap: 0.35rem; flex-wrap: wrap; }
87
+ .dda-tree-all {
88
+ display: block;
89
+ width: 100%;
90
+ text-align: left;
91
+ padding: 0.4rem 0.55rem;
92
+ margin-bottom: 0.35rem;
93
+ border: none;
94
+ background: transparent;
95
+ color: var(--text);
96
+ cursor: pointer;
97
+ border-radius: 6px;
98
+ font-weight: 600;
99
+ }
100
+ .dda-tree-all:hover, .dda-tree-all.active {
101
+ background: var(--bg-hover);
102
+ color: var(--grad-start);
103
+ }
104
+ .dda-tree-zone { margin-bottom: 0.25rem; }
105
+ .dda-tree-zone-summary, .dda-tree-folder-summary {
106
+ cursor: pointer;
107
+ font-weight: 600;
108
+ padding: 0.25rem 0;
109
+ color: var(--text);
110
+ }
111
+ .dda-tree-folders { padding-left: 0.5rem; }
112
+ .dda-tree-folder { margin: 0.15rem 0; }
113
+ .dda-tree-years { padding-left: 0.75rem; margin-bottom: 0.25rem; }
114
+ .dda-tree-empty { font-size: 0.8rem; display: block; padding: 0.2rem 0; }
115
+ .dda-tree-legacy .dda-tree-zone-summary { color: var(--text-muted); }
116
+ .dda-crumb { font-size: 0.78rem; line-height: 1.3; }
117
+ .dda-assign-btn { margin-top: 0.35rem; }
118
+ .dda-manage-panel { max-width: 520px; width: 100%; }
119
+ .dda-manage-section { margin-bottom: 1rem; }
120
+ .dda-manage-section h4 { margin: 0 0 0.5rem; font-size: 0.9rem; }
121
+ .dda-manage-section .location-row { margin-bottom: 0.5rem; flex-wrap: wrap; gap: 0.4rem; }
122
+ .dda-manage-section input, .dda-manage-section select { flex: 1 1 auto; min-width: 120px; }
123
 
124
  .dda-folder-path {
125
  font-size: 0.78rem;
static/js/dda/app.js CHANGED
@@ -55,15 +55,32 @@ function formatBytes(n) {
55
  }
56
 
57
  let ddaConfig = null;
58
- let localYears = [];
59
- let selectedYear = null;
60
 
61
  window.ddaState = {
62
  get config() { return ddaConfig; },
63
  get localCfg() { return window._localCfg; },
64
- get years() { return localYears; },
65
- get selectedYear() { return selectedYear; },
66
- setYear(year) { selectedYear = year; },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
67
  refreshImages: () => loadLibraryImages(),
68
  rescan: () => rescanLibrary(),
69
  };
@@ -83,8 +100,8 @@ document.querySelectorAll('.dda-tab').forEach((btn) => {
83
 
84
  async function rescanLibrary() {
85
  const data = await ddaApi('POST', '/api/dda/local/rescan');
86
- localYears = data.years || [];
87
- if (typeof renderYearTree === 'function') renderYearTree(localYears);
88
  await loadLibraryImages();
89
  return data;
90
  }
@@ -96,6 +113,13 @@ async function initDda() {
96
  const localCfg = await ddaApi('GET', '/api/dda/local/config');
97
  window._localCfg = localCfg;
98
 
 
 
 
 
 
 
 
99
  const hint = document.getElementById('lib-config-hint');
100
  if (hint) {
101
  hint.textContent = localCfg.geotiffEnabled ? 'GeoTIFF ready' : 'GeoTIFF limited';
@@ -106,6 +130,7 @@ async function initDda() {
106
  pathEl.textContent = [
107
  localCfg.isHosted ? 'HF writable storage:' : 'Local folders:',
108
  localCfg.writablePath || paths[0] || '',
 
109
  ...paths.filter((p) => p !== localCfg.writablePath),
110
  ].filter(Boolean).join('\n');
111
  }
@@ -118,8 +143,17 @@ async function initDda() {
118
  const instr = document.getElementById('lib-instructions');
119
  if (instr && localCfg.instructions) instr.textContent = localCfg.instructions;
120
 
121
- const hfUpload = document.getElementById('hf-upload-card');
122
- if (hfUpload) hfUpload.classList.toggle('hidden', !localCfg.isHosted);
 
 
 
 
 
 
 
 
 
123
 
124
  const resHint = document.getElementById('dda-detect-res-hint');
125
  if (resHint && localCfg.detectionMaxSide) {
@@ -128,7 +162,7 @@ async function initDda() {
128
 
129
  const uploadLimit = document.getElementById('hf-upload-limit');
130
  if (uploadLimit && localCfg.maxUploadGb) {
131
- uploadLimit.textContent = `Files on your computer are not on the server. Upload .tif images here (up to ${localCfg.maxUploadGb} GB each).`;
132
  }
133
 
134
  const appMode = ddaConfig.appMode || ddaConfig.mode || localCfg.appMode || 'dda';
@@ -141,76 +175,61 @@ async function initDda() {
141
  document.querySelector(`.dda-tab[data-tab="${urlTab}"]`)?.click();
142
  }
143
 
144
- const yearsData = await ddaApi('GET', '/api/dda/local/years');
145
- localYears = yearsData.years || [];
146
- if (typeof renderYearTree === 'function') renderYearTree(localYears);
147
  await loadLibraryImages();
148
- await loadHierarchyTree();
149
  } catch (err) {
150
  showDdaError(err.message || 'Failed to load library');
151
  }
152
  }
153
 
154
- async function loadHierarchyTree() {
155
- const el = document.getElementById('lib-hierarchy');
156
- if (!el) return;
157
- try {
158
- const data = await ddaApi('GET', '/api/dda/hierarchy');
159
- const zones = data.zones || [];
160
- if (!zones.length) {
161
- el.innerHTML = '<p class="dim">No zones seeded.</p>';
162
- return;
163
- }
164
- el.innerHTML = `<p class="dim" style="margin-bottom:0.5rem">DB hierarchy (upload API). Local library uses year folders.</p>` +
165
- zones.map((z) => {
166
- const villages = z.villages || [];
167
- const zoneCount = villages.reduce((s, v) => s + (v.imageCount || 0), 0);
168
- const villageItems = villages.map((v) =>
169
- `<li class="dda-hierarchy-village">${v.name}${v.imageCount ? ` <span class="dim">(${v.imageCount})</span>` : ''}</li>`
170
- ).join('');
171
- return `
172
- <details class="dda-hierarchy-zone" open>
173
- <summary>${z.name}${zoneCount ? ` <span class="dim">(${zoneCount})</span>` : ''}</summary>
174
- <ul class="dda-hierarchy-list">${villageItems || '<li class="dim">No villages</li>'}</ul>
175
- </details>`;
176
- }).join('');
177
- } catch (_) {
178
- el.innerHTML = '<p class="dim">Zone tree unavailable.</p>';
179
- }
180
  }
181
 
182
- window.loadHierarchyTree = loadHierarchyTree;
183
-
184
  async function loadLibraryImages() {
185
  const grid = document.getElementById('lib-grid');
186
  const title = document.getElementById('lib-grid-title');
187
  if (!grid) return;
188
  const q = document.getElementById('lib-filter')?.value?.trim() || '';
 
189
  const params = new URLSearchParams();
190
- if (selectedYear) params.set('year', String(selectedYear));
191
- if (q) params.set('q', q);
192
- if (title) {
193
- title.textContent = selectedYear ? `Images — ${selectedYear}` : 'Images — all years';
 
 
 
194
  }
 
 
195
  try {
196
  const items = await ddaApi('GET', '/api/dda/local/images?' + params.toString());
197
  window.ddaState.libraryItems = items;
198
  if (!items.length) {
199
- const hf = window.ddaState?.localCfg?.isHosted;
200
- grid.innerHTML = hf
201
- ? `<p class="dim">No images on this Space yet. Use <strong>Upload to Space storage</strong> above (2025 / 2026), then click Refresh.</p>`
202
- : `<p class="dim">No images in ${selectedYear || 'library_sources'}. Copy .tif files into <code>library_sources/${selectedYear || 'YEAR'}/</code> and click Refresh.</p>`;
203
  return;
204
  }
205
  grid.innerHTML = items.map((img) => {
206
  const thumb = img.thumbUrl ? img.thumbUrl.replace(/path=[^&]+/, 'path=' + encodeURIComponent(img.path)) : '';
 
 
 
 
207
  return `
208
  <div class="dda-card-img" draggable="true" data-image-path="${img.path.replace(/"/g, '&quot;')}" title="${escapeHtml(img.filename)}">
209
  ${thumb ? `<img src="${thumb}" alt="" loading="lazy" />` : '<div class="meta">No preview</div>'}
210
  <div class="meta">
211
- <strong>${escapeHtml(String(img.year))}</strong><br/>
212
- ${escapeHtml(img.filename)}<br/>
213
  <span class="dim">${formatBytes(img.fileSizeBytes)}</span>
 
214
  </div>
215
  </div>`;
216
  }).join('');
@@ -220,6 +239,14 @@ async function loadLibraryImages() {
220
  e.dataTransfer.setData('text/plain', card.dataset.imagePath);
221
  });
222
  });
 
 
 
 
 
 
 
 
223
  if (typeof loadCompareLibraryGrid === 'function') loadCompareLibraryGrid();
224
  } catch (err) {
225
  grid.innerHTML = `<p class="dim">Could not load images: ${err.message}</p>`;
 
55
  }
56
 
57
  let ddaConfig = null;
58
+ let libraryTree = null;
59
+ const selection = { zoneId: null, zoneName: '', folderId: null, folderName: '', year: null, legacy: false };
60
 
61
  window.ddaState = {
62
  get config() { return ddaConfig; },
63
  get localCfg() { return window._localCfg; },
64
+ get tree() { return libraryTree; },
65
+ get selection() { return { ...selection }; },
66
+ get userRole() { return window._ddaUserRole || 'analyst'; },
67
+ set userRole(r) { window._ddaUserRole = r; },
68
+ setSelection(s) {
69
+ selection.zoneId = s.zoneId ?? null;
70
+ selection.zoneName = s.zoneName || '';
71
+ selection.folderId = s.folderId ?? null;
72
+ selection.folderName = s.folderName || '';
73
+ selection.year = s.year ?? null;
74
+ selection.legacy = !!s.legacy;
75
+ },
76
+ clearSelection() {
77
+ selection.zoneId = null;
78
+ selection.zoneName = '';
79
+ selection.folderId = null;
80
+ selection.folderName = '';
81
+ selection.year = null;
82
+ selection.legacy = false;
83
+ },
84
  refreshImages: () => loadLibraryImages(),
85
  rescan: () => rescanLibrary(),
86
  };
 
100
 
101
  async function rescanLibrary() {
102
  const data = await ddaApi('POST', '/api/dda/local/rescan');
103
+ libraryTree = data.tree || null;
104
+ if (typeof renderLibraryTree === 'function' && libraryTree) renderLibraryTree(libraryTree);
105
  await loadLibraryImages();
106
  return data;
107
  }
 
113
  const localCfg = await ddaApi('GET', '/api/dda/local/config');
114
  window._localCfg = localCfg;
115
 
116
+ try {
117
+ const me = await ddaApi('GET', '/api/dda/me');
118
+ window.ddaState.userRole = me.role || 'analyst';
119
+ } catch (_) {
120
+ window.ddaState.userRole = 'analyst';
121
+ }
122
+
123
  const hint = document.getElementById('lib-config-hint');
124
  if (hint) {
125
  hint.textContent = localCfg.geotiffEnabled ? 'GeoTIFF ready' : 'GeoTIFF limited';
 
130
  pathEl.textContent = [
131
  localCfg.isHosted ? 'HF writable storage:' : 'Local folders:',
132
  localCfg.writablePath || paths[0] || '',
133
+ 'Layout: {zone}/{folder}/{year}/image.tif',
134
  ...paths.filter((p) => p !== localCfg.writablePath),
135
  ].filter(Boolean).join('\n');
136
  }
 
143
  const instr = document.getElementById('lib-instructions');
144
  if (instr && localCfg.instructions) instr.textContent = localCfg.instructions;
145
 
146
+ const uploadTitle = document.getElementById('upload-card-title');
147
+ const uploadHint = document.getElementById('upload-card-hint');
148
+ if (uploadTitle) {
149
+ uploadTitle.textContent = localCfg.isHosted ? 'Upload to Space storage' : 'Upload to library';
150
+ }
151
+ if (uploadHint) {
152
+ uploadHint.textContent = localCfg.isHosted ? 'Required on Hugging Face' : 'Optional — or copy files manually';
153
+ }
154
+
155
+ const manageBtn = document.getElementById('btn-manage-library');
156
+ if (manageBtn) manageBtn.classList.toggle('hidden', window.ddaState.userRole !== 'admin');
157
 
158
  const resHint = document.getElementById('dda-detect-res-hint');
159
  if (resHint && localCfg.detectionMaxSide) {
 
162
 
163
  const uploadLimit = document.getElementById('hf-upload-limit');
164
  if (uploadLimit && localCfg.maxUploadGb) {
165
+ uploadLimit.textContent = `Select zone, folder, and year. Upload .tif images (up to ${localCfg.maxUploadGb} GB each).`;
166
  }
167
 
168
  const appMode = ddaConfig.appMode || ddaConfig.mode || localCfg.appMode || 'dda';
 
175
  document.querySelector(`.dda-tab[data-tab="${urlTab}"]`)?.click();
176
  }
177
 
178
+ libraryTree = await (typeof loadLibraryTree === 'function' ? loadLibraryTree() : null);
179
+ if (typeof populateUploadZones === 'function') await populateUploadZones();
 
180
  await loadLibraryImages();
 
181
  } catch (err) {
182
  showDdaError(err.message || 'Failed to load library');
183
  }
184
  }
185
 
186
+ function selectionTitle() {
187
+ const s = window.ddaState.selection;
188
+ if (s.legacy && s.year) return `Unassigned — ${s.year}`;
189
+ const parts = [];
190
+ if (s.zoneName) parts.push(s.zoneName);
191
+ if (s.folderName) parts.push(s.folderName);
192
+ if (s.year) parts.push(String(s.year));
193
+ return parts.length ? parts.join(' / ') : 'All images';
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
194
  }
195
 
 
 
196
  async function loadLibraryImages() {
197
  const grid = document.getElementById('lib-grid');
198
  const title = document.getElementById('lib-grid-title');
199
  if (!grid) return;
200
  const q = document.getElementById('lib-filter')?.value?.trim() || '';
201
+ const s = window.ddaState.selection;
202
  const params = new URLSearchParams();
203
+ if (s.legacy && s.year) {
204
+ params.set('year', String(s.year));
205
+ params.set('legacy_only', 'true');
206
+ } else {
207
+ if (s.zoneId) params.set('zone_id', String(s.zoneId));
208
+ if (s.folderId) params.set('folder_id', String(s.folderId));
209
+ if (s.year) params.set('year', String(s.year));
210
  }
211
+ if (q) params.set('q', q);
212
+ if (title) title.textContent = `Images — ${selectionTitle()}`;
213
  try {
214
  const items = await ddaApi('GET', '/api/dda/local/images?' + params.toString());
215
  window.ddaState.libraryItems = items;
216
  if (!items.length) {
217
+ grid.innerHTML = `<p class="dim">No images in ${escapeHtml(selectionTitle())}. Upload or copy files into <code>library_sources/zone/folder/year/</code>, then Refresh.</p>`;
 
 
 
218
  return;
219
  }
220
  grid.innerHTML = items.map((img) => {
221
  const thumb = img.thumbUrl ? img.thumbUrl.replace(/path=[^&]+/, 'path=' + encodeURIComponent(img.path)) : '';
222
+ const crumb = img.breadcrumb || `${img.year} / ${img.filename}`;
223
+ const assignBtn = img.legacy
224
+ ? `<button type="button" class="btn btn-secondary btn-sm dda-assign-btn" data-path="${img.path.replace(/"/g, '&quot;')}" data-filename="${escapeHtml(img.filename)}">Assign location</button>`
225
+ : '';
226
  return `
227
  <div class="dda-card-img" draggable="true" data-image-path="${img.path.replace(/"/g, '&quot;')}" title="${escapeHtml(img.filename)}">
228
  ${thumb ? `<img src="${thumb}" alt="" loading="lazy" />` : '<div class="meta">No preview</div>'}
229
  <div class="meta">
230
+ <span class="dim dda-crumb">${escapeHtml(crumb)}</span><br/>
 
231
  <span class="dim">${formatBytes(img.fileSizeBytes)}</span>
232
+ ${assignBtn}
233
  </div>
234
  </div>`;
235
  }).join('');
 
239
  e.dataTransfer.setData('text/plain', card.dataset.imagePath);
240
  });
241
  });
242
+ grid.querySelectorAll('.dda-assign-btn').forEach((btn) => {
243
+ btn.addEventListener('click', (e) => {
244
+ e.stopPropagation();
245
+ if (typeof openReassignModal === 'function') {
246
+ openReassignModal(btn.dataset.path, btn.dataset.filename);
247
+ }
248
+ });
249
+ });
250
  if (typeof loadCompareLibraryGrid === 'function') loadCompareLibraryGrid();
251
  } catch (err) {
252
  grid.innerHTML = `<p class="dim">Could not load images: ${err.message}</p>`;
static/js/dda/hierarchy.js ADDED
@@ -0,0 +1,359 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /** Zone → folder → year library tree and admin management. */
2
+
3
+ let libraryTreeData = null;
4
+
5
+ function isAdminRole() {
6
+ const role = window.ddaState?.userRole || '';
7
+ return role === 'admin';
8
+ }
9
+
10
+ function canUploadRole() {
11
+ const rank = { viewer: 0, uploader: 1, analyst: 2, admin: 3 };
12
+ const role = window.ddaState?.userRole || 'analyst';
13
+ return (rank[role] || 0) >= rank.uploader;
14
+ }
15
+
16
+ function selectionLabel() {
17
+ const s = window.ddaState?.selection || {};
18
+ const parts = [];
19
+ if (s.zoneName) parts.push(s.zoneName);
20
+ if (s.folderName) parts.push(s.folderName);
21
+ if (s.year) parts.push(String(s.year));
22
+ return parts.length ? parts.join(' / ') : 'All images';
23
+ }
24
+
25
+ function renderLibraryTree(tree) {
26
+ const el = document.getElementById('lib-tree');
27
+ if (!el) return;
28
+ libraryTreeData = tree;
29
+ const filter = (document.getElementById('lib-tree-search')?.value || '').toLowerCase();
30
+ const sel = window.ddaState?.selection || {};
31
+
32
+ const allActive = !sel.zoneId && !sel.legacy && !sel.year;
33
+ const allBtn = `
34
+ <button type="button" class="dda-tree-node dda-tree-all ${allActive ? 'active' : ''}">
35
+ All images
36
+ </button>`;
37
+
38
+ let html = allBtn;
39
+
40
+ const zones = (tree?.zones || []).filter((z) => {
41
+ if (!filter) return true;
42
+ const zMatch = z.name.toLowerCase().includes(filter);
43
+ const fMatch = (z.folders || []).some((f) =>
44
+ f.name.toLowerCase().includes(filter) ||
45
+ (f.years || []).some((y) => String(y.year).includes(filter))
46
+ );
47
+ return zMatch || fMatch;
48
+ });
49
+
50
+ zones.forEach((zone) => {
51
+ const zoneOpen = sel.zoneId === zone.id || !filter;
52
+ html += `<details class="dda-tree-zone" ${zoneOpen ? 'open' : ''} data-zone-id="${zone.id}">`;
53
+ html += `<summary class="dda-tree-zone-summary">${escapeHtml(zone.name)}</summary>`;
54
+ html += '<div class="dda-tree-folders">';
55
+
56
+ (zone.folders || []).forEach((folder) => {
57
+ if (filter && !zone.name.toLowerCase().includes(filter) &&
58
+ !folder.name.toLowerCase().includes(filter) &&
59
+ !(folder.years || []).some((y) => String(y.year).includes(filter))) {
60
+ return;
61
+ }
62
+ const folderOpen = sel.folderId === folder.id || filter;
63
+ html += `<details class="dda-tree-folder" ${folderOpen ? 'open' : ''} data-folder-id="${folder.id}">`;
64
+ html += `<summary class="dda-tree-folder-summary">${escapeHtml(folder.name)}</summary>`;
65
+ html += '<div class="dda-tree-years">';
66
+
67
+ const years = folder.years || [];
68
+ if (!years.length) {
69
+ html += '<span class="dim dda-tree-empty">No year folders yet</span>';
70
+ }
71
+ years.forEach((y) => {
72
+ if (filter && !String(y.year).includes(filter) &&
73
+ !folder.name.toLowerCase().includes(filter) &&
74
+ !zone.name.toLowerCase().includes(filter)) return;
75
+ const active = sel.zoneId === zone.id && sel.folderId === folder.id && sel.year === y.year;
76
+ html += `<button type="button" class="dda-tree-year ${active ? 'active' : ''}"
77
+ data-zone-id="${zone.id}" data-folder-id="${folder.id}" data-year="${y.year}">
78
+ ${y.year} <span class="dim">(${y.imageCount})</span>
79
+ </button>`;
80
+ });
81
+ html += '</div></details>';
82
+ });
83
+
84
+ html += '</div></details>';
85
+ });
86
+
87
+ const legacy = tree?.legacyYears || [];
88
+ if (legacy.length) {
89
+ html += '<details class="dda-tree-zone dda-tree-legacy" open>';
90
+ html += '<summary class="dda-tree-zone-summary">Unassigned (legacy)</summary><div class="dda-tree-years">';
91
+ legacy.forEach((y) => {
92
+ const active = sel.legacy && sel.year === y.year;
93
+ html += `<button type="button" class="dda-tree-year dda-tree-legacy-year ${active ? 'active' : ''}"
94
+ data-legacy="1" data-year="${y.year}">
95
+ ${y.year} <span class="dim">(${y.imageCount})</span>
96
+ </button>`;
97
+ });
98
+ html += '</div></details>';
99
+ }
100
+
101
+ if (!zones.length && !legacy.length) {
102
+ html += '<p class="dim">No zones yet. Admins can add zones under Manage library.</p>';
103
+ }
104
+
105
+ el.innerHTML = html;
106
+
107
+ el.querySelector('.dda-tree-all')?.addEventListener('click', () => {
108
+ window.ddaState.clearSelection();
109
+ window.ddaState.refreshImages();
110
+ renderLibraryTree(libraryTreeData);
111
+ });
112
+
113
+ el.querySelectorAll('.dda-tree-year').forEach((btn) => {
114
+ btn.addEventListener('click', () => {
115
+ if (btn.dataset.legacy) {
116
+ window.ddaState.setSelection({ legacy: true, year: parseInt(btn.dataset.year, 10) });
117
+ } else {
118
+ const zoneId = parseInt(btn.dataset.zoneId, 10);
119
+ const folderId = parseInt(btn.dataset.folderId, 10);
120
+ const year = parseInt(btn.dataset.year, 10);
121
+ let zoneName = '';
122
+ let folderName = '';
123
+ for (const z of (libraryTreeData?.zones || [])) {
124
+ if (z.id === zoneId) {
125
+ zoneName = z.name;
126
+ const f = (z.folders || []).find((x) => x.id === folderId);
127
+ if (f) folderName = f.name;
128
+ break;
129
+ }
130
+ }
131
+ window.ddaState.setSelection({
132
+ zoneId, zoneName, folderId, folderName, year, legacy: false,
133
+ });
134
+ }
135
+ if (typeof syncUploadPickers === 'function') syncUploadPickers();
136
+ window.ddaState.refreshImages();
137
+ renderLibraryTree(libraryTreeData);
138
+ });
139
+ });
140
+ }
141
+
142
+ document.getElementById('lib-tree-search')?.addEventListener('input', () => {
143
+ if (libraryTreeData) renderLibraryTree(libraryTreeData);
144
+ });
145
+
146
+ async function loadLibraryTree() {
147
+ try {
148
+ const tree = await ddaApi('GET', '/api/dda/hierarchy/tree');
149
+ renderLibraryTree(tree);
150
+ return tree;
151
+ } catch (_) {
152
+ const el = document.getElementById('lib-tree');
153
+ if (el) el.innerHTML = '<p class="dim">Could not load library tree.</p>';
154
+ return null;
155
+ }
156
+ }
157
+
158
+ window.loadLibraryTree = loadLibraryTree;
159
+ window.renderLibraryTree = renderLibraryTree;
160
+
161
+ /* ---- Manage library modal ---- */
162
+
163
+ function openManageModal() {
164
+ document.getElementById('dda-manage-modal')?.classList.remove('hidden');
165
+ populateManageZones();
166
+ }
167
+
168
+ function closeManageModal() {
169
+ document.getElementById('dda-manage-modal')?.classList.add('hidden');
170
+ }
171
+
172
+ async function populateManageZones() {
173
+ const sel = document.getElementById('manage-zone-select');
174
+ if (!sel) return;
175
+ try {
176
+ const tree = await ddaApi('GET', '/api/dda/hierarchy/tree');
177
+ libraryTreeData = tree;
178
+ sel.innerHTML = (tree.zones || []).map((z) =>
179
+ `<option value="${z.id}">${escapeHtml(z.name)}</option>`
180
+ ).join('');
181
+ populateManageFolders();
182
+ } catch (err) {
183
+ showDdaError?.(err.message);
184
+ }
185
+ }
186
+
187
+ function populateManageFolders() {
188
+ const zoneId = parseInt(document.getElementById('manage-zone-select')?.value || '0', 10);
189
+ const folderSel = document.getElementById('manage-folder-select');
190
+ if (!folderSel || !libraryTreeData) return;
191
+ const zone = (libraryTreeData.zones || []).find((z) => z.id === zoneId);
192
+ folderSel.innerHTML = (zone?.folders || []).map((f) =>
193
+ `<option value="${f.id}">${escapeHtml(f.name)}</option>`
194
+ ).join('');
195
+ }
196
+
197
+ document.getElementById('btn-manage-library')?.addEventListener('click', openManageModal);
198
+ document.getElementById('dda-manage-close')?.addEventListener('click', closeManageModal);
199
+ document.getElementById('manage-zone-select')?.addEventListener('change', populateManageFolders);
200
+
201
+ document.getElementById('btn-add-zone')?.addEventListener('click', async () => {
202
+ const name = document.getElementById('manage-zone-name')?.value?.trim();
203
+ if (!name) return showDdaError?.('Enter a zone name.');
204
+ try {
205
+ await ddaApi('POST', '/api/dda/hierarchy/zones', { body: JSON.stringify({ name }) });
206
+ document.getElementById('manage-zone-name').value = '';
207
+ showDdaSuccess?.('Zone created.');
208
+ await loadLibraryTree();
209
+ await populateManageZones();
210
+ if (typeof populateUploadZones === 'function') populateUploadZones();
211
+ } catch (err) {
212
+ showDdaError?.(err.message);
213
+ }
214
+ });
215
+
216
+ document.getElementById('btn-rename-zone')?.addEventListener('click', async () => {
217
+ const zoneId = parseInt(document.getElementById('manage-zone-select')?.value || '0', 10);
218
+ const name = document.getElementById('manage-zone-rename')?.value?.trim();
219
+ if (!zoneId || !name) return showDdaError?.('Select a zone and enter a new name.');
220
+ try {
221
+ await ddaApi('PATCH', `/api/dda/hierarchy/zones/${zoneId}`, { body: JSON.stringify({ name }) });
222
+ document.getElementById('manage-zone-rename').value = '';
223
+ showDdaSuccess?.('Zone renamed.');
224
+ await loadLibraryTree();
225
+ await populateManageZones();
226
+ if (typeof populateUploadZones === 'function') populateUploadZones();
227
+ } catch (err) {
228
+ showDdaError?.(err.message);
229
+ }
230
+ });
231
+
232
+ document.getElementById('btn-add-folder')?.addEventListener('click', async () => {
233
+ const zoneId = parseInt(document.getElementById('manage-zone-select')?.value || '0', 10);
234
+ const name = document.getElementById('manage-folder-name')?.value?.trim();
235
+ if (!zoneId || !name) return showDdaError?.('Select a zone and enter a folder name.');
236
+ try {
237
+ await ddaApi('POST', `/api/dda/hierarchy/zones/${zoneId}/folders`, { body: JSON.stringify({ name }) });
238
+ document.getElementById('manage-folder-name').value = '';
239
+ showDdaSuccess?.('Folder created.');
240
+ await loadLibraryTree();
241
+ await populateManageZones();
242
+ if (typeof populateUploadZones === 'function') populateUploadZones();
243
+ } catch (err) {
244
+ showDdaError?.(err.message);
245
+ }
246
+ });
247
+
248
+ document.getElementById('btn-rename-folder')?.addEventListener('click', async () => {
249
+ const folderId = parseInt(document.getElementById('manage-folder-select')?.value || '0', 10);
250
+ const name = document.getElementById('manage-folder-rename')?.value?.trim();
251
+ if (!folderId || !name) return showDdaError?.('Select a folder and enter a new name.');
252
+ try {
253
+ await ddaApi('PATCH', `/api/dda/hierarchy/folders/${folderId}`, { body: JSON.stringify({ name }) });
254
+ document.getElementById('manage-folder-rename').value = '';
255
+ showDdaSuccess?.('Folder renamed.');
256
+ await loadLibraryTree();
257
+ await populateManageZones();
258
+ if (typeof populateUploadZones === 'function') populateUploadZones();
259
+ } catch (err) {
260
+ showDdaError?.(err.message);
261
+ }
262
+ });
263
+
264
+ document.getElementById('btn-create-year')?.addEventListener('click', async () => {
265
+ const folderId = parseInt(document.getElementById('manage-folder-select')?.value || '0', 10);
266
+ const year = parseInt(document.getElementById('manage-year')?.value || '0', 10);
267
+ if (!folderId || !year) return showDdaError?.('Select a folder and enter a year.');
268
+ try {
269
+ await ddaApi('POST', `/api/dda/hierarchy/folders/${folderId}/years`, { body: JSON.stringify({ year }) });
270
+ showDdaSuccess?.(`Year folder ${year} created.`);
271
+ await loadLibraryTree();
272
+ } catch (err) {
273
+ showDdaError?.(err.message);
274
+ }
275
+ });
276
+
277
+ document.getElementById('btn-delete-folder')?.addEventListener('click', async () => {
278
+ const folderId = parseInt(document.getElementById('manage-folder-select')?.value || '0', 10);
279
+ if (!folderId || !confirm('Delete this empty folder?')) return;
280
+ try {
281
+ await ddaApi('DELETE', `/api/dda/hierarchy/folders/${folderId}`);
282
+ showDdaSuccess?.('Folder deleted.');
283
+ await loadLibraryTree();
284
+ await populateManageZones();
285
+ } catch (err) {
286
+ showDdaError?.(err.message);
287
+ }
288
+ });
289
+
290
+ document.getElementById('btn-delete-zone')?.addEventListener('click', async () => {
291
+ const zoneId = parseInt(document.getElementById('manage-zone-select')?.value || '0', 10);
292
+ if (!zoneId || !confirm('Delete this empty zone and all empty folders?')) return;
293
+ try {
294
+ await ddaApi('DELETE', `/api/dda/hierarchy/zones/${zoneId}`);
295
+ showDdaSuccess?.('Zone deleted.');
296
+ await loadLibraryTree();
297
+ await populateManageZones();
298
+ } catch (err) {
299
+ showDdaError?.(err.message);
300
+ }
301
+ });
302
+
303
+ /* ---- Reassign legacy images ---- */
304
+
305
+ function openReassignModal(imagePath, filename) {
306
+ const modal = document.getElementById('dda-reassign-modal');
307
+ if (!modal) return;
308
+ modal.dataset.path = imagePath;
309
+ document.getElementById('reassign-filename').textContent = filename || imagePath;
310
+ populateReassignPickers();
311
+ modal.classList.remove('hidden');
312
+ }
313
+
314
+ function closeReassignModal() {
315
+ document.getElementById('dda-reassign-modal')?.classList.add('hidden');
316
+ }
317
+
318
+ async function populateReassignPickers() {
319
+ const zoneSel = document.getElementById('reassign-zone');
320
+ const folderSel = document.getElementById('reassign-folder');
321
+ if (!zoneSel) return;
322
+ try {
323
+ const tree = await ddaApi('GET', '/api/dda/hierarchy/tree');
324
+ zoneSel.innerHTML = (tree.zones || []).filter((z) => z.slug !== '_unassigned').map((z) =>
325
+ `<option value="${z.id}">${escapeHtml(z.name)}</option>`
326
+ ).join('');
327
+ const syncFolders = () => {
328
+ const zoneId = parseInt(zoneSel.value, 10);
329
+ const zone = (tree.zones || []).find((z) => z.id === zoneId);
330
+ folderSel.innerHTML = (zone?.folders || []).map((f) =>
331
+ `<option value="${f.id}">${escapeHtml(f.name)}</option>`
332
+ ).join('');
333
+ };
334
+ zoneSel.onchange = syncFolders;
335
+ syncFolders();
336
+ } catch (_) {}
337
+ }
338
+
339
+ document.getElementById('dda-reassign-close')?.addEventListener('click', closeReassignModal);
340
+ document.getElementById('btn-reassign-submit')?.addEventListener('click', async () => {
341
+ const modal = document.getElementById('dda-reassign-modal');
342
+ const path = modal?.dataset.path;
343
+ const zoneId = parseInt(document.getElementById('reassign-zone')?.value || '0', 10);
344
+ const folderId = parseInt(document.getElementById('reassign-folder')?.value || '0', 10);
345
+ const year = parseInt(document.getElementById('reassign-year')?.value || '0', 10);
346
+ if (!path || !zoneId || !folderId || !year) return;
347
+ try {
348
+ await ddaApi('POST', '/api/dda/local/reassign', {
349
+ body: JSON.stringify({ path, zone_id: zoneId, folder_id: folderId, year }),
350
+ });
351
+ showDdaSuccess?.('Image reassigned.');
352
+ closeReassignModal();
353
+ await window.ddaState.rescan();
354
+ } catch (err) {
355
+ showDdaError?.(err.message);
356
+ }
357
+ });
358
+
359
+ window.openReassignModal = openReassignModal;
static/js/dda/library.js CHANGED
@@ -1,37 +1,106 @@
1
- function renderYearTree(years) {
2
- const tree = document.getElementById('lib-tree');
3
- if (!tree) return;
4
- const filter = (document.getElementById('lib-tree-search')?.value || '').toLowerCase();
5
 
6
- const allBtn = `
7
- <button type="button" class="dda-tree-year ${window.ddaState.selectedYear === null ? 'active' : ''}" data-year="">
8
- All years
9
- </button>`;
 
 
 
 
 
 
 
 
 
 
10
 
11
- const yearBtns = (years || [])
12
- .filter((y) => !filter || String(y.year).includes(filter))
13
- .map((y) => `
14
- <button type="button" class="dda-tree-year ${window.ddaState.selectedYear === y.year ? 'active' : ''}" data-year="${y.year}">
15
- ${y.year} <span class="dim">(${y.imageCount})</span>
16
- </button>`).join('');
 
 
 
 
17
 
18
- tree.innerHTML = allBtn + yearBtns;
 
 
 
 
 
 
 
 
 
 
 
 
19
 
20
- tree.querySelectorAll('.dda-tree-year').forEach((btn) => {
21
- btn.addEventListener('click', () => {
22
- tree.querySelectorAll('.dda-tree-year').forEach((b) => b.classList.remove('active'));
23
- btn.classList.add('active');
24
- const raw = btn.dataset.year;
25
- window.ddaState.setYear(raw ? parseInt(raw, 10) : null);
26
- window.ddaState.refreshImages();
27
- });
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  });
29
  }
30
 
31
- document.getElementById('lib-tree-search')?.addEventListener('input', () => {
32
- if (window.ddaState?.years) renderYearTree(window.ddaState.years);
33
- });
34
-
35
  function uploadWithProgress(url, formData, onProgress) {
36
  return new Promise((resolve, reject) => {
37
  const xhr = new XMLHttpRequest();
@@ -51,55 +120,7 @@ function uploadWithProgress(url, formData, onProgress) {
51
  });
52
  }
53
 
54
- function formatBytes(n) {
55
- if (n >= 1024 ** 3) return (n / 1024 ** 3).toFixed(1) + ' GB';
56
- if (n >= 1024 ** 2) return (n / 1024 ** 2).toFixed(1) + ' MB';
57
- return (n / 1024).toFixed(0) + ' KB';
58
- }
59
-
60
- document.getElementById('form-hf-upload')?.addEventListener('submit', async (e) => {
61
- e.preventDefault();
62
- hideDdaError?.();
63
- const fileInput = document.getElementById('hf-file');
64
- const file = fileInput?.files?.[0];
65
- if (!file) {
66
- showDdaError?.('Select a .tif file.');
67
- return;
68
- }
69
-
70
- const maxBytes = window.ddaState?.localCfg?.maxGeotiffBytes
71
- || (window.ddaState?.localCfg?.maxGeotiffMb || 5120) * 1024 * 1024;
72
- if (file.size > maxBytes) {
73
- showDdaError?.(`File is ${formatBytes(file.size)} — maximum upload size is ${formatBytes(maxBytes)}.`);
74
- return;
75
- }
76
-
77
- const form = new FormData();
78
- form.append('file', file);
79
- form.append('year', document.getElementById('hf-year').value);
80
-
81
- const btn = document.getElementById('btn-hf-upload');
82
- const progWrap = document.getElementById('hf-upload-progress');
83
- const progFill = document.getElementById('hf-upload-progress-fill');
84
- const progLabel = document.getElementById('hf-upload-progress-label');
85
-
86
- btn.disabled = true;
87
- progWrap?.classList.remove('hidden');
88
- if (progFill) progFill.style.width = '0%';
89
 
90
- try {
91
- await uploadWithProgress('/api/dda/local/upload', form, (loaded, total) => {
92
- const pct = total ? Math.round((loaded / total) * 100) : 0;
93
- if (progFill) progFill.style.width = pct + '%';
94
- if (progLabel) progLabel.textContent = `Uploading… ${pct}% (${formatBytes(loaded)} / ${formatBytes(total)})`;
95
- });
96
- showDdaSuccess?.('Uploaded to Space library. Click Refresh if images do not appear.');
97
- fileInput.value = '';
98
- await window.ddaState.rescan();
99
- } catch (err) {
100
- showDdaError?.(err.message || 'Upload failed. Large files may exceed HF timeout — try a smaller file or run locally.');
101
- } finally {
102
- btn.disabled = false;
103
- setTimeout(() => progWrap?.classList.add('hidden'), 2000);
104
- }
105
- });
 
1
+ let uploadTreeCache = null;
 
 
 
2
 
3
+ async function populateUploadZones() {
4
+ const zoneSel = document.getElementById('upload-zone');
5
+ if (!zoneSel) return;
6
+ try {
7
+ const tree = await ddaApi('GET', '/api/dda/hierarchy/tree');
8
+ uploadTreeCache = tree;
9
+ zoneSel.innerHTML = '<option value="">— Select zone —</option>' +
10
+ (tree.zones || []).filter((z) => z.slug !== '_unassigned').map((z) =>
11
+ `<option value="${z.id}">${escapeHtml(z.name)}</option>`
12
+ ).join('');
13
+ populateUploadFolders();
14
+ syncUploadPickers();
15
+ } catch (_) {}
16
+ }
17
 
18
+ function populateUploadFolders() {
19
+ const zoneId = parseInt(document.getElementById('upload-zone')?.value || '0', 10);
20
+ const folderSel = document.getElementById('upload-folder');
21
+ if (!folderSel || !uploadTreeCache) return;
22
+ const zone = (uploadTreeCache.zones || []).find((z) => z.id === zoneId);
23
+ folderSel.innerHTML = '<option value="">— Select folder —</option>' +
24
+ (zone?.folders || []).map((f) =>
25
+ `<option value="${f.id}">${escapeHtml(f.name)}</option>`
26
+ ).join('');
27
+ }
28
 
29
+ function syncUploadPickers() {
30
+ const s = window.ddaState?.selection;
31
+ if (!s) return;
32
+ const zoneSel = document.getElementById('upload-zone');
33
+ const folderSel = document.getElementById('upload-folder');
34
+ const yearInput = document.getElementById('upload-year');
35
+ if (s.zoneId && zoneSel) {
36
+ zoneSel.value = String(s.zoneId);
37
+ populateUploadFolders();
38
+ }
39
+ if (s.folderId && folderSel) folderSel.value = String(s.folderId);
40
+ if (s.year && yearInput) yearInput.value = String(s.year);
41
+ }
42
 
43
+ document.getElementById('upload-zone')?.addEventListener('change', populateUploadFolders);
44
+
45
+ function bindUploadForm(formId, fileId, btnId) {
46
+ document.getElementById(formId)?.addEventListener('submit', async (e) => {
47
+ e.preventDefault();
48
+ hideDdaError?.();
49
+ const fileInput = document.getElementById(fileId);
50
+ const file = fileInput?.files?.[0];
51
+ if (!file) {
52
+ showDdaError?.('Select a .tif file.');
53
+ return;
54
+ }
55
+
56
+ const zoneId = document.getElementById('upload-zone')?.value;
57
+ const folderId = document.getElementById('upload-folder')?.value;
58
+ const year = document.getElementById('upload-year')?.value;
59
+ if (!zoneId || !folderId || !year) {
60
+ showDdaError?.('Select zone, folder, and year.');
61
+ return;
62
+ }
63
+
64
+ const maxBytes = window.ddaState?.localCfg?.maxGeotiffBytes
65
+ || (window.ddaState?.localCfg?.maxGeotiffMb || 5120) * 1024 * 1024;
66
+ if (file.size > maxBytes) {
67
+ showDdaError?.(`File is ${formatBytes(file.size)} — maximum upload size is ${formatBytes(maxBytes)}.`);
68
+ return;
69
+ }
70
+
71
+ const form = new FormData();
72
+ form.append('file', file);
73
+ form.append('zone_id', zoneId);
74
+ form.append('folder_id', folderId);
75
+ form.append('year', year);
76
+
77
+ const btn = document.getElementById(btnId);
78
+ const progWrap = document.getElementById('upload-progress');
79
+ const progFill = document.getElementById('upload-progress-fill');
80
+ const progLabel = document.getElementById('upload-progress-label');
81
+
82
+ btn.disabled = true;
83
+ progWrap?.classList.remove('hidden');
84
+ if (progFill) progFill.style.width = '0%';
85
+
86
+ try {
87
+ await uploadWithProgress('/api/dda/local/upload', form, (loaded, total) => {
88
+ const pct = total ? Math.round((loaded / total) * 100) : 0;
89
+ if (progFill) progFill.style.width = pct + '%';
90
+ if (progLabel) progLabel.textContent = `Uploading… ${pct}% (${formatBytes(loaded)} / ${formatBytes(total)})`;
91
+ });
92
+ showDdaSuccess?.('Uploaded to library. Refreshing…');
93
+ fileInput.value = '';
94
+ await window.ddaState.rescan();
95
+ } catch (err) {
96
+ showDdaError?.(err.message || 'Upload failed.');
97
+ } finally {
98
+ btn.disabled = false;
99
+ setTimeout(() => progWrap?.classList.add('hidden'), 2000);
100
+ }
101
  });
102
  }
103
 
 
 
 
 
104
  function uploadWithProgress(url, formData, onProgress) {
105
  return new Promise((resolve, reject) => {
106
  const xhr = new XMLHttpRequest();
 
120
  });
121
  }
122
 
123
+ bindUploadForm('form-hf-upload', 'hf-file', 'btn-hf-upload');
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
124
 
125
+ window.populateUploadZones = populateUploadZones;
126
+ window.syncUploadPickers = syncUploadPickers;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
templates/index_dda.html CHANGED
@@ -5,7 +5,7 @@
5
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
6
  <title>DDA Change Detection</title>
7
  <link rel="stylesheet" href="/static/css/style.css?v=30" />
8
- <link rel="stylesheet" href="/static/css/dda.css?v=11" />
9
  </head>
10
  <body>
11
  <div class="app dda-app">
@@ -32,53 +32,60 @@
32
  <div id="dda-error" class="alert alert-error hidden"></div>
33
  <div id="dda-success" class="alert alert-success hidden"></div>
34
 
35
- <!-- Tab: Image Library — reads from library_sources/ year folders -->
36
  <section id="tab-library" class="dda-panel active" role="tabpanel">
37
  <div class="dda-layout">
38
  <aside class="dda-sidebar card">
39
  <div class="card-header">
40
- <h3>Years</h3>
41
- <button type="button" class="btn btn-secondary btn-sm" id="btn-refresh-lib" title="Rescan folder">Refresh</button>
 
 
 
42
  </div>
43
  <div id="lib-folder-path" class="dda-folder-path dim"></div>
44
- <input type="search" id="lib-tree-search" class="dda-search" placeholder="Filter years…" />
45
  <div id="lib-tree" class="dda-tree"><p class="dim">Loading…</p></div>
46
- <div class="dda-hierarchy-wrap">
47
- <h4 class="dda-hierarchy-title">DDA zones</h4>
48
- <div id="lib-hierarchy" class="dda-hierarchy"><p class="dim">Loading…</p></div>
49
- </div>
50
  </aside>
51
  <main class="dda-main">
52
  <div class="card dda-instructions">
53
  <div class="card-header">
54
- <h3>Local folder library</h3>
55
  <span class="dim" id="lib-config-hint"></span>
56
  </div>
57
  <p class="sub" id="lib-instructions">
58
- Copy <code>.tif</code> images into <code>library_sources/YEAR/</code> in your project folder, then click <strong>Refresh</strong>.
59
  </p>
60
  <pre id="lib-path-display" class="dda-path-code"></pre>
61
  </div>
62
- <div class="card hidden" id="hf-upload-card">
63
  <div class="card-header">
64
- <h3>Upload to Space storage</h3>
65
- <span class="dim">Required on Hugging Face</span>
66
  </div>
67
- <p class="sub" id="hf-upload-limit">Files on your computer are not on the server. Upload .tif images here (up to 5 GB each).</p>
68
  <form id="form-hf-upload" class="dda-upload-form">
69
  <div class="location-row">
70
  <div class="form-group">
71
- <label for="hf-year">Year folder</label>
72
- <input type="number" id="hf-year" required min="2000" max="2100" value="2025" />
 
 
 
 
 
 
 
 
73
  </div>
74
  <div class="form-group">
75
  <label for="hf-file">GeoTIFF file</label>
76
- <input type="file" id="hf-file" accept=".tif,.tiff" required />
77
  </div>
78
  </div>
79
- <div id="hf-upload-progress" class="dda-upload-progress hidden">
80
- <div class="dda-progress-bar"><div id="hf-upload-progress-fill" class="dda-progress-fill"></div></div>
81
- <span id="hf-upload-progress-label" class="dim">Uploading…</span>
82
  </div>
83
  <button type="submit" class="btn btn-primary" id="btn-hf-upload">Upload to library</button>
84
  </form>
@@ -88,7 +95,7 @@
88
  <h3 id="lib-grid-title">Images</h3>
89
  <input type="search" id="lib-filter" class="dda-search" placeholder="Search filenames…" />
90
  </div>
91
- <div id="lib-grid" class="dda-grid"><p class="dim">Select a year or add images to library_sources/.</p></div>
92
  </div>
93
  </main>
94
  </div>
@@ -259,8 +266,76 @@
259
  </div>
260
  </div>
261
 
262
- <script src="/static/js/dda/app.js?v=12"></script>
263
- <script src="/static/js/dda/library.js?v=6"></script>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
264
  <script src="/static/js/dda/result.js?v=6"></script>
265
  <script src="/static/js/dda/compare.js?v=9"></script>
266
  <script src="/static/js/dda/reports.js?v=4"></script>
 
5
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
6
  <title>DDA Change Detection</title>
7
  <link rel="stylesheet" href="/static/css/style.css?v=30" />
8
+ <link rel="stylesheet" href="/static/css/dda.css?v=12" />
9
  </head>
10
  <body>
11
  <div class="app dda-app">
 
32
  <div id="dda-error" class="alert alert-error hidden"></div>
33
  <div id="dda-success" class="alert alert-success hidden"></div>
34
 
35
+ <!-- Tab: Image Library — zone/folder/year hierarchy under library_sources/ -->
36
  <section id="tab-library" class="dda-panel active" role="tabpanel">
37
  <div class="dda-layout">
38
  <aside class="dda-sidebar card">
39
  <div class="card-header">
40
+ <h3>Library</h3>
41
+ <div class="dda-sidebar-actions">
42
+ <button type="button" class="btn btn-secondary btn-sm hidden" id="btn-manage-library" title="Manage zones and folders">Manage</button>
43
+ <button type="button" class="btn btn-secondary btn-sm" id="btn-refresh-lib" title="Rescan folder">Refresh</button>
44
+ </div>
45
  </div>
46
  <div id="lib-folder-path" class="dda-folder-path dim"></div>
47
+ <input type="search" id="lib-tree-search" class="dda-search" placeholder="Filter zones, folders, years…" />
48
  <div id="lib-tree" class="dda-tree"><p class="dim">Loading…</p></div>
 
 
 
 
49
  </aside>
50
  <main class="dda-main">
51
  <div class="card dda-instructions">
52
  <div class="card-header">
53
+ <h3>Folder library</h3>
54
  <span class="dim" id="lib-config-hint"></span>
55
  </div>
56
  <p class="sub" id="lib-instructions">
57
+ Organize images as <code>library_sources/{zone}/{folder}/{year}/</code>, then click <strong>Refresh</strong>.
58
  </p>
59
  <pre id="lib-path-display" class="dda-path-code"></pre>
60
  </div>
61
+ <div class="card" id="upload-card">
62
  <div class="card-header">
63
+ <h3 id="upload-card-title">Upload to library</h3>
64
+ <span class="dim" id="upload-card-hint"></span>
65
  </div>
66
+ <p class="sub" id="hf-upload-limit">Select zone, folder, and year before uploading.</p>
67
  <form id="form-hf-upload" class="dda-upload-form">
68
  <div class="location-row">
69
  <div class="form-group">
70
+ <label for="upload-zone">Zone</label>
71
+ <select id="upload-zone" required><option value="">— Select zone —</option></select>
72
+ </div>
73
+ <div class="form-group">
74
+ <label for="upload-folder">Folder</label>
75
+ <select id="upload-folder" required><option value="">— Select folder —</option></select>
76
+ </div>
77
+ <div class="form-group">
78
+ <label for="upload-year">Year</label>
79
+ <input type="number" id="upload-year" required min="1990" max="2100" value="2025" />
80
  </div>
81
  <div class="form-group">
82
  <label for="hf-file">GeoTIFF file</label>
83
+ <input type="file" id="hf-file" accept=".tif,.tiff,.png,.jpg,.jpeg" required />
84
  </div>
85
  </div>
86
+ <div id="upload-progress" class="dda-upload-progress hidden">
87
+ <div class="dda-progress-bar"><div id="upload-progress-fill" class="dda-progress-fill"></div></div>
88
+ <span id="upload-progress-label" class="dim">Uploading…</span>
89
  </div>
90
  <button type="submit" class="btn btn-primary" id="btn-hf-upload">Upload to library</button>
91
  </form>
 
95
  <h3 id="lib-grid-title">Images</h3>
96
  <input type="search" id="lib-filter" class="dda-search" placeholder="Search filenames…" />
97
  </div>
98
+ <div id="lib-grid" class="dda-grid"><p class="dim">Select a zone, folder, or year in the sidebar.</p></div>
99
  </div>
100
  </main>
101
  </div>
 
266
  </div>
267
  </div>
268
 
269
+ <div id="dda-manage-modal" class="dda-modal hidden" role="dialog" aria-modal="true">
270
+ <div class="dda-modal-panel card dda-manage-panel">
271
+ <div class="card-header">
272
+ <h3>Manage library structure</h3>
273
+ <button type="button" class="btn btn-secondary btn-sm" id="dda-manage-close">Close</button>
274
+ </div>
275
+ <div class="dda-manage-section">
276
+ <h4>Zones</h4>
277
+ <div class="location-row">
278
+ <input type="text" id="manage-zone-name" placeholder="New zone name" />
279
+ <button type="button" class="btn btn-primary btn-sm" id="btn-add-zone">Add zone</button>
280
+ </div>
281
+ <div class="location-row">
282
+ <select id="manage-zone-select"></select>
283
+ <input type="text" id="manage-zone-rename" placeholder="Rename selected zone" />
284
+ <button type="button" class="btn btn-secondary btn-sm" id="btn-rename-zone">Rename</button>
285
+ <button type="button" class="btn btn-secondary btn-sm" id="btn-delete-zone">Delete</button>
286
+ </div>
287
+ </div>
288
+ <div class="dda-manage-section">
289
+ <h4>Folders</h4>
290
+ <div class="location-row">
291
+ <input type="text" id="manage-folder-name" placeholder="New folder name" />
292
+ <button type="button" class="btn btn-primary btn-sm" id="btn-add-folder">Add folder</button>
293
+ </div>
294
+ <div class="location-row">
295
+ <select id="manage-folder-select"></select>
296
+ <input type="text" id="manage-folder-rename" placeholder="Rename selected folder" />
297
+ <button type="button" class="btn btn-secondary btn-sm" id="btn-rename-folder">Rename</button>
298
+ <button type="button" class="btn btn-secondary btn-sm" id="btn-delete-folder">Delete</button>
299
+ </div>
300
+ </div>
301
+ <div class="dda-manage-section">
302
+ <h4>Year folder</h4>
303
+ <div class="location-row">
304
+ <input type="number" id="manage-year" min="1990" max="2100" value="2025" />
305
+ <button type="button" class="btn btn-secondary btn-sm" id="btn-create-year">Create year folder</button>
306
+ </div>
307
+ </div>
308
+ </div>
309
+ </div>
310
+
311
+ <div id="dda-reassign-modal" class="dda-modal hidden" role="dialog" aria-modal="true">
312
+ <div class="dda-modal-panel card">
313
+ <div class="card-header">
314
+ <h3>Assign location</h3>
315
+ <button type="button" class="btn btn-secondary btn-sm" id="dda-reassign-close">Close</button>
316
+ </div>
317
+ <p class="sub">Move <strong id="reassign-filename"></strong> to:</p>
318
+ <div class="location-row">
319
+ <div class="form-group">
320
+ <label for="reassign-zone">Zone</label>
321
+ <select id="reassign-zone"></select>
322
+ </div>
323
+ <div class="form-group">
324
+ <label for="reassign-folder">Folder</label>
325
+ <select id="reassign-folder"></select>
326
+ </div>
327
+ <div class="form-group">
328
+ <label for="reassign-year">Year</label>
329
+ <input type="number" id="reassign-year" min="1990" max="2100" value="2025" />
330
+ </div>
331
+ </div>
332
+ <button type="button" class="btn btn-primary" id="btn-reassign-submit">Move image</button>
333
+ </div>
334
+ </div>
335
+
336
+ <script src="/static/js/dda/app.js?v=13"></script>
337
+ <script src="/static/js/dda/hierarchy.js?v=1"></script>
338
+ <script src="/static/js/dda/library.js?v=7"></script>
339
  <script src="/static/js/dda/result.js?v=6"></script>
340
  <script src="/static/js/dda/compare.js?v=9"></script>
341
  <script src="/static/js/dda/reports.js?v=4"></script>