Spaces:
Build error
Build error
| title: IIIF Studio | |
| emoji: 📜 | |
| colorFrom: blue | |
| colorTo: yellow | |
| sdk: docker | |
| app_port: 7860 | |
| pinned: false | |
| # IIIF Studio | |
| A generic platform for generating AI-augmented scholarly editions from digitized heritage documents — medieval manuscripts, incunabula, cartularies, archives, charters, papyri. Any document type, any era, any language. | |
| IIIF Studio ingests images from any [IIIF](https://iiif.io/)-compliant server, analyzes them with multimodal AI (Google Gemini, Mistral), and produces structured scholarly data: diplomatic OCR, layout detection, translations, commentaries, and iconographic analysis — all exportable as ALTO XML, METS, and IIIF Presentation 3.0 manifests. | |
| **Images are never stored locally.** The platform streams them from origin servers using the IIIF Image API, storing only the AI-generated metadata (~5 KB per page instead of ~50 MB). | |
| --- | |
| ## Features | |
| - **IIIF-native architecture** — images streamed from origin servers (Gallica, BnF, Bodleian, etc.) with tiled deep zoom via OpenSeadragon | |
| - **Multi-provider AI** — Google AI Studio, Vertex AI, Mistral AI. Model selected per corpus, auto-detected from environment | |
| - **Profile-driven analysis** — 4 built-in corpus profiles (medieval illuminated, medieval textual, early modern print, modern handwritten), each with tailored prompts and active layers | |
| - **Structured output** — layout regions with bounding boxes, diplomatic OCR, translations (FR/EN), scholarly and public commentary, iconographic analysis, uncertainty tracking | |
| - **Standards-compliant export** — IIIF Presentation 3.0 manifests (with Image Service for tiled zoom), ALTO XML, METS XML, ZIP bundles | |
| - **Human-in-the-loop** — editorial correction interface with versioned history and rollback | |
| - **Full-text search** — accent-insensitive search across OCR text, translations, and iconographic tags | |
| --- | |
| ## Architecture | |
| ``` | |
| ┌─────────────────────────────────────────────────────────────────────────────────┐ | |
| │ IIIF IMAGE SERVERS │ | |
| │ Gallica · BnF · Bodleian · Europeana · ... │ | |
| │ (origin — images are never copied) │ | |
| └──────────┬────────────────────┬─────────────────────────────┬───────────────────┘ | |
| │ │ │ | |
| │ info.json │ /full/!1500,1500/ │ /full/max/ | |
| │ + tiles │ 0/default.jpg │ 0/default.jpg | |
| │ │ (1500px for AI) │ | |
| │ │ │ | |
| ┌──────────▼──────────┐ ┌──────▼───────────┐ ┌──────────────▼──────────────────┐ | |
| │ │ │ │ │ │ | |
| │ FRONTEND (SPA) │ │ BACKEND (API) │ │ EXPORT GENERATORS │ | |
| │ React + Vite │ │ FastAPI │ │ │ | |
| │ │ │ │ │ IIIF Manifest 3.0 │ | |
| │ ┌─────────────────┐ │ │ ┌──────────────┐ │ │ (with Image Service refs) │ | |
| │ │ OpenSeadragon │ │ │ │ Ingestion │ │ │ │ | |
| │ │ IIIF tiled zoom │ │ │ │ │ │ │ METS XML │ | |
| │ │ (info.json → │ │ │ │ manifest URL │ │ │ (IIIF URLs, not file paths) │ | |
| │ │ deep zoom) │ │ │ │ → detect svc │ │ │ │ | |
| │ └────────┬─────────┘ │ │ │ → store meta │ │ │ ALTO XML │ | |
| │ │ │ │ └──────┬───────┘ │ │ (text geometry per page) │ | |
| │ ┌────────▼─────────┐ │ │ │ │ │ │ | |
| │ │ Region overlays │ │ │ ┌─────▼────────┐│ │ ZIP bundle │ | |
| │ │ (bbox from │ │ │ │ AI Pipeline ││ │ (manifest + METS + ALTO) │ | |
| │ │ master.json, │ │ │ │ ││ └─────────────────────────────────┘ | |
| │ │ scaled to │ │ │ │ fetch 1500px││ | |
| │ │ canvas coords) │ │ │ │ in memory ││ ┌──────────────────────┐ | |
| │ └──────────────────┘ │ │ │ │ ││ │ │ | |
| │ │ │ │ ▼ ││ │ AI PROVIDERS │ | |
| │ ┌──────────────────┐ │ │ │ send bytes │├───────►│ │ | |
| │ │ Pages │ │ │ │ to AI ││ │ Google Gemini │ | |
| │ │ Home · Reader │ │ │ │ │ ││◄───────│ Vertex AI │ | |
| │ │ Editor · Admin │ │ │ │ ▼ ││ JSON │ Mistral AI │ | |
| │ └────────┬─────────┘ │ │ │ discard img ││ │ │ | |
| │ │ │ │ │ keep JSON ││ │ (auto-detected from │ | |
| │ │ REST API │ │ │ scale bbox ││ │ environment vars) │ | |
| │ │ /api/v1/* │ │ └─────┬───────┘│ └──────────────────────┘ | |
| └──────────┼───────────┘ │ │ │ | |
| │ │ ┌─────▼────────┐│ | |
| │ │ │ Response ││ | |
| │ │ │ Parser ││ | |
| └─────────────┤ │ ││ | |
| │ │ raw JSON ││ | |
| │ │ → layout ││ | |
| │ │ → OCR ││ | |
| │ │ → regions ││ | |
| │ └─────┬───────┘│ | |
| │ │ │ | |
| │ ┌─────▼────────┐│ | |
| │ │ Master ││ | |
| │ │ Writer ││ | |
| │ │ ││ | |
| │ │ ai_raw.json ││ ┌───────────────────────────────┐ | |
| │ │ master.json ││ │ │ | |
| │ └─────┬───────┘│ │ LOCAL STORAGE │ | |
| │ │ │ │ │ | |
| └───────┼─────────┘ │ SQLite (corpus, pages, │ | |
| │ │ manuscripts, jobs, models) │ | |
| └──────────────► │ | |
| │ data/corpora/{slug}/pages/ │ | |
| │ {folio}/master.json │ | |
| │ {folio}/ai_raw.json │ | |
| │ {folio}/alto.xml │ | |
| │ │ | |
| │ ~5 KB per page (JSON only) │ | |
| │ NO image binaries │ | |
| └───────────────────────────────┘ | |
| PIPELINE FLOW (per page): | |
| ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ | |
| │ 1.INGEST │───►│ 2.DETECT │───►│ 3.FETCH │───►│ 4.AI │───►│ 5.PARSE │ | |
| │ │ │ │ │ │ │ │ │ │ | |
| │ manifest │ │ IIIF svc │ │ 1500px │ │ send │ │ layout │ | |
| │ URL │ │ URL + │ │ JPEG in │ │ image + │ │ regions │ | |
| │ │ │ canvas │ │ memory │ │ prompt │ │ OCR │ | |
| │ │ │ dims │ │ (discard │ │ to │ │ bbox │ | |
| │ │ │ │ │ after) │ │ provider │ │ │ | |
| └──────────┘ └──────────┘ └──────────┘ └──────────┘ └────┬─────┘ | |
| │ | |
| ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────▼─────┐ | |
| │ 8.EXPORT │◄───│ 7.REVIEW │◄───│ 6.WRITE │◄──────────────────│ 5b.SCALE │ | |
| │ │ │ │ │ │ │ │ | |
| │ IIIF 3.0 │ │ human │ │ ai_raw + │ │ bbox │ | |
| │ ALTO XML │ │ correct │ │ master │ │ deriv → │ | |
| │ METS XML │ │ validate │ │ .json │ │ canvas │ | |
| │ ZIP │ │ version │ │ + ALTO │ │ coords │ | |
| └──────────┘ └──────────┘ └──────────┘ └──────────┘ | |
| ``` | |
| ### Tech stack | |
| | Layer | Technology | | |
| |-------|-----------| | |
| | Backend | Python 3.11+, FastAPI, Uvicorn | | |
| | Database | SQLite via SQLAlchemy 2.0 async + aiosqlite | | |
| | Validation | Pydantic v2 | | |
| | AI providers | Google Gemini (google-genai SDK), Mistral AI | | |
| | Image viewer | OpenSeadragon (IIIF tiled zoom) | | |
| | Frontend | React 18, TypeScript, Vite, Tailwind CSS, React Router | | |
| | Exports | lxml (ALTO/METS XML), IIIF Presentation 3.0 | | |
| | Deployment | Docker (HuggingFace Spaces) | | |
| --- | |
| ## Quick start | |
| ### Docker (recommended) | |
| ```bash | |
| git clone https://github.com/maribakulj/IIIF-Studio.git && cd IIIF-Studio | |
| # Configure at least one AI provider key | |
| cp .env.example .env | |
| # Edit .env and add your API key(s) | |
| # Build and run | |
| docker compose -f infra/docker-compose.yml up --build | |
| # Open http://localhost:7860 | |
| ``` | |
| ### Local development | |
| ```bash | |
| # Backend | |
| cd backend | |
| pip install -e ".[dev]" | |
| uvicorn app.main:app --reload --port 7860 | |
| # Frontend (separate terminal) | |
| cd frontend | |
| npm install | |
| npm run dev | |
| ``` | |
| The API is available at `http://localhost:7860/api/v1/`. Interactive Swagger docs at `http://localhost:7860/docs`. | |
| --- | |
| ## Usage workflow | |
| 1. **Create a corpus** — select a profile matching your document type | |
| 2. **Ingest pages** — provide a IIIF manifest URL, direct image URLs, or upload files | |
| 3. **Select an AI model** — choose a provider and model from the detected options | |
| 4. **Run the pipeline** — AI analyzes each page: layout detection, OCR, translation, commentary | |
| 5. **Review and correct** — use the Editor to validate, correct OCR, adjust regions | |
| 6. **Export** — download IIIF manifest, ALTO XML, METS XML, or a ZIP bundle | |
| --- | |
| ## Corpus profiles | |
| Profiles control which analysis layers are active, which prompt templates are used, and what uncertainty thresholds apply. | |
| | Profile | Script | Languages | Key layers | | |
| |---------|--------|-----------|------------| | |
| | `medieval-illuminated` | Caroline | Latin, French | OCR, translation, iconography, commentary, material notes | | |
| | `medieval-textual` | Gothic | Latin, French | OCR, translation, scholarly commentary | | |
| | `early-modern-print` | Print | French, Latin | OCR, summary | | |
| | `modern-handwritten` | Cursive | French | OCR, summary | | |
| Custom profiles can be added as JSON files in the `profiles/` directory with matching prompt templates in `prompts/`. | |
| --- | |
| ## AI providers | |
| The backend auto-detects available providers from environment variables. No global selector — the model is chosen per corpus from the admin interface. | |
| | Provider | Environment variable | Notes | | |
| |----------|---------------------|-------| | |
| | Google AI Studio | `GOOGLE_AI_STUDIO_API_KEY` | Free tier, good for development | | |
| | Vertex AI (service account) | `VERTEX_SERVICE_ACCOUNT_JSON` | Institutional deployments | | |
| | Mistral AI | `MISTRAL_API_KEY` | Alternative provider | | |
| At least **one** key is required for the pipeline to function. Keys must **never** appear in code, commits, or Docker images. | |
| --- | |
| ## API reference | |
| All endpoints are prefixed with `/api/v1/`. Full OpenAPI docs available at `/docs`. | |
| ### Corpus management | |
| | Method | Endpoint | Description | | |
| |--------|----------|-------------| | |
| | `GET` | `/corpora` | List all corpora | | |
| | `POST` | `/corpora` | Create a corpus (slug + title + profile) | | |
| | `GET` | `/corpora/{id}` | Get a corpus | | |
| | `DELETE` | `/corpora/{id}` | Delete a corpus (cascades) | | |
| | `GET` | `/corpora/{id}/manuscripts` | List manuscripts in a corpus | | |
| ### Ingestion | |
| | Method | Endpoint | Description | | |
| |--------|----------|-------------| | |
| | `POST` | `/corpora/{id}/ingest/iiif-manifest` | Ingest from a IIIF manifest URL | | |
| | `POST` | `/corpora/{id}/ingest/iiif-images` | Ingest from direct image URLs | | |
| | `POST` | `/corpora/{id}/ingest/files` | Upload image files | | |
| ### AI pipeline | |
| | Method | Endpoint | Description | | |
| |--------|----------|-------------| | |
| | `GET` | `/providers` | List detected AI providers | | |
| | `GET` | `/providers/{type}/models` | List models for a provider | | |
| | `PUT` | `/corpora/{id}/model` | Set AI model for a corpus | | |
| | `POST` | `/corpora/{id}/run` | Run pipeline on all pages | | |
| | `POST` | `/pages/{id}/run` | Run pipeline on a single page | | |
| | `GET` | `/jobs/{id}` | Check job status | | |
| | `POST` | `/jobs/{id}/retry` | Retry a failed job | | |
| ### Pages and content | |
| | Method | Endpoint | Description | | |
| |--------|----------|-------------| | |
| | `GET` | `/pages/{id}` | Page metadata | | |
| | `GET` | `/pages/{id}/master-json` | Full page master (canonical JSON) | | |
| | `GET` | `/pages/{id}/layers` | List annotation layers | | |
| | `POST` | `/pages/{id}/corrections` | Apply editorial corrections | | |
| | `GET` | `/pages/{id}/history` | Version history | | |
| | `GET` | `/search?q=` | Full-text search across all pages | | |
| ### Export | |
| | Method | Endpoint | Description | | |
| |--------|----------|-------------| | |
| | `GET` | `/manuscripts/{id}/iiif-manifest` | IIIF Presentation 3.0 manifest | | |
| | `GET` | `/manuscripts/{id}/mets` | METS XML | | |
| | `GET` | `/pages/{id}/alto` | ALTO XML | | |
| | `GET` | `/manuscripts/{id}/export.zip` | ZIP bundle (manifest + METS + ALTO) | | |
| --- | |
| ## Data model | |
| Each analyzed page produces a `master.json` — the canonical source of truth for all exports. | |
| ``` | |
| PageMaster | |
| ├── image → IIIF service URL, canvas dimensions, provenance | |
| ├── layout → regions with bounding boxes [x, y, w, h] in absolute pixels | |
| ├── ocr → diplomatic text, confidence, uncertain segments | |
| ├── translation → French, English | |
| ├── summary → short + detailed | |
| ├── commentary → public, scholarly, sourced claims with certainty levels | |
| ├── extensions → profile-specific data (iconography, materiality, etc.) | |
| ├── processing → provider, model, prompt version, timestamp | |
| └── editorial → status (machine_draft → validated → published), version | |
| ``` | |
| Bounding boxes follow the convention `[x, y, width, height]` in absolute pixels of the original image. Coordinates are automatically scaled from AI analysis space to full canvas dimensions. | |
| --- | |
| ## IIIF-native image handling | |
| IIIF Studio operates in two modes: | |
| ### IIIF-native mode (default for manifest/URL ingestion) | |
| - Images are **never downloaded or stored** locally | |
| - At ingestion: IIIF Image Service URL and canvas dimensions are extracted from the manifest | |
| - At analysis: a 1500px derivative is fetched in memory via the IIIF Image API (`{service}/full/!1500,1500/0/default.jpg`), sent to the AI, then discarded | |
| - In the viewer: OpenSeadragon loads `info.json` from the IIIF server for native tiled deep zoom | |
| - Storage per page: **~5 KB** (JSON metadata only) | |
| ### File upload mode (for non-IIIF sources) | |
| - Uploaded images are stored locally in `data/corpora/{slug}/` | |
| - Derivatives (1500px) and thumbnails (256px) are created on disk | |
| - Storage per page: **~50 MB** (images + JSON) | |
| --- | |
| ## Project structure | |
| ``` | |
| IIIF-Studio/ | |
| ├── backend/ | |
| │ ├── app/ | |
| │ │ ├── main.py # FastAPI entry point | |
| │ │ ├── config.py # Pydantic settings from env vars | |
| │ │ ├── api/v1/ # REST endpoints | |
| │ │ ├── models/ # SQLAlchemy ORM models | |
| │ │ ├── schemas/ # Pydantic v2 schemas (canonical) | |
| │ │ └── services/ | |
| │ │ ├── ai/ # Provider factory, analyzer, prompt loader | |
| │ │ ├── ingest/ # IIIF fetcher, service detection | |
| │ │ ├── image/ # Normalizer (in-memory + legacy disk) | |
| │ │ └── export/ # ALTO, METS, IIIF manifest generators | |
| │ ├── tests/ # 585 tests (pytest + pytest-asyncio) | |
| │ └── pyproject.toml | |
| ├── frontend/ | |
| │ ├── src/ | |
| │ │ ├── App.tsx # React Router (/, /admin, /reader, /editor) | |
| │ │ ├── lib/api.ts # Typed API client | |
| │ │ ├── pages/ # Home, Reader, Editor, Admin | |
| │ │ └── components/ # Viewer (OpenSeadragon), retro UI system | |
| │ └── package.json | |
| ├── profiles/ # 4 corpus profile JSON files | |
| ├── prompts/ # 9 prompt templates organized by profile | |
| ├── Dockerfile # Multi-stage build (Node + Python) | |
| ├── infra/docker-compose.yml # Local development | |
| └── .env.example # Environment variable template | |
| ``` | |
| --- | |
| ## Testing | |
| ```bash | |
| cd backend | |
| pip install -e ".[dev]" | |
| pytest tests/ -v --cov=app | |
| ``` | |
| Expected result: **585 passed, 3 skipped**. | |
| All AI calls are mocked in tests — no API keys required to run the test suite. | |
| --- | |
| ## Deployment | |
| ### HuggingFace Spaces | |
| This repository is configured for [HuggingFace Spaces](https://huggingface.co/spaces) with Docker SDK on port 7860. AI keys are stored as Space secrets (Settings → Repository secrets). | |
| The CI pipeline (`.github/workflows/`) runs tests on every push and auto-deploys to HuggingFace Spaces on merge to `main`. | |
| ### Self-hosted | |
| ```bash | |
| docker build -t iiif-studio . | |
| docker run -p 7860:7860 \ | |
| -e GOOGLE_AI_STUDIO_API_KEY=your_key \ | |
| -v ./data:/app/data \ | |
| iiif-studio | |
| ``` | |
| --- | |
| ## License | |
| [Apache License 2.0](LICENSE) | |