Visual Document Retrieval
ColPali
coreai
coreai-aimodel
core-ai
apple
on-device
late-interaction
colbert
retrieval
Instructions to use mlboydaisuke/ColModernVBERT-CoreAI with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- ColPali
How to use mlboydaisuke/ColModernVBERT-CoreAI with ColPali:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
gen-cards: regenerate Use-it block
Browse files
README.md
CHANGED
|
@@ -42,19 +42,24 @@ dense) and [Qwen3-Reranker](https://huggingface.co/mlboydaisuke/Qwen3-Reranker-0
|
|
| 42 |
<!-- gen-cards:use-it begin id=colmodernvbert (managed by scripts/gen-cards β edit cards.json / QuickStart.swift, not this block) -->
|
| 43 |
## Use it
|
| 44 |
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
(visual page search over bundled sample pages; the GUI (iPhone) adds tiled where-it-matched highlights):
|
| 47 |
|
| 48 |
```bash
|
| 49 |
-
git clone https://github.com/john-rocky/coreai-kit
|
| 50 |
-
|
|
|
|
| 51 |
# β Run, then pick "ColModernVBERT" in the model picker
|
| 52 |
|
| 53 |
# agents / headless (macOS):
|
| 54 |
cd coreai-kit/Examples/DocSearch
|
| 55 |
-
swift run docsearch-cli --model colmodernvbert --query "monthly revenue trend"
|
| 56 |
```
|
| 57 |
|
|
|
|
|
|
|
| 58 |
π» **Build with it** β complete; the glue is kit API, copy-paste runs:
|
| 59 |
|
| 60 |
```swift
|
|
@@ -70,7 +75,7 @@ let hits = try await retriever.retrieve(query: query, over: corpus, topK: pages.
|
|
| 70 |
// hits: pages ranked by MaxSim, best match first β no OCR, pages are matched as pictures
|
| 71 |
```
|
| 72 |
|
| 73 |
-
The take-home is [`Examples/DocSearch/Sources/QuickStart.swift`](https://github.com/john-rocky/coreai-kit/blob/
|
| 74 |
β this exact code as one typed function, no UI; the CLI is an argument shell over it, and
|
| 75 |
the GUI drives the same `VisualDocumentRetriever(catalog:)` with tiled per-page encoding.
|
| 76 |
Encode your corpus once and keep the `PageEmbedding`s β scoring a query is then host-side
|
|
@@ -78,10 +83,10 @@ MaxSim, no model call per page. `encodeTiled(page:)` localizes *where* a query m
|
|
| 78 |
|
| 79 |
**Integration checklist**
|
| 80 |
|
| 81 |
-
- SPM: `https://github.com/john-rocky/coreai-kit` β product **CoreAIKitEmbeddings**
|
| 82 |
- Info.plist: `NSPhotoLibraryUsageDescription` β only if you use PhotosPicker to import pages
|
| 83 |
- Entitlements: none needed
|
| 84 |
-
- First run downloads the model β
|
| 85 |
local cache (Application Support; progress via the `downloadProgress` callback)
|
| 86 |
- Measure in Release β Debug is ~3Γ slower on per-token host work
|
| 87 |
<!-- gen-cards:use-it end -->
|
|
|
|
| 42 |
<!-- gen-cards:use-it begin id=colmodernvbert (managed by scripts/gen-cards β edit cards.json / QuickStart.swift, not this block) -->
|
| 43 |
## Use it
|
| 44 |
|
| 45 |
+
**New to Core AI? [Start with CoreAIKit 0.7.1](https://github.com/john-rocky/coreai-kit#readme).** Follow its requirements and first-run steps for `qwen3-0.6b`, then open the same release's [ChatDemo](https://github.com/john-rocky/coreai-kit/tree/0.7.1/Examples/ChatDemo). The README records the tested OS/SDK and download size; model and device coverage is stated per example.
|
| 46 |
+
|
| 47 |
+
βΆοΈ **Run it (source)** β the [DocSearch runner](https://github.com/john-rocky/coreai-kit/tree/0.7.1/Examples/DocSearch)
|
| 48 |
(visual page search over bundled sample pages; the GUI (iPhone) adds tiled where-it-matched highlights):
|
| 49 |
|
| 50 |
```bash
|
| 51 |
+
git clone --branch 0.7.1 --depth 1 https://github.com/john-rocky/coreai-kit
|
| 52 |
+
export DEVELOPER_DIR=/Applications/Xcode-27.0.0-RC.app/Contents/Developer
|
| 53 |
+
open -a /Applications/Xcode-27.0.0-RC.app coreai-kit/Examples/DocSearch/DocSearch.xcodeproj
|
| 54 |
# β Run, then pick "ColModernVBERT" in the model picker
|
| 55 |
|
| 56 |
# agents / headless (macOS):
|
| 57 |
cd coreai-kit/Examples/DocSearch
|
| 58 |
+
swift run -c release docsearch-cli --model colmodernvbert --query "monthly revenue trend"
|
| 59 |
```
|
| 60 |
|
| 61 |
+
Use Xcode build **27A266a** from the release's `.xcode-pin`; adjust the app path if your installation is named differently.
|
| 62 |
+
|
| 63 |
π» **Build with it** β complete; the glue is kit API, copy-paste runs:
|
| 64 |
|
| 65 |
```swift
|
|
|
|
| 75 |
// hits: pages ranked by MaxSim, best match first β no OCR, pages are matched as pictures
|
| 76 |
```
|
| 77 |
|
| 78 |
+
The take-home is [`Examples/DocSearch/Sources/QuickStart.swift`](https://github.com/john-rocky/coreai-kit/blob/0.7.1/Examples/DocSearch/Sources/QuickStart.swift)
|
| 79 |
β this exact code as one typed function, no UI; the CLI is an argument shell over it, and
|
| 80 |
the GUI drives the same `VisualDocumentRetriever(catalog:)` with tiled per-page encoding.
|
| 81 |
Encode your corpus once and keep the `PageEmbedding`s β scoring a query is then host-side
|
|
|
|
| 83 |
|
| 84 |
**Integration checklist**
|
| 85 |
|
| 86 |
+
- SPM: `https://github.com/john-rocky/coreai-kit` (exact **0.7.1**) β product **CoreAIKitEmbeddings**
|
| 87 |
- Info.plist: `NSPhotoLibraryUsageDescription` β only if you use PhotosPicker to import pages
|
| 88 |
- Entitlements: none needed
|
| 89 |
+
- First run downloads the model β ~741 MB (Mac) / ~741 MB (iPhone) β then it loads from the
|
| 90 |
local cache (Application Support; progress via the `downloadProgress` callback)
|
| 91 |
- Measure in Release β Debug is ~3Γ slower on per-token host work
|
| 92 |
<!-- gen-cards:use-it end -->
|