IMvision12 commited on
Commit
d2ffbe2
·
verified ·
1 Parent(s): bcc48c4

Migrate to zeromodels (rename kf_*.json -> zm_*.json, fix refs in config + README, ensure tag + badge)

Browse files
README.md CHANGED
@@ -2,10 +2,10 @@
2
  pipeline_tag: zero-shot-object-detection
3
  license: apache-2.0
4
  base_model: google/owlvit-base-patch32
5
- library_name: kerasformers
6
  tags:
7
  - keras
8
- - kerasformers
9
  - owlvit
10
  - open-vocabulary
11
  - zero-shot-object-detection
@@ -16,13 +16,13 @@ tags:
16
  - tf
17
  ---
18
 
19
- ## ***See [our collection](https://huggingface.co/collections/kerasformers/owl-vit-6a6a7af2bb61206cd50397ff) for all versions of OWL-ViT.***
20
 
21
  # Run OWL-ViT with Keras 3: JAX, PyTorch, or TensorFlow
22
 
23
- [![GitHub](https://img.shields.io/badge/GitHub-KerasFormers-black?logo=github)](https://github.com/IMvision12/KerasFormers) [![Docs](https://img.shields.io/badge/Docs-OWL--ViT-blue)](https://imvision12.github.io/KerasFormers/owlvit/) [![Collection](https://img.shields.io/badge/HF-OWL--ViT%20collection-yellow)](https://huggingface.co/collections/kerasformers/owl-vit-6a6a7af2bb61206cd50397ff)
24
 
25
- # kerasformers/owlvit-base-patch32
26
 
27
  Paper: [Simple Open-Vocabulary Object Detection with Vision Transformers (arXiv:2205.06230)](https://arxiv.org/abs/2205.06230) · [HF Papers](https://huggingface.co/papers/2205.06230)
28
 
@@ -30,7 +30,7 @@ OWL-ViT detects objects described by free text, with no fixed class list. It sta
30
 
31
  For more details on the model, please go to Google's original [model card](https://huggingface.co/google/owlvit-base-patch32).
32
 
33
- Pure-**Keras 3** conversion of [`google/owlvit-base-patch32`](https://huggingface.co/google/owlvit-base-patch32) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
34
 
35
  This is an **open-vocabulary object detection** checkpoint (`OwlViTDetect`): pass free-text prompts at inference time.
36
 
@@ -41,15 +41,15 @@ import os
41
  os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
42
 
43
  from PIL import Image
44
- from kerasformers.models.owlvit import (
45
  OwlViTDetect,
46
  OwlViTProcessor,
47
  OwlViTImageProcessor,
48
  )
49
 
50
- model = OwlViTDetect.from_weights("kerasformers/owlvit-base-patch32")
51
- processor = OwlViTProcessor.from_weights("kerasformers/owlvit-base-patch32")
52
- image_processor = OwlViTImageProcessor.from_weights("kerasformers/owlvit-base-patch32")
53
 
54
  image = Image.open("your_image.jpg").convert("RGB")
55
  prompts = ["a photo of a mug", "a photo of a knife"]
@@ -72,27 +72,27 @@ for score, name, box in zip(
72
  print(f"{name}: {float(score):.3f} {box}")
73
  ```
74
 
75
- Load any OWL-ViT / OWLv2 variant the same way with `from_weights("kerasformers/<variant>")` (use `OwlViTDetect` for this repo):
76
 
77
  | Variant | Hub | Family |
78
  |---|---|---|
79
- | `owlvit-base-patch32` | [`kerasformers/owlvit-base-patch32`](https://huggingface.co/kerasformers/owlvit-base-patch32) | OWL-ViT |
80
- | `owlvit-base-patch16` | [`kerasformers/owlvit-base-patch16`](https://huggingface.co/kerasformers/owlvit-base-patch16) | OWL-ViT |
81
- | `owlvit-large-patch14` | [`kerasformers/owlvit-large-patch14`](https://huggingface.co/kerasformers/owlvit-large-patch14) | OWL-ViT |
82
- | `owlv2-base-patch16` | [`kerasformers/owlv2-base-patch16`](https://huggingface.co/kerasformers/owlv2-base-patch16) | OWLv2 |
83
- | `owlv2-base-patch16-ensemble` | [`kerasformers/owlv2-base-patch16-ensemble`](https://huggingface.co/kerasformers/owlv2-base-patch16-ensemble) | OWLv2 |
84
- | `owlv2-base-patch16-finetuned` | [`kerasformers/owlv2-base-patch16-finetuned`](https://huggingface.co/kerasformers/owlv2-base-patch16-finetuned) | OWLv2 |
85
- | `owlv2-large-patch14` | [`kerasformers/owlv2-large-patch14`](https://huggingface.co/kerasformers/owlv2-large-patch14) | OWLv2 |
86
- | `owlv2-large-patch14-ensemble` | [`kerasformers/owlv2-large-patch14-ensemble`](https://huggingface.co/kerasformers/owlv2-large-patch14-ensemble) | OWLv2 |
87
- | `owlv2-large-patch14-finetuned` | [`kerasformers/owlv2-large-patch14-finetuned`](https://huggingface.co/kerasformers/owlv2-large-patch14-finetuned) | OWLv2 |
88
 
89
  ## Tips
90
 
91
- - Set `KERAS_BACKEND` **before** importing Keras / kerasformers.
92
  - Prefer `Processor.from_weights(...)` so image size matches the variant.
93
  - Open-vocab thresholds are often much lower than closed-set detectors (try `0.1`).
94
  - OWLv2 pads to square before resize; pass the original `(height, width)` as `target_sizes` carefully (see the OWLv2 docs for the padding trap).
95
- - See [OWL-ViT docs](https://imvision12.github.io/KerasFormers/owlvit/) and [Loading Weights](https://imvision12.github.io/KerasFormers/loading_weights/).
96
  - Community / upstream safetensors still work via the `hf:` prefix, e.g. `OwlViTDetect.from_weights("hf:google/owlvit-base-patch32")`.
97
 
98
  ## Special Thanks
 
2
  pipeline_tag: zero-shot-object-detection
3
  license: apache-2.0
4
  base_model: google/owlvit-base-patch32
5
+ library_name: zeromodels
6
  tags:
7
  - keras
8
+ - zeromodels
9
  - owlvit
10
  - open-vocabulary
11
  - zero-shot-object-detection
 
16
  - tf
17
  ---
18
 
19
+ ## ***See [our collection](https://huggingface.co/collections/zeromodels/owl-vit-6a6a7af2bb61206cd50397ff) for all versions of OWL-ViT.***
20
 
21
  # Run OWL-ViT with Keras 3: JAX, PyTorch, or TensorFlow
22
 
23
+ [![GitHub](https://img.shields.io/badge/GitHub-ZeroModels-black?logo=github)](https://github.com/IMvision12/ZeroModels) [![Docs](https://img.shields.io/badge/Docs-OWL--ViT-blue)](https://imvision12.github.io/ZeroModels/owlvit/) [![Collection](https://img.shields.io/badge/HF-OWL--ViT%20collection-yellow)](https://huggingface.co/collections/zeromodels/owl-vit-6a6a7af2bb61206cd50397ff)
24
 
25
+ # zeromodels/owlvit-base-patch32
26
 
27
  Paper: [Simple Open-Vocabulary Object Detection with Vision Transformers (arXiv:2205.06230)](https://arxiv.org/abs/2205.06230) · [HF Papers](https://huggingface.co/papers/2205.06230)
28
 
 
30
 
31
  For more details on the model, please go to Google's original [model card](https://huggingface.co/google/owlvit-base-patch32).
32
 
33
+ Pure-**Keras 3** conversion of [`google/owlvit-base-patch32`](https://huggingface.co/google/owlvit-base-patch32) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
34
 
35
  This is an **open-vocabulary object detection** checkpoint (`OwlViTDetect`): pass free-text prompts at inference time.
36
 
 
41
  os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
42
 
43
  from PIL import Image
44
+ from zeromodels.models.owlvit import (
45
  OwlViTDetect,
46
  OwlViTProcessor,
47
  OwlViTImageProcessor,
48
  )
49
 
50
+ model = OwlViTDetect.from_weights("zeromodels/owlvit-base-patch32")
51
+ processor = OwlViTProcessor.from_weights("zeromodels/owlvit-base-patch32")
52
+ image_processor = OwlViTImageProcessor.from_weights("zeromodels/owlvit-base-patch32")
53
 
54
  image = Image.open("your_image.jpg").convert("RGB")
55
  prompts = ["a photo of a mug", "a photo of a knife"]
 
72
  print(f"{name}: {float(score):.3f} {box}")
73
  ```
74
 
75
+ Load any OWL-ViT / OWLv2 variant the same way with `from_weights("zeromodels/<variant>")` (use `OwlViTDetect` for this repo):
76
 
77
  | Variant | Hub | Family |
78
  |---|---|---|
79
+ | `owlvit-base-patch32` | [`zeromodels/owlvit-base-patch32`](https://huggingface.co/zeromodels/owlvit-base-patch32) | OWL-ViT |
80
+ | `owlvit-base-patch16` | [`zeromodels/owlvit-base-patch16`](https://huggingface.co/zeromodels/owlvit-base-patch16) | OWL-ViT |
81
+ | `owlvit-large-patch14` | [`zeromodels/owlvit-large-patch14`](https://huggingface.co/zeromodels/owlvit-large-patch14) | OWL-ViT |
82
+ | `owlv2-base-patch16` | [`zeromodels/owlv2-base-patch16`](https://huggingface.co/zeromodels/owlv2-base-patch16) | OWLv2 |
83
+ | `owlv2-base-patch16-ensemble` | [`zeromodels/owlv2-base-patch16-ensemble`](https://huggingface.co/zeromodels/owlv2-base-patch16-ensemble) | OWLv2 |
84
+ | `owlv2-base-patch16-finetuned` | [`zeromodels/owlv2-base-patch16-finetuned`](https://huggingface.co/zeromodels/owlv2-base-patch16-finetuned) | OWLv2 |
85
+ | `owlv2-large-patch14` | [`zeromodels/owlv2-large-patch14`](https://huggingface.co/zeromodels/owlv2-large-patch14) | OWLv2 |
86
+ | `owlv2-large-patch14-ensemble` | [`zeromodels/owlv2-large-patch14-ensemble`](https://huggingface.co/zeromodels/owlv2-large-patch14-ensemble) | OWLv2 |
87
+ | `owlv2-large-patch14-finetuned` | [`zeromodels/owlv2-large-patch14-finetuned`](https://huggingface.co/zeromodels/owlv2-large-patch14-finetuned) | OWLv2 |
88
 
89
  ## Tips
90
 
91
+ - Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
92
  - Prefer `Processor.from_weights(...)` so image size matches the variant.
93
  - Open-vocab thresholds are often much lower than closed-set detectors (try `0.1`).
94
  - OWLv2 pads to square before resize; pass the original `(height, width)` as `target_sizes` carefully (see the OWLv2 docs for the padding trap).
95
+ - See [OWL-ViT docs](https://imvision12.github.io/ZeroModels/owlvit/) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
96
  - Community / upstream safetensors still work via the `hf:` prefix, e.g. `OwlViTDetect.from_weights("hf:google/owlvit-base-patch32")`.
97
 
98
  ## Special Thanks
kf_config.json → zm_config.json RENAMED
@@ -1,29 +1,29 @@
1
- {
2
- "library_name": "kerasformers",
3
- "kerasformers_version": "1.2.1",
4
- "model_module": "kerasformers.models.owlvit",
5
- "model_class": "OwlViTDetect",
6
- "variant": "owlvit-base-patch32",
7
- "weights": "model.weights.h5",
8
- "schema_version": 2,
9
- "weight_dtype": "float32",
10
- "model_type": "owlvit",
11
- "text_config": {
12
- "hidden_dim": 512,
13
- "intermediate_size": 2048,
14
- "num_heads": 8,
15
- "num_layers": 12,
16
- "max_position_embeddings": 16,
17
- "vocab_size": 49408
18
- },
19
- "vision_config": {
20
- "image_size": 768,
21
- "patch_size": 32,
22
- "hidden_dim": 768,
23
- "intermediate_size": 3072,
24
- "num_layers": 12,
25
- "num_heads": 12
26
- },
27
- "projection_dim": 512,
28
- "image_size": null
29
  }
 
1
+ {
2
+ "library_name": "zeromodels",
3
+ "zeromodels_version": "1.2.1",
4
+ "model_module": "zeromodels.models.owlvit",
5
+ "model_class": "OwlViTDetect",
6
+ "variant": "owlvit-base-patch32",
7
+ "weights": "model.weights.h5",
8
+ "schema_version": 2,
9
+ "weight_dtype": "float32",
10
+ "model_type": "owlvit",
11
+ "text_config": {
12
+ "hidden_dim": 512,
13
+ "intermediate_size": 2048,
14
+ "num_heads": 8,
15
+ "num_layers": 12,
16
+ "max_position_embeddings": 16,
17
+ "vocab_size": 49408
18
+ },
19
+ "vision_config": {
20
+ "image_size": 768,
21
+ "patch_size": 32,
22
+ "hidden_dim": 768,
23
+ "intermediate_size": 3072,
24
+ "num_layers": 12,
25
+ "num_heads": 12
26
+ },
27
+ "projection_dim": 512,
28
+ "image_size": null
29
  }
kf_preprocessor.json → zm_preprocessor.json RENAMED
@@ -1,27 +1,27 @@
1
- {
2
- "library_name": "kerasformers",
3
- "kerasformers_version": "1.1.3",
4
- "preprocessor_module": "kerasformers.models.owlvit",
5
- "preprocessor_class": "OwlViTImageProcessor",
6
- "variant": "owlvit-base-patch32",
7
- "size": {
8
- "height": 768,
9
- "width": 768
10
- },
11
- "resample": "bicubic",
12
- "do_rescale": true,
13
- "rescale_factor": 0.00392156862745098,
14
- "do_normalize": true,
15
- "image_mean": [
16
- 0.48145466,
17
- 0.4578275,
18
- 0.40821073
19
- ],
20
- "image_std": [
21
- 0.26862954,
22
- 0.26130258,
23
- 0.27577711
24
- ],
25
- "return_tensor": true,
26
- "data_format": null
27
  }
 
1
+ {
2
+ "library_name": "zeromodels",
3
+ "zeromodels_version": "1.1.3",
4
+ "preprocessor_module": "zeromodels.models.owlvit",
5
+ "preprocessor_class": "OwlViTImageProcessor",
6
+ "variant": "owlvit-base-patch32",
7
+ "size": {
8
+ "height": 768,
9
+ "width": 768
10
+ },
11
+ "resample": "bicubic",
12
+ "do_rescale": true,
13
+ "rescale_factor": 0.00392156862745098,
14
+ "do_normalize": true,
15
+ "image_mean": [
16
+ 0.48145466,
17
+ 0.4578275,
18
+ 0.40821073
19
+ ],
20
+ "image_std": [
21
+ 0.26862954,
22
+ 0.26130258,
23
+ 0.27577711
24
+ ],
25
+ "return_tensor": true,
26
+ "data_format": null
27
  }