IMvision12 commited on
Commit
24db011
·
verified ·
1 Parent(s): 08d783d

Migrate to zeromodels (rename kf_*.json -> zm_*.json, fix refs in config + README, ensure tag + badge)

Browse files
Files changed (2) hide show
  1. README.md +23 -23
  2. kf_config.json → zm_config.json +26 -26
README.md CHANGED
@@ -2,10 +2,10 @@
2
  pipeline_tag: fill-mask
3
  license: mit
4
  base_model: microsoft/deberta-base
5
- library_name: kerasformers
6
  tags:
7
  - keras
8
- - kerasformers
9
  - deberta
10
  - deberta-v1
11
  - fill-mask
@@ -17,13 +17,13 @@ tags:
17
  - tf
18
  ---
19
 
20
- ## ***See [our collection](https://huggingface.co/collections/kerasformers/deberta-v1-v2-v3-6a6e90bac01e412b478562f3) for all versions of DeBERTa (v1 / v2 / v3).***
21
 
22
  # Run DeBERTa with Keras 3: JAX, PyTorch, or TensorFlow
23
 
24
- [![GitHub](https://img.shields.io/badge/GitHub-KerasFormers-black?logo=github)](https://github.com/IMvision12/KerasFormers) [![Docs](https://img.shields.io/badge/Docs-DeBERTa-blue)](https://imvision12.github.io/KerasFormers/deberta/) [![Collection](https://img.shields.io/badge/HF-DeBERTa%20collection-yellow)](https://huggingface.co/collections/kerasformers/deberta-v1-v2-v3-6a6e90bac01e412b478562f3)
25
 
26
- # kerasformers/deberta_base
27
 
28
  Papers: [DeBERTa: Decoding-enhanced BERT with Disentangled Attention (arXiv:2006.03654)](https://arxiv.org/abs/2006.03654) · [DeBERTaV3 (arXiv:2111.09543)](https://arxiv.org/abs/2111.09543) · [HF Papers](https://huggingface.co/papers/2006.03654)
29
 
@@ -31,7 +31,7 @@ DeBERTa is Microsoft's disentangled-attention text encoder (content + relative p
31
 
32
  For more details on the model, please go to the upstream [model card](https://huggingface.co/microsoft/deberta-base).
33
 
34
- Pure-**Keras 3** conversion of [`microsoft/deberta-base`](https://huggingface.co/microsoft/deberta-base) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
35
 
36
  This is a **fill-mask / encoder** checkpoint (`DebertaMaskedLM`, v1 base). Task heads (sequence/token classify, QA, …) load via `hf:` fine-tunes.
37
 
@@ -41,13 +41,13 @@ This is a **fill-mask / encoder** checkpoint (`DebertaMaskedLM`, v1 base). Task
41
  import os
42
  os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
43
 
44
- from kerasformers.models.deberta import (
45
  DebertaMaskedLM,
46
  DebertaTokenizer,
47
  )
48
 
49
- mlm = DebertaMaskedLM.from_weights("kerasformers/deberta_base")
50
- tokenizer = DebertaTokenizer.from_weights("kerasformers/deberta_base")
51
 
52
  inputs = tokenizer("The capital of France is [MASK].")
53
  logits = mlm(inputs) # (1, L, vocab_size)
@@ -55,22 +55,22 @@ mask = int((inputs["input_ids"][0] == tokenizer.mask_token_id).argmax())
55
  print(tokenizer.decode([int(logits[0, mask].argmax())]))
56
  ```
57
 
58
- Load any DeBERTa variant the same way with `from_weights("kerasformers/<variant>")`:
59
 
60
  | Variant | Hub | Generation |
61
  |---|---|---|
62
- | `deberta_base` | [`kerasformers/deberta_base`](https://huggingface.co/kerasformers/deberta_base) | v1 |
63
- | `deberta_large` | [`kerasformers/deberta_large`](https://huggingface.co/kerasformers/deberta_large) | v1 |
64
- | `deberta_v2_xlarge` | [`kerasformers/deberta_v2_xlarge`](https://huggingface.co/kerasformers/deberta_v2_xlarge) | v2 |
65
- | `deberta_v2_xxlarge` | [`kerasformers/deberta_v2_xxlarge`](https://huggingface.co/kerasformers/deberta_v2_xxlarge) | v2 |
66
- | `deberta_v3_xsmall` | [`kerasformers/deberta_v3_xsmall`](https://huggingface.co/kerasformers/deberta_v3_xsmall) | v3 |
67
- | `deberta_v3_small` | [`kerasformers/deberta_v3_small`](https://huggingface.co/kerasformers/deberta_v3_small) | v3 |
68
- | `deberta_v3_base` | [`kerasformers/deberta_v3_base`](https://huggingface.co/kerasformers/deberta_v3_base) | v3 |
69
- | `deberta_v3_large` | [`kerasformers/deberta_v3_large`](https://huggingface.co/kerasformers/deberta_v3_large) | v3 |
70
 
71
  ## Available classes
72
 
73
- Load any of these from this repo with `from_weights("kerasformers/deberta_base")` (or on the fly via the `hf:` prefix). The pretrained backbone is shared; task heads not stored in this checkpoint start randomly initialized, ready for fine-tuning (or load a `hf:` fine-tune).
74
 
75
  | Class | Task |
76
  |---|---|
@@ -81,16 +81,16 @@ Load any of these from this repo with `from_weights("kerasformers/deberta_base")
81
  | `DebertaQnA` | Extractive question answering |
82
 
83
  ```python
84
- from kerasformers.models.deberta import DebertaSequenceClassify
85
- model = DebertaSequenceClassify.from_weights("kerasformers/deberta_base")
86
  ```
87
 
88
  ## Tips
89
 
90
- - Set `KERAS_BACKEND` **before** importing Keras / kerasformers.
91
  - Prefer `Tokenizer.from_weights(...)` so vocab and mask token match.
92
  - Do not mix packages across generations (v1 ≠ v2 ≠ v3).
93
- - See [DeBERTa docs](https://imvision12.github.io/KerasFormers/deberta/) and [Loading Weights](https://imvision12.github.io/KerasFormers/loading_weights/).
94
  - Community / upstream safetensors still work via the `hf:` prefix, e.g. `DebertaMaskedLM.from_weights("hf:microsoft/deberta-base")`.
95
 
96
  ## Special Thanks
 
2
  pipeline_tag: fill-mask
3
  license: mit
4
  base_model: microsoft/deberta-base
5
+ library_name: zeromodels
6
  tags:
7
  - keras
8
+ - zeromodels
9
  - deberta
10
  - deberta-v1
11
  - fill-mask
 
17
  - tf
18
  ---
19
 
20
+ ## ***See [our collection](https://huggingface.co/collections/zeromodels/deberta-v1-v2-v3-6a6e90bac01e412b478562f3) for all versions of DeBERTa (v1 / v2 / v3).***
21
 
22
  # Run DeBERTa with Keras 3: JAX, PyTorch, or TensorFlow
23
 
24
+ [![GitHub](https://img.shields.io/badge/GitHub-ZeroModels-black?logo=github)](https://github.com/IMvision12/ZeroModels) [![Docs](https://img.shields.io/badge/Docs-DeBERTa-blue)](https://imvision12.github.io/ZeroModels/deberta/) [![Collection](https://img.shields.io/badge/HF-DeBERTa%20collection-yellow)](https://huggingface.co/collections/zeromodels/deberta-v1-v2-v3-6a6e90bac01e412b478562f3)
25
 
26
+ # zeromodels/deberta_base
27
 
28
  Papers: [DeBERTa: Decoding-enhanced BERT with Disentangled Attention (arXiv:2006.03654)](https://arxiv.org/abs/2006.03654) · [DeBERTaV3 (arXiv:2111.09543)](https://arxiv.org/abs/2111.09543) · [HF Papers](https://huggingface.co/papers/2006.03654)
29
 
 
31
 
32
  For more details on the model, please go to the upstream [model card](https://huggingface.co/microsoft/deberta-base).
33
 
34
+ Pure-**Keras 3** conversion of [`microsoft/deberta-base`](https://huggingface.co/microsoft/deberta-base) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
35
 
36
  This is a **fill-mask / encoder** checkpoint (`DebertaMaskedLM`, v1 base). Task heads (sequence/token classify, QA, …) load via `hf:` fine-tunes.
37
 
 
41
  import os
42
  os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
43
 
44
+ from zeromodels.models.deberta import (
45
  DebertaMaskedLM,
46
  DebertaTokenizer,
47
  )
48
 
49
+ mlm = DebertaMaskedLM.from_weights("zeromodels/deberta_base")
50
+ tokenizer = DebertaTokenizer.from_weights("zeromodels/deberta_base")
51
 
52
  inputs = tokenizer("The capital of France is [MASK].")
53
  logits = mlm(inputs) # (1, L, vocab_size)
 
55
  print(tokenizer.decode([int(logits[0, mask].argmax())]))
56
  ```
57
 
58
+ Load any DeBERTa variant the same way with `from_weights("zeromodels/<variant>")`:
59
 
60
  | Variant | Hub | Generation |
61
  |---|---|---|
62
+ | `deberta_base` | [`zeromodels/deberta_base`](https://huggingface.co/zeromodels/deberta_base) | v1 |
63
+ | `deberta_large` | [`zeromodels/deberta_large`](https://huggingface.co/zeromodels/deberta_large) | v1 |
64
+ | `deberta_v2_xlarge` | [`zeromodels/deberta_v2_xlarge`](https://huggingface.co/zeromodels/deberta_v2_xlarge) | v2 |
65
+ | `deberta_v2_xxlarge` | [`zeromodels/deberta_v2_xxlarge`](https://huggingface.co/zeromodels/deberta_v2_xxlarge) | v2 |
66
+ | `deberta_v3_xsmall` | [`zeromodels/deberta_v3_xsmall`](https://huggingface.co/zeromodels/deberta_v3_xsmall) | v3 |
67
+ | `deberta_v3_small` | [`zeromodels/deberta_v3_small`](https://huggingface.co/zeromodels/deberta_v3_small) | v3 |
68
+ | `deberta_v3_base` | [`zeromodels/deberta_v3_base`](https://huggingface.co/zeromodels/deberta_v3_base) | v3 |
69
+ | `deberta_v3_large` | [`zeromodels/deberta_v3_large`](https://huggingface.co/zeromodels/deberta_v3_large) | v3 |
70
 
71
  ## Available classes
72
 
73
+ Load any of these from this repo with `from_weights("zeromodels/deberta_base")` (or on the fly via the `hf:` prefix). The pretrained backbone is shared; task heads not stored in this checkpoint start randomly initialized, ready for fine-tuning (or load a `hf:` fine-tune).
74
 
75
  | Class | Task |
76
  |---|---|
 
81
  | `DebertaQnA` | Extractive question answering |
82
 
83
  ```python
84
+ from zeromodels.models.deberta import DebertaSequenceClassify
85
+ model = DebertaSequenceClassify.from_weights("zeromodels/deberta_base")
86
  ```
87
 
88
  ## Tips
89
 
90
+ - Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
91
  - Prefer `Tokenizer.from_weights(...)` so vocab and mask token match.
92
  - Do not mix packages across generations (v1 ≠ v2 ≠ v3).
93
+ - See [DeBERTa docs](https://imvision12.github.io/ZeroModels/deberta/) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
94
  - Community / upstream safetensors still work via the `hf:` prefix, e.g. `DebertaMaskedLM.from_weights("hf:microsoft/deberta-base")`.
95
 
96
  ## Special Thanks
kf_config.json → zm_config.json RENAMED
@@ -1,27 +1,27 @@
1
- {
2
- "library_name": "kerasformers",
3
- "kerasformers_version": "1.2.1",
4
- "model_module": "kerasformers.models.deberta",
5
- "model_class": "DebertaModel",
6
- "variant": "deberta_base",
7
- "weights": "model.weights.h5",
8
- "schema_version": 2,
9
- "weight_dtype": "float32",
10
- "model_type": "deberta",
11
- "text_config": {
12
- "vocab_size": 50265,
13
- "embed_dim": 768,
14
- "num_layers": 12,
15
- "num_heads": 12,
16
- "mlp_dim": 3072,
17
- "max_position_embeddings": 512,
18
- "max_relative_positions": 512,
19
- "pos_att_type": [
20
- "c2p",
21
- "p2c"
22
- ],
23
- "hidden_act": "gelu",
24
- "layer_norm_eps": 1e-07,
25
- "pad_token_id": 0
26
- }
27
  }
 
1
+ {
2
+ "library_name": "zeromodels",
3
+ "zeromodels_version": "1.2.1",
4
+ "model_module": "zeromodels.models.deberta",
5
+ "model_class": "DebertaModel",
6
+ "variant": "deberta_base",
7
+ "weights": "model.weights.h5",
8
+ "schema_version": 2,
9
+ "weight_dtype": "float32",
10
+ "model_type": "deberta",
11
+ "text_config": {
12
+ "vocab_size": 50265,
13
+ "embed_dim": 768,
14
+ "num_layers": 12,
15
+ "num_heads": 12,
16
+ "mlp_dim": 3072,
17
+ "max_position_embeddings": 512,
18
+ "max_relative_positions": 512,
19
+ "pos_att_type": [
20
+ "c2p",
21
+ "p2c"
22
+ ],
23
+ "hidden_act": "gelu",
24
+ "layer_norm_eps": 1e-07,
25
+ "pad_token_id": 0
26
+ }
27
  }