Instructions to use Synthyra/ESMplusplus_large with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Synthyra/ESMplusplus_large with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="Synthyra/ESMplusplus_large", trust_remote_code=True)# Load model directly from transformers import AutoModelForMaskedLM model = AutoModelForMaskedLM.from_pretrained("Synthyra/ESMplusplus_large", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Improve generated model cards
Browse filesAdd-only FastPLMs files-only publication. Checkpoint weights and complete-artifact attestations are unchanged.
README.md
CHANGED
|
@@ -8,28 +8,21 @@ tags:
|
|
| 8 |
|
| 9 |
<!-- Generated from src/fastplms/models.toml. Do not edit. -->
|
| 10 |
|
| 11 |
-
#
|
| 12 |
|
| 13 |
-
|
| 14 |
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
`AutoModelForTokenClassification`.
|
| 19 |
|
| 20 |
-
|
|
|
|
|
|
|
| 21 |
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
| Token classification | Supported: base weights with an untrained task head |
|
| 26 |
-
| PEFT fine-tuning | Supported pattern: preserve the separately trained `classifier` |
|
| 27 |
-
| Embeddings | Supported: shared ordered embedding API |
|
| 28 |
-
| Test-time training | Supported: low-rank masked-residue adaptation |
|
| 29 |
-
| Attention variants | Special: SDPA fidelity path; alternate backends have explicit bands |
|
| 30 |
-
| Compliance | Declared: exact release evidence is required |
|
| 31 |
-
|
| 32 |
-
A supported interface is not a pretrained downstream predictor. Classification heads start untrained. Compliance metadata does not show that a local build passed its release gate.
|
| 33 |
|
| 34 |
## Install and platform requirements
|
| 35 |
|
|
@@ -43,9 +36,14 @@ python -m pip install -r \
|
|
| 43 |
The FastPLMs implementation itself is embedded in the model repository.
|
| 44 |
Transformers loads it through `trust_remote_code=True`.
|
| 45 |
|
| 46 |
-
This model requires Python 3.11-3.14, PyTorch 2.13, and Transformers 5.13.
|
| 47 |
-
|
| 48 |
-
artifact
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
|
| 50 |
## Quick start
|
| 51 |
|
|
@@ -63,17 +61,17 @@ model = AutoModel.from_pretrained(
|
|
| 63 |
For offline validation, replace `model_id` with the manifest-built
|
| 64 |
`dist/hub/ESMplusplus_large` path. Pass `local_files_only=True`.
|
| 65 |
|
| 66 |
-
## Attention
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
|
| 68 |
-
The quick start selects `sdpa` explicitly. Declared variants are `eager`, `sdpa`, `flex_attention`, `flash_attention_2`,
|
| 69 |
-
`flash_attention_3`. An unavailable requested backend raises. It does not
|
| 70 |
-
silently change implementation.
|
| 71 |
`output_attentions=True` can use the documented one-call eager fallback to
|
| 72 |
materialize attention tensors. The configured backend does not change.
|
| 73 |
|
| 74 |
-
This family declares the `compliance` tier. Release evidence identifies the
|
| 75 |
-
checkpoint, backend, dtype, hardware, inputs, and reference revision.
|
| 76 |
-
|
| 77 |
## Tokenization and forward inference
|
| 78 |
|
| 79 |
Load the tokenizer from the same artifact as the model. The attention mask
|
|
@@ -308,24 +306,28 @@ and
|
|
| 308 |
[release evidence manifest](https://github.com/Synthyra/FastPLMs/blob/main/docs/generated/capability_evidence.md).
|
| 309 |
|
| 310 |
|
| 311 |
-
##
|
| 312 |
|
| 313 |
-
-
|
| 314 |
-
-
|
| 315 |
-
-
|
| 316 |
-
- Attention
|
| 317 |
-
- Precision
|
| 318 |
- BF16 execution: `static_parameters`
|
| 319 |
- Generation contract: `not_applicable`
|
| 320 |
-
-
|
| 321 |
- Weight publication allowed: `true`
|
| 322 |
- Weight license status: `resolved`
|
| 323 |
- Redistributable: `true`
|
| 324 |
- Complete weight publication required: `false`
|
| 325 |
|
| 326 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
| 327 |
|
| 328 |
-
- FastPLMs
|
| 329 |
- Runtime revision: recorded separately in the built artifact and published commit
|
| 330 |
- Runtime source identities: recorded in `source-record.json`
|
| 331 |
- Official checkpoint: `biohub/ESMC-600M`
|
|
@@ -335,14 +337,13 @@ and
|
|
| 335 |
- Release tiers: `check`, `compliance`, `feature`, `artifact`, `benchmark`
|
| 336 |
- Unresolved required file identities: `0`
|
| 337 |
|
| 338 |
-
|
| 339 |
-
|
| 340 |
-
|
| 341 |
-
## Validation boundary
|
| 342 |
|
| 343 |
Declared tiers compare configuration, tokenizer behavior, state, and
|
| 344 |
-
representative inference with the pinned reference.
|
| 345 |
-
|
|
|
|
| 346 |
|
| 347 |
## License
|
| 348 |
|
|
|
|
| 8 |
|
| 9 |
<!-- Generated from src/fastplms/models.toml. Do not edit. -->
|
| 10 |
|
| 11 |
+
# ESM++ Large
|
| 12 |
|
| 13 |
+
## Model overview
|
| 14 |
|
| 15 |
+
`Synthyra/ESMplusplus_large` packages the `biohub/ESMC-600M` checkpoint with
|
| 16 |
+
the FastPLMs runtime for Hugging Face Transformers. It accepts amino-acid
|
| 17 |
+
sequences tokenized to residue IDs.
|
|
|
|
| 18 |
|
| 19 |
+
The repository uses the standard Transformers loading interface with
|
| 20 |
+
`trust_remote_code=True`. See Technical details for each registered class and
|
| 21 |
+
whether its weights come from the checkpoint.
|
| 22 |
|
| 23 |
+
The sequence- and token-classification classes reuse the pretrained backbone,
|
| 24 |
+
but their task heads are newly initialized. Fine-tune those heads before
|
| 25 |
+
interpreting their logits as predictions.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
|
| 27 |
## Install and platform requirements
|
| 28 |
|
|
|
|
| 36 |
The FastPLMs implementation itself is embedded in the model repository.
|
| 37 |
Transformers loads it through `trust_remote_code=True`.
|
| 38 |
|
| 39 |
+
This model requires Python 3.11-3.14, PyTorch 2.13, and Transformers 5.13.
|
| 40 |
+
|
| 41 |
+
The artifact requirements include the FlashAttention loader dependency.
|
| 42 |
+
FlashAttention also requires compatible CUDA hardware and BF16 execution.
|
| 43 |
+
|
| 44 |
+
The Hub quick start needs network access for the first download. For an
|
| 45 |
+
air-gapped run, build the manifest-pinned local artifact first and use the
|
| 46 |
+
offline example.
|
| 47 |
|
| 48 |
## Quick start
|
| 49 |
|
|
|
|
| 61 |
For offline validation, replace `model_id` with the manifest-built
|
| 62 |
`dist/hub/ESMplusplus_large` path. Pass `local_files_only=True`.
|
| 63 |
|
| 64 |
+
## Attention backends
|
| 65 |
+
|
| 66 |
+
The quick start uses `sdpa`.
|
| 67 |
+
|
| 68 |
+
Available backends are `eager`, `sdpa`, `flex_attention`, `flash_attention_2`,
|
| 69 |
+
`flash_attention_3`. Requesting an unavailable backend raises instead of
|
| 70 |
+
silently changing implementation.
|
| 71 |
|
|
|
|
|
|
|
|
|
|
| 72 |
`output_attentions=True` can use the documented one-call eager fallback to
|
| 73 |
materialize attention tensors. The configured backend does not change.
|
| 74 |
|
|
|
|
|
|
|
|
|
|
| 75 |
## Tokenization and forward inference
|
| 76 |
|
| 77 |
Load the tokenizer from the same artifact as the model. The attention mask
|
|
|
|
| 306 |
[release evidence manifest](https://github.com/Synthyra/FastPLMs/blob/main/docs/generated/capability_evidence.md).
|
| 307 |
|
| 308 |
|
| 309 |
+
## Technical details
|
| 310 |
|
| 311 |
+
- Inputs: Amino-acid sequences tokenized to residue IDs
|
| 312 |
+
- Transformers classes: `AutoConfig`, `AutoModel`, `AutoModelForMaskedLM`, `AutoModelForSequenceClassification`, `AutoModelForTokenClassification`
|
| 313 |
+
- Checkpoint weights: `AutoConfig` = `FastPLMs extension`, `AutoModel` = `pretrained`, `AutoModelForMaskedLM` = `pretrained`, `AutoModelForSequenceClassification` = `base weights + untrained task head`, `AutoModelForTokenClassification` = `base weights + untrained task head`
|
| 314 |
+
- Attention backends: `eager`, `sdpa`, `flex_attention`, `flash_attention_2`, `flash_attention_3`
|
| 315 |
+
- Precision: `default`, `fp8` (experimental)
|
| 316 |
- BF16 execution: `static_parameters`
|
| 317 |
- Generation contract: `not_applicable`
|
| 318 |
+
- Dependencies: `core`
|
| 319 |
- Weight publication allowed: `true`
|
| 320 |
- Weight license status: `resolved`
|
| 321 |
- Redistributable: `true`
|
| 322 |
- Complete weight publication required: `false`
|
| 323 |
|
| 324 |
+
## Validation and provenance
|
| 325 |
+
|
| 326 |
+
FastPLMs pins the checkpoint, upstream source revisions, state transformation,
|
| 327 |
+
and required files in `models.toml`. Built artifacts record exact source
|
| 328 |
+
identities and conversion details in `source-record.json`.
|
| 329 |
|
| 330 |
+
- FastPLMs checkpoint: `Synthyra/ESMplusplus_large`
|
| 331 |
- Runtime revision: recorded separately in the built artifact and published commit
|
| 332 |
- Runtime source identities: recorded in `source-record.json`
|
| 333 |
- Official checkpoint: `biohub/ESMC-600M`
|
|
|
|
| 337 |
- Release tiers: `check`, `compliance`, `feature`, `artifact`, `benchmark`
|
| 338 |
- Unresolved required file identities: `0`
|
| 339 |
|
| 340 |
+
Release validation includes the `compliance` tier. Its evidence identifies the
|
| 341 |
+
checkpoint, backend, dtype, hardware, inputs, and reference revision.
|
|
|
|
|
|
|
| 342 |
|
| 343 |
Declared tiers compare configuration, tokenizer behavior, state, and
|
| 344 |
+
representative inference with the pinned reference. A nonzero unresolved count
|
| 345 |
+
blocks release. Metadata alone does not show that a build passed, that a backend
|
| 346 |
+
is faster, or that an output is biologically valid.
|
| 347 |
|
| 348 |
## License
|
| 349 |
|