See our collection for all versions of Next-ViT.

Run Next-ViT with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/nextvit_base_bd_in1k

Paper: Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarios (arXiv:2207.05501) · HF Papers

Next-ViT targets efficient industrial deployment with a hybrid CNN/Transformer stack. Classifier or hierarchical backbone.

For more details on the model, please go to the upstream model card.

Pure-Keras 3 conversion of timm/nextvit_base.bd_in1k for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is an image-classification / backbone checkpoint (NextViTImageClassify / NextViTModel).

✨ Quick start

import os

os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from zeromodels.models.nextvit import NextViTImageClassify, NextViTModel, NextViTImageProcessor

model = NextViTImageClassify.from_weights("zeromodels/nextvit_base_bd_in1k")
processor = NextViTImageProcessor.from_weights("zeromodels/nextvit_base_bd_in1k")

image = Image.open("your_image.jpg").convert("RGB")
pixels = processor(image)  # resize + normalize (normalization lives in the processor)
logits = model(pixels, training=False)
print(logits.shape)  # (1, num_classes)

# Feature extraction: the backbone without the classifier head
backbone = NextViTModel.from_weights("zeromodels/nextvit_base_bd_in1k", as_backbone=True)
features = backbone(pixels, training=False)

Load any Next-ViT variant the same way with from_weights("zeromodels/<variant>"):

Variant	Hub
`nextvit_base_bd_in1k`	`zeromodels/nextvit_base_bd_in1k`
`nextvit_base_bd_in1k_384`	`zeromodels/nextvit_base_bd_in1k_384`
`nextvit_base_bd_ssld_6m_in1k`	`zeromodels/nextvit_base_bd_ssld_6m_in1k`
`nextvit_base_bd_ssld_6m_in1k_384`	`zeromodels/nextvit_base_bd_ssld_6m_in1k_384`
`nextvit_large_bd_in1k`	`zeromodels/nextvit_large_bd_in1k`
`nextvit_large_bd_in1k_384`	`zeromodels/nextvit_large_bd_in1k_384`
`nextvit_large_bd_ssld_6m_in1k`	`zeromodels/nextvit_large_bd_ssld_6m_in1k`
`nextvit_large_bd_ssld_6m_in1k_384`	`zeromodels/nextvit_large_bd_ssld_6m_in1k_384`
`nextvit_small_bd_in1k`	`zeromodels/nextvit_small_bd_in1k`
`nextvit_small_bd_in1k_384`	`zeromodels/nextvit_small_bd_in1k_384`
`nextvit_small_bd_ssld_6m_in1k`	`zeromodels/nextvit_small_bd_ssld_6m_in1k`
`nextvit_small_bd_ssld_6m_in1k_384`	`zeromodels/nextvit_small_bd_ssld_6m_in1k_384`

Tips

Set KERAS_BACKEND before importing Keras / zeromodels.
NextViTImageClassify returns class logits; NextViTModel returns features (as_backbone=True for multi-scale stages).
See docs and Loading Weights.
Upstream / timm checkpoints: NextViTImageClassify.from_weights("hf:timm/nextvit_base.bd_in1k").