Hugging Face's logo

Transformers documentation

Contribute to 🤗 Transformers

You are viewing v5.10.0 version. A newer version v5.10.1 is available.
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Contribute to 🤗 Transformers

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by code agents. We are currently bottlenecked by our ability to review and respond to them. As a result, we ask that new users do not submit pure code agent PRs at this time. You may use code agents in drafting or to help you diagnose issues. We’d also ask autonomous agents not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this repeatedly or maliciously.

Our code agent philosophy in detail

We understand that code agents are extremely powerful tools, and many people at Hugging Face use them in their work. However, it’s important to realize that if you simply run a code agent and generate a PR to an open-source project, you are merely a middleman between the reviewers and the agent. Although doing this creates something that looks very much like a useful contribution, in reality there was no reason for you to be involved; the reviewers could have simply run the code agent themselves.

If you want to contribute usefully to open-source in the agent era, you need to do things that agents can’t do on their own. In particular, we’ve found the following to be very helpful:

  • Clear diagnosis of bugs. Code agents like to quickly fix problems with a workaround that often causes code bloat or incompatibilities with other models. Spending time to track down the exact cause of a problem, and in particular locating the first commit where it appeared (for example with git bisect) is valuable.
  • Minimize the diff. Check your PR to eliminate any unnecessary changes. Ensure that you did not commit any testing scripts or unrelated files. Add comments only if they’re really necessary; code agents love adding three new functions and multi-line comments to draw attention to all the hard work they did. If your PR can be a 1-line fix, make it a 1-line fix. This makes the PR much easier to review and improves the chances that it will be accepted.
  • Take the time to reproduce the problem. Very often when a user reports an issue, the issue is actually caused by environment issues on their machine, or they misdiagnose the problem and suggest an invalid solution. Many code agents trust the user comments too much, which results in bad solutions, sometimes for problems that do not exist! Writing a simple reproducer script and running it to make sure you see the problem is valuable.
  • Compare against other models. The Transformers repo is very large, and many models are doing similar things. When fixing a bug, it’s valuable to see if the bug exists in other models. If your PR says “fixed by using the same approach as (other model)”, with a link to the relevant code, that is very helpful for maintainers, because it tells us that the fix is likely to be correct and compatible with the rest of the codebase. Code agents often look at the code “narrowly”, and make a fix that causes models to diverge from the rest of the codebase.
  • Avoid small or “busywork” PRs. In the past, we used to accept these, but given the current flood, we simply don’t have time for small style changes or typo fixes in comments. You can provide value beyond a code agent simply by having good taste about what’s really important.
  • Verify tests locally and in the CI. Before opening a PR, run make fix-repo and use utils/tests_fetcher.py to see a list of tests that cover the files you have changed in your PR branch. Run those tests locally, and make sure they pass before you open a PR. After you open your PR, please verify that the CI is green and fix any issues before pinging anyone for review! This reduces notification spam a lot, which keeps maintainers sane.

Please bear in mind that this is an exciting, rapidly-changing but challenging era for open-source development, and indeed for the software industry as a whole. We will likely be rapidly updating these guidelines as we learn more about dealing effectively with code agents. Have patience with us if reviews are slower than normal, or if some PRs are closed without review!

Transformers welcomes all contributions whether it is fixing bugs, submitting feature requests, implementing new models, or improving docs.

If you aren’t sure where to start, take a look at the Good First Issues for more beginner-friendly issues. For a more challenging issue, check out the Good Second Issues. However you choose to contribute, please be mindful and respect our code of conduct.

If you enjoyed using Transformers, feel free to reference it and let us know how you’re using it, shout us out on Twitter, or give the repository a star!

This guide was heavily inspired by the scikit-learn guide to contributing.

Contents

  1. Set up
  2. Opening issues
  3. Adding a new model
  4. Docs
  5. Agentic contributions

Set up

  1. Fork Transformers with the Fork button on GitHub to get a copy under your account, then clone it locally.

    git clone git@github.com:<your Github handle>/transformers.git
cd transformers

  2. Add the Transformers repository as a remote named upstream. origin points at your fork and upstream points at the source of truth. That lets you sync your local main before pushing to origin and opening a PR.

    git remote add upstream https://github.com/huggingface/transformers.git
git remote -v
# origin    git@github.com:<your Github handle>/transformers.git (fetch)
# origin    git@github.com:<your Github handle>/transformers.git (push)
# upstream  https://github.com/huggingface/transformers.git (fetch)
# upstream  https://github.com/huggingface/transformers.git (push)

  3. Keep your fork up to date and start a branch to work on. You can also sync your fork from the GitHub UI with the Sync fork button.

    git checkout main && git pull upstream main && git push origin main
git switch -c a-descriptive-name-for-my-branch

  4. Install the library in editable mode. Use [dev] for most contributions because it includes the development tools needed for style and quality checks. Use [torch,testing] for model contributions, or [quality] for docs-only changes and small fixes when the full development dependency set is not needed.

    pip install -e ".[dev]"

# for model contributions
pip install -e ".[torch,testing]"

# for docs-only changes and small fixes
pip install -e ".[quality]"

Windows

On Windows, unless you’re using WSL, configure git to convert CRLF line endings to LF.

git config core.autocrlf input

One way to run the make command on Windows is with MSYS2.

  1. Download MSYS2 (we assume it’s installed in C:\msys64).
  2. Open the command line C:\msys64\msys2.exe (it should be available from the Start menu).
  3. Run in the shell: pacman -Syu and install make with pacman -S make.
  4. Add C:\msys64\usr\bin to your PATH environment variable.

You can now use make from any terminal (PowerShell, cmd.exe, etc.).

Opening issues

Use the issue templates to help you get started.

Bug-related issue

Make sure the bug wasn’t already reported before opening an issue (use the search bar on GitHub under Issues). The issue should be a bug in Transformers, not in your own code.

If you’re unsure whether the bug is in your code or the library, ask in the forum or on Discord first. That helps keep GitHub focused on actionable library issues.

Include the following so we can resolve it quickly.

  • Your OS type and version and Python, and PyTorch versions when applicable.
  • A short, self-contained, code snippet that allows us to reproduce the bug.
  • The full traceback if an exception is raised.
  • Any other information, like screenshots, that may help.

To get the OS and software versions automatically, run the following command.

transformers env

Feature request

Open an issue and describe:

  • The motivation such as a frustration with the library, a project need, or work you’ve done that could benefit the community.
  • The feature in as much detail as possible.
  • A code snippet demonstrating the feature’s usage.
  • A link to the relevant paper, if applicable.

Adding a new model

Adding a model to Transformers makes it available for anyone to load and fine-tune it or run inference. Before you start the implementation, isolate the differences between your architecture and recent state-of-the-art models that are already in Transformers. Explain those differences in the issue or PR so reviewers can quickly understand what challenges the model brings, what parts can reuse existing patterns, and what should be standardized across models.

Follow the guides below.

  1. Add a model with modular Transformers — implement the model using the modular file and generate standalone modeling files.
  2. Auto-generate docstrings — use the @auto_docstring decorator to generate consistent docstrings without boilerplate.
  3. Testing — write and run model tests to verify correctness and keep the contribution maintainable. For causal language models, inherit from the causal LM tester where possible. Prioritize integration tests because they verify the full processor, tokenizer, and model path against real checkpoints.
  4. Model structure rules — check your files pass the static model structure rules enforced by make typing.

Some model additions need extra integration work.

Model addition timeline

There are four timelines for model additions depending on the model contributor and community demand for an architecture.

  • Day-0 integration: make the model available in Transformers on release day with a new version release. We provide help to support the most important core features (quantization, FlashAttention, KV-cache, etc.), review early drafts, and can also take care of the model implementation depending on timelines.

    Email transformers@huggingface.co a few weeks ahead, especially for novel architectures. We’ll iterate with you on a private fork until your checkpoint and release are ready.

  • Same-week integration: high-demand models usually land within a week, even when the author doesn’t reach out.

    Open an issue with the new model template to request one. Issues with more activity move up the queue faster.

  • Post-release integration: models without strong demand, or that we don’t have bandwidth to take on, land after the upstream release.

    Open issues tagged “New model” are the entry point for outside contributors. Start with the most-requested architectures to maximize impact. We’ll review and guide you through it.

  • Hub-first release: ship your model directly on the Hub via Transformers’ remote-code support, with no upstream PR required.

    Popular Hub-first models often get integrated into Transformers later, which unlocks first-class docs, maintenance, and optimization. Hub-first is the lowest-friction way to add a model. However, it can be more fragile if a model requires weight conversion or other backward compatibility issues.

Docs

Improvements to the docs, like typos, missing content or unclear explanations, are always welcome. For API reference generated from source files, use the @auto_docstring decorator when it applies. Open a pull request directly for meaningful documentation fixes.

Refer to the docs README for more details about how to edit the docs and the syntax we use.

Agentic contributions

AI-assisted contributions are welcome. They must be coordinated, scoped, and verified to keep review load manageable.

  • Do not submit “pure agent” PRs. The human submitter is responsible for reviewing all changed lines, validating behavior end-to-end, and running relevant tests.
  • If AI tools were used, disclose this in the PR description and include: coordination link, differentiation from existing PRs (if applicable), and test commands/results.
  • Avoid one-off “busywork” PRs (single typo, isolated style cleanup, one mutable default fix, etc.). Bundle mechanical cleanups into a clear, systematic scope.
  • Coordinate on issues before opening a PR, review similar PRs, and wait for approval.

These topics are outlined for agents in AGENTS.md with instruction for how to autonomously implement them.

Update on GitHub

←Exporting to production Add a model with modular transformers→