Hugging Face's logo

Buckets:
hf-doc-build
/
doc

Files
xet
hf-doc-build/doc / diffusers /main /en /using-diffusers /using_safetensors.html
rtrm's picture
rtrm
download
raw
21.1 kB
<meta charset="utf-8" /><meta name="hf:doc:metadata" content="{&quot;title&quot;:&quot;Load safetensors&quot;,&quot;local&quot;:&quot;load-safetensors&quot;,&quot;sections&quot;:[{&quot;title&quot;:&quot;Convert to safetensors&quot;,&quot;local&quot;:&quot;convert-to-safetensors&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;Why use safetensors?&quot;,&quot;local&quot;:&quot;why-use-safetensors&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2}],&quot;depth&quot;:1}">
 <link href="/docs/diffusers/main/en/_app/immutable/assets/0.e3b0c442.css" rel="modulepreload">
 <link rel="modulepreload" href="/docs/diffusers/main/en/_app/immutable/entry/start.523ae648.js">
 <link rel="modulepreload" href="/docs/diffusers/main/en/_app/immutable/chunks/scheduler.8c3d61f6.js">
 <link rel="modulepreload" href="/docs/diffusers/main/en/_app/immutable/chunks/singletons.28a910fe.js">
 <link rel="modulepreload" href="/docs/diffusers/main/en/_app/immutable/chunks/index.0997d446.js">
 <link rel="modulepreload" href="/docs/diffusers/main/en/_app/immutable/chunks/paths.68e3a669.js">
 <link rel="modulepreload" href="/docs/diffusers/main/en/_app/immutable/entry/app.4a6d659a.js">
 <link rel="modulepreload" href="/docs/diffusers/main/en/_app/immutable/chunks/index.589a98e8.js">
 <link rel="modulepreload" href="/docs/diffusers/main/en/_app/immutable/nodes/0.7a153d85.js">
 <link rel="modulepreload" href="/docs/diffusers/main/en/_app/immutable/chunks/each.e59479a4.js">
 <link rel="modulepreload" href="/docs/diffusers/main/en/_app/immutable/nodes/202.9791c202.js">
 <link rel="modulepreload" href="/docs/diffusers/main/en/_app/immutable/chunks/CodeBlock.36627b28.js">
 <link rel="modulepreload" href="/docs/diffusers/main/en/_app/immutable/chunks/DocNotebookDropdown.108e4998.js">
 <link rel="modulepreload" href="/docs/diffusers/main/en/_app/immutable/chunks/globals.7f7f1b26.js">
 <link rel="modulepreload" href="/docs/diffusers/main/en/_app/immutable/chunks/EditOnGithub.e5a8d9cb.js"><!-- HEAD_svelte-u9bgzb_START --><meta name="hf:doc:metadata" content="{&quot;title&quot;:&quot;Load safetensors&quot;,&quot;local&quot;:&quot;load-safetensors&quot;,&quot;sections&quot;:[{&quot;title&quot;:&quot;Convert to safetensors&quot;,&quot;local&quot;:&quot;convert-to-safetensors&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;Why use safetensors?&quot;,&quot;local&quot;:&quot;why-use-safetensors&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2}],&quot;depth&quot;:1}"><!-- HEAD_svelte-u9bgzb_END --> <p></p> <h1 class="relative group"><a id="load-safetensors" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#load-safetensors"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Load safetensors</span></h1> <div class="flex space-x-1 absolute z-10 right-0 top-0"> <div class="relative colab-dropdown "> <button class=" " type="button"> <img alt="Open In Colab" class="!m-0" src="https://colab.research.google.com/assets/colab-badge.svg"> </button> </div> <div class="relative colab-dropdown "> <button class=" " type="button"> <img alt="Open In Studio Lab" class="!m-0" src="https://studiolab.sagemaker.aws/studiolab.svg"> </button> </div></div> <p data-svelte-h="svelte-ejaj9i"><a href="https://github.com/huggingface/safetensors" rel="nofollow">safetensors</a> is a safe and fast file format for storing and loading tensors. Typically, PyTorch model weights are saved or <em>pickled</em> into a <code>.bin</code> file with Python’s <a href="https://docs.python.org/3/library/pickle.html" rel="nofollow"><code>pickle</code></a> utility. However, <code>pickle</code> is not secure and pickled files may contain malicious code that can be executed. safetensors is a secure alternative to <code>pickle</code>, making it ideal for sharing model weights.</p> <p data-svelte-h="svelte-19452ph">This guide will show you how you load <code>.safetensor</code> files, and how to convert Stable Diffusion model weights stored in other formats to <code>.safetensor</code>. Before you start, make sure you have safetensors installed:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-comment"># uncomment to install the necessary libraries in Colab</span>
<span class="hljs-comment">#!pip install safetensors</span><!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1sm59ne">If you look at the <a href="https://huggingface.co/runwayml/stable-diffusion-v1-5/tree/main" rel="nofollow"><code>runwayml/stable-diffusion-v1-5</code></a> repository, you’ll see weights inside the <code>text_encoder</code>, <code>unet</code> and <code>vae</code> subfolders are stored in the <code>.safetensors</code> format. By default, 🤗 Diffusers automatically loads these <code>.safetensors</code> files from their subfolders if they’re available in the model repository.</p> <p data-svelte-h="svelte-1qgf6oc">For more explicit control, you can optionally set <code>use_safetensors=True</code> (if <code>safetensors</code> is not installed, you’ll get an error message asking you to install it):</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> DiffusionPipeline
pipeline = DiffusionPipeline.from_pretrained(<span class="hljs-string">&quot;runwayml/stable-diffusion-v1-5&quot;</span>, use_safetensors=<span class="hljs-literal">True</span>)<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1atxmeh">However, model weights are not necessarily stored in separate subfolders like in the example above. Sometimes, all the weights are stored in a single <code>.safetensors</code> file. In this case, if the weights are Stable Diffusion weights, you can load the file directly with the <a href="/docs/diffusers/main/en/api/loaders/single_file#diffusers.loaders.FromSingleFileMixin.from_single_file">from_single_file()</a> method:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> StableDiffusionPipeline
pipeline = StableDiffusionPipeline.from_single_file(
 <span class="hljs-string">&quot;https://huggingface.co/WarriorMama777/OrangeMixs/blob/main/Models/AbyssOrangeMix/AbyssOrangeMix.safetensors&quot;</span>
)<!-- HTML_TAG_END --></pre></div> <h2 class="relative group"><a id="convert-to-safetensors" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#convert-to-safetensors"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Convert to safetensors</span></h2> <p data-svelte-h="svelte-i3hl9t">Not all weights on the Hub are available in the <code>.safetensors</code> format, and you may encounter weights stored as <code>.bin</code>. In this case, use the <a href="https://huggingface.co/spaces/diffusers/convert" rel="nofollow">Convert Space</a> to convert the weights to <code>.safetensors</code>. The Convert Space downloads the pickled weights, converts them, and opens a Pull Request to upload the newly converted <code>.safetensors</code> file on the Hub. This way, if there is any malicious code contained in the pickled files, they’re uploaded to the Hub - which has a <a href="https://huggingface.co/docs/hub/security-pickle#hubs-security-scanner" rel="nofollow">security scanner</a> to detect unsafe files and suspicious pickle imports - instead of your computer.</p> <p data-svelte-h="svelte-uqf6y9">You can use the model with the new <code>.safetensors</code> weights by specifying the reference to the Pull Request in the <code>revision</code> parameter (you can also test it in this <a href="https://huggingface.co/spaces/diffusers/check_pr" rel="nofollow">Check PR</a> Space on the Hub), for example <code>refs/pr/22</code>:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> DiffusionPipeline
pipeline = DiffusionPipeline.from_pretrained(
 <span class="hljs-string">&quot;stabilityai/stable-diffusion-2-1&quot;</span>, revision=<span class="hljs-string">&quot;refs/pr/22&quot;</span>, use_safetensors=<span class="hljs-literal">True</span>
)<!-- HTML_TAG_END --></pre></div> <h2 class="relative group"><a id="why-use-safetensors" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#why-use-safetensors"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Why use safetensors?</span></h2> <p data-svelte-h="svelte-kqqg29">There are several reasons for using safetensors:</p> <ul><li data-svelte-h="svelte-1ci1eo5"><p>Safety is the number one reason for using safetensors. As open-source and model distribution grows, it is important to be able to trust the model weights you downloaded don’t contain any malicious code. The current size of the header in safetensors prevents parsing extremely large JSON files.</p></li> <li><p data-svelte-h="svelte-x9zzp2">Loading speed between switching models is another reason to use safetensors, which performs zero-copy of the tensors. It is especially fast compared to <code>pickle</code> if you’re loading the weights to CPU (the default case), and just as fast if not faster when directly loading the weights to GPU. You’ll only notice the performance difference if the model is already loaded, and not if you’re downloading the weights or loading the model for the first time.</p> <p data-svelte-h="svelte-gqu57m">The time it takes to load the entire pipeline:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> StableDiffusionPipeline
pipeline = StableDiffusionPipeline.from_pretrained(<span class="hljs-string">&quot;stabilityai/stable-diffusion-2-1&quot;</span>, use_safetensors=<span class="hljs-literal">True</span>)
<span class="hljs-string">&quot;Loaded in safetensors 0:00:02.033658&quot;</span>
<span class="hljs-string">&quot;Loaded in PyTorch 0:00:02.663379&quot;</span><!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-15nsa67">But the actual time it takes to load 500MB of the model weights is only:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->safetensors: 3.4873ms
PyTorch: 172.7537ms<!-- HTML_TAG_END --></pre></div></li> <li data-svelte-h="svelte-bvwr8d"><p>Lazy loading is also supported in safetensors, which is useful in distributed settings to only load some of the tensors. This format allowed the <a href="https://huggingface.co/bigscience/bloom" rel="nofollow">BLOOM</a> model to be loaded in 45 seconds on 8 GPUs instead of 10 minutes with regular PyTorch weights.</p></li></ul> <a class="!text-gray-400 !no-underline text-sm flex items-center not-prose mt-4" href="https://github.com/huggingface/diffusers/blob/main/docs/source/en/using-diffusers/using_safetensors.md" target="_blank"><span data-svelte-h="svelte-1kd6by1">&lt;</span> <span data-svelte-h="svelte-x0xyl0">&gt;</span> <span data-svelte-h="svelte-1dajgef"><span class="underline ml-1.5">Update</span> on GitHub</span></a> <p></p>
<script>
 {
 __sveltekit_f2rkda = {
 assets: "/docs/diffusers/main/en",
 base: "/docs/diffusers/main/en",
 env: {}
 };
 const element = document.currentScript.parentElement;
 const data = [null,null];
 Promise.all([
 import("/docs/diffusers/main/en/_app/immutable/entry/start.523ae648.js"),
 import("/docs/diffusers/main/en/_app/immutable/entry/app.4a6d659a.js")
 ]).then(([kit, app]) => {
 kit.start(app, element, {
 node_ids: [0, 202],
 data,
 form: null,
 error: null
 });
 });
 }
 </script>

Xet Storage Details

Size:
21.1 kB
·
Xet hash:
007a61a2d9508359fbc4fc5a0c6d3b00d36b694cd94d0f04b0015988684fb9d3

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.