Buckets:
| <meta charset="utf-8" /><meta name="hf:doc:metadata" content="{"title":"T-GATE","local":"t-gate","sections":[{"title":"Benchmarks","local":"benchmarks","sections":[],"depth":2}],"depth":1}"> | |
| <link href="/docs/diffusers/pr_10312/en/_app/immutable/assets/0.e3b0c442.css" rel="modulepreload"> | |
| <link rel="modulepreload" href="/docs/diffusers/pr_10312/en/_app/immutable/entry/start.203b6290.js"> | |
| <link rel="modulepreload" href="/docs/diffusers/pr_10312/en/_app/immutable/chunks/scheduler.8c3d61f6.js"> | |
| <link rel="modulepreload" href="/docs/diffusers/pr_10312/en/_app/immutable/chunks/singletons.8b179f45.js"> | |
| <link rel="modulepreload" href="/docs/diffusers/pr_10312/en/_app/immutable/chunks/index.0997d446.js"> | |
| <link rel="modulepreload" href="/docs/diffusers/pr_10312/en/_app/immutable/chunks/paths.3fd58d56.js"> | |
| <link rel="modulepreload" href="/docs/diffusers/pr_10312/en/_app/immutable/entry/app.423ea23f.js"> | |
| <link rel="modulepreload" href="/docs/diffusers/pr_10312/en/_app/immutable/chunks/index.da70eac4.js"> | |
| <link rel="modulepreload" href="/docs/diffusers/pr_10312/en/_app/immutable/nodes/0.e544eae6.js"> | |
| <link rel="modulepreload" href="/docs/diffusers/pr_10312/en/_app/immutable/chunks/each.e59479a4.js"> | |
| <link rel="modulepreload" href="/docs/diffusers/pr_10312/en/_app/immutable/nodes/197.a256fb81.js"> | |
| <link rel="modulepreload" href="/docs/diffusers/pr_10312/en/_app/immutable/chunks/CodeBlock.00a903b3.js"> | |
| <link rel="modulepreload" href="/docs/diffusers/pr_10312/en/_app/immutable/chunks/EditOnGithub.1e64e623.js"> | |
| <link rel="modulepreload" href="/docs/diffusers/pr_10312/en/_app/immutable/chunks/HfOption.c1483eb1.js"> | |
| <link rel="modulepreload" href="/docs/diffusers/pr_10312/en/_app/immutable/chunks/stores.d6eecc38.js"><!-- HEAD_svelte-u9bgzb_START --><meta name="hf:doc:metadata" content="{"title":"T-GATE","local":"t-gate","sections":[{"title":"Benchmarks","local":"benchmarks","sections":[],"depth":2}],"depth":1}"><!-- HEAD_svelte-u9bgzb_END --> <p></p> <h1 class="relative group"><a id="t-gate" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#t-gate"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>T-GATE</span></h1> <p data-svelte-h="svelte-1qox7gi"><a href="https://github.com/HaozheLiu-ST/T-GATE/tree/main" rel="nofollow">T-GATE</a> accelerates inference for <a href="../api/pipelines/stable_diffusion/overview">Stable Diffusion</a>, <a href="../api/pipelines/pixart">PixArt</a>, and <a href="../api/pipelines/latent_consistency_models.md">Latency Consistency Model</a> pipelines by skipping the cross-attention calculation once it converges. This method doesn’t require any additional training and it can speed up inference from 10-50%. T-GATE is also compatible with other optimization methods like <a href="./deepcache">DeepCache</a>.</p> <p data-svelte-h="svelte-rx8j02">Before you begin, make sure you install T-GATE.</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->pip install tgate | |
| pip install -U torch diffusers transformers accelerate DeepCache<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-113mmog">To use T-GATE with a pipeline, you need to use its corresponding loader.</p> <table data-svelte-h="svelte-17jk9ut"><thead><tr><th>Pipeline</th> <th>T-GATE Loader</th></tr></thead> <tbody><tr><td>PixArt</td> <td>TgatePixArtLoader</td></tr> <tr><td>Stable Diffusion XL</td> <td>TgateSDXLLoader</td></tr> <tr><td>Stable Diffusion XL + DeepCache</td> <td>TgateSDXLDeepCacheLoader</td></tr> <tr><td>Stable Diffusion</td> <td>TgateSDLoader</td></tr> <tr><td>Stable Diffusion + DeepCache</td> <td>TgateSDDeepCacheLoader</td></tr></tbody></table> <p data-svelte-h="svelte-1uere4x">Next, create a <code>TgateLoader</code> with a pipeline, the gate step (the time step to stop calculating the cross attention), and the number of inference steps. Then call the <code>tgate</code> method on the pipeline with a prompt, gate step, and the number of inference steps.</p> <p data-svelte-h="svelte-1tyd5nl">Let’s see how to enable this for several different pipelines.</p> <div class="flex space-x-2 items-center my-1.5 mr-8 h-7 !pl-0 -mx-3 md:mx-0"><div class="flex items-center border rounded-lg px-1.5 py-1 leading-none select-none text-smd border-gray-800 bg-black dark:bg-gray-700 text-white">PixArt </div><div class="flex items-center border rounded-lg px-1.5 py-1 leading-none select-none text-smd text-gray-500 cursor-pointer opacity-90 hover:text-gray-700 dark:hover:text-gray-200 hover:shadow-sm">Stable Diffusion XL </div><div class="flex items-center border rounded-lg px-1.5 py-1 leading-none select-none text-smd text-gray-500 cursor-pointer opacity-90 hover:text-gray-700 dark:hover:text-gray-200 hover:shadow-sm">StableDiffusionXL with DeepCache </div><div class="flex items-center border rounded-lg px-1.5 py-1 leading-none select-none text-smd text-gray-500 cursor-pointer opacity-90 hover:text-gray-700 dark:hover:text-gray-200 hover:shadow-sm">Latent Consistency Model </div></div> <div class="language-select"><p data-svelte-h="svelte-10eb54u">Accelerate <code>PixArtAlphaPipeline</code> with T-GATE:</p> <div class="code-block relative"><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">import</span> torch | |
| <span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> PixArtAlphaPipeline | |
| <span class="hljs-keyword">from</span> tgate <span class="hljs-keyword">import</span> TgatePixArtLoader | |
| pipe = PixArtAlphaPipeline.from_pretrained(<span class="hljs-string">"PixArt-alpha/PixArt-XL-2-1024-MS"</span>, torch_dtype=torch.float16) | |
| gate_step = <span class="hljs-number">8</span> | |
| inference_step = <span class="hljs-number">25</span> | |
| pipe = TgatePixArtLoader( | |
| pipe, | |
| gate_step=gate_step, | |
| num_inference_steps=inference_step, | |
| ).to(<span class="hljs-string">"cuda"</span>) | |
| image = pipe.tgate( | |
| <span class="hljs-string">"An alpaca made of colorful building blocks, cyberpunk."</span>, | |
| gate_step=gate_step, | |
| num_inference_steps=inference_step, | |
| ).images[<span class="hljs-number">0</span>]<!-- HTML_TAG_END --></pre></div> </div> <p data-svelte-h="svelte-1lz4cqf">T-GATE also supports <a href="/docs/diffusers/pr_10312/en/api/pipelines/stable_diffusion/text2img#diffusers.StableDiffusionPipeline">StableDiffusionPipeline</a> and <a href="https://hf.co/PixArt-alpha/PixArt-LCM-XL-2-1024-MS" rel="nofollow">PixArt-alpha/PixArt-LCM-XL-2-1024-MS</a>.</p> <h2 class="relative group"><a id="benchmarks" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#benchmarks"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>Benchmarks</span></h2> <table data-svelte-h="svelte-7mhhwa"><thead><tr><th>Model</th> <th>MACs</th> <th>Param</th> <th>Latency</th> <th>Zero-shot 10K-FID on MS-COCO</th></tr></thead> <tbody><tr><td>SD-1.5</td> <td>16.938T</td> <td>859.520M</td> <td>7.032s</td> <td>23.927</td></tr> <tr><td>SD-1.5 w/ T-GATE</td> <td>9.875T</td> <td>815.557M</td> <td>4.313s</td> <td>20.789</td></tr> <tr><td>SD-2.1</td> <td>38.041T</td> <td>865.785M</td> <td>16.121s</td> <td>22.609</td></tr> <tr><td>SD-2.1 w/ T-GATE</td> <td>22.208T</td> <td>815.433 M</td> <td>9.878s</td> <td>19.940</td></tr> <tr><td>SD-XL</td> <td>149.438T</td> <td>2.570B</td> <td>53.187s</td> <td>24.628</td></tr> <tr><td>SD-XL w/ T-GATE</td> <td>84.438T</td> <td>2.024B</td> <td>27.932s</td> <td>22.738</td></tr> <tr><td>Pixart-Alpha</td> <td>107.031T</td> <td>611.350M</td> <td>61.502s</td> <td>38.669</td></tr> <tr><td>Pixart-Alpha w/ T-GATE</td> <td>65.318T</td> <td>462.585M</td> <td>37.867s</td> <td>35.825</td></tr> <tr><td>DeepCache (SD-XL)</td> <td>57.888T</td> <td>-</td> <td>19.931s</td> <td>23.755</td></tr> <tr><td>DeepCache w/ T-GATE</td> <td>43.868T</td> <td>-</td> <td>14.666s</td> <td>23.999</td></tr> <tr><td>LCM (SD-XL)</td> <td>11.955T</td> <td>2.570B</td> <td>3.805s</td> <td>25.044</td></tr> <tr><td>LCM w/ T-GATE</td> <td>11.171T</td> <td>2.024B</td> <td>3.533s</td> <td>25.028</td></tr> <tr><td>LCM (Pixart-Alpha)</td> <td>8.563T</td> <td>611.350M</td> <td>4.733s</td> <td>36.086</td></tr> <tr><td>LCM w/ T-GATE</td> <td>7.623T</td> <td>462.585M</td> <td>4.543s</td> <td>37.048</td></tr></tbody></table> <p data-svelte-h="svelte-17hd8in">The latency is tested on an NVIDIA 1080TI, MACs and Params are calculated with <a href="https://github.com/MrYxJ/calculate-flops.pytorch" rel="nofollow">calflops</a>, and the FID is calculated with <a href="https://github.com/mseitzer/pytorch-fid" rel="nofollow">PytorchFID</a>.</p> <a class="!text-gray-400 !no-underline text-sm flex items-center not-prose mt-4" href="https://github.com/huggingface/diffusers/blob/main/docs/source/en/optimization/tgate.md" target="_blank"><span data-svelte-h="svelte-1kd6by1"><</span> <span data-svelte-h="svelte-x0xyl0">></span> <span data-svelte-h="svelte-1dajgef"><span class="underline ml-1.5">Update</span> on GitHub</span></a> <p></p> | |
| <script> | |
| { | |
| __sveltekit_zvhs44 = { | |
| assets: "/docs/diffusers/pr_10312/en", | |
| base: "/docs/diffusers/pr_10312/en", | |
| env: {} | |
| }; | |
| const element = document.currentScript.parentElement; | |
| const data = [null,null]; | |
| Promise.all([ | |
| import("/docs/diffusers/pr_10312/en/_app/immutable/entry/start.203b6290.js"), | |
| import("/docs/diffusers/pr_10312/en/_app/immutable/entry/app.423ea23f.js") | |
| ]).then(([kit, app]) => { | |
| kit.start(app, element, { | |
| node_ids: [0, 197], | |
| data, | |
| form: null, | |
| error: null | |
| }); | |
| }); | |
| } | |
| </script> | |
Xet Storage Details
- Size:
- 13.8 kB
- Xet hash:
- 85f126c4b5d76248d9246d6b0c832914ed4a93929c25ea7d017e49ea83bfea58
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.