Buckets:

hf-doc-build
/

doc-dev

Files

xet

hf-doc-build/doc-dev / diffusers /pr_11797 /en /modular_diffusers /auto_pipeline_blocks.html

rtrm

15 days ago

download

raw

42.2 kB

	<meta charset="utf-8" /><meta name="hf:doc:metadata" content="{"title":"AutoPipelineBlocks","local":"autopipelineblocks","sections":[],"depth":1}">
	<link href="/docs/diffusers/pr_11797/en/_app/immutable/assets/0.e3b0c442.css" rel="modulepreload">
	<link rel="modulepreload" href="/docs/diffusers/pr_11797/en/_app/immutable/entry/start.2e1ee9a7.js">
	<link rel="modulepreload" href="/docs/diffusers/pr_11797/en/_app/immutable/chunks/scheduler.8c3d61f6.js">
	<link rel="modulepreload" href="/docs/diffusers/pr_11797/en/_app/immutable/chunks/singletons.702cc21f.js">
	<link rel="modulepreload" href="/docs/diffusers/pr_11797/en/_app/immutable/chunks/index.0997d446.js">
	<link rel="modulepreload" href="/docs/diffusers/pr_11797/en/_app/immutable/chunks/paths.ec7f49a7.js">
	<link rel="modulepreload" href="/docs/diffusers/pr_11797/en/_app/immutable/entry/app.87c95e21.js">
	<link rel="modulepreload" href="/docs/diffusers/pr_11797/en/_app/immutable/chunks/index.da70eac4.js">
	<link rel="modulepreload" href="/docs/diffusers/pr_11797/en/_app/immutable/nodes/0.7e207bc6.js">
	<link rel="modulepreload" href="/docs/diffusers/pr_11797/en/_app/immutable/chunks/each.e59479a4.js">
	<link rel="modulepreload" href="/docs/diffusers/pr_11797/en/_app/immutable/nodes/227.4aa4b852.js">
	<link rel="modulepreload" href="/docs/diffusers/pr_11797/en/_app/immutable/chunks/Tip.1d9b8c37.js">
	<link rel="modulepreload" href="/docs/diffusers/pr_11797/en/_app/immutable/chunks/CodeBlock.a9c4becf.js">
	<link rel="modulepreload" href="/docs/diffusers/pr_11797/en/_app/immutable/chunks/getInferenceSnippets.725ed3d4.js"><!-- HEAD_svelte-u9bgzb_START --><meta name="hf:doc:metadata" content="{"title":"AutoPipelineBlocks","local":"autopipelineblocks","sections":[],"depth":1}"><!-- HEAD_svelte-u9bgzb_END --> <p></p> <h1 class="relative group"><a id="autopipelineblocks" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#autopipelineblocks"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a> <span>AutoPipelineBlocks</span></h1> <div class="course-tip course-tip-orange bg-gradient-to-br dark:bg-gradient-to-r before:border-orange-500 dark:before:border-orange-800 from-orange-50 dark:from-gray-900 to-white dark:to-gray-950 border border-orange-50 text-orange-700 dark:text-gray-400"><p data-svelte-h="svelte-11qh5l3">🧪 <strong>Experimental Feature</strong>: Modular Diffusers is an experimental feature we are actively developing. The API may be subject to breaking changes.</p></div> <p data-svelte-h="svelte-mnhl3k"><code>AutoPipelineBlocks</code> is a subclass of <code>ModularPipelineBlocks</code>. It is a multi-block that automatically selects which sub-blocks to run based on the inputs provided at runtime, creating conditional workflows that adapt to different scenarios. The main purpose is convenience and portability - for developers, you can package everything into one workflow, making it easier to share and use.</p> <p data-svelte-h="svelte-2zzq1v">In this tutorial, we will show you how to create an <code>AutoPipelineBlocks</code> and learn more about how the conditional selection works.</p> <div class="course-tip bg-gradient-to-br dark:bg-gradient-to-r before:border-green-500 dark:before:border-green-800 from-green-50 dark:from-gray-900 to-white dark:to-gray-950 border border-green-50 text-green-700 dark:text-gray-400"><p data-svelte-h="svelte-1sajd2r">Other types of multi-blocks include <a href="sequential_pipeline_blocks.md">SequentialPipelineBlocks</a> (for linear workflows) and <a href="loop_sequential_pipeline_blocks.md">LoopSequentialPipelineBlocks</a> (for iterative workflows). For information on creating individual blocks, see the <a href="pipeline_block.md">PipelineBlock guide</a>.</p> <p data-svelte-h="svelte-670hni">Additionally, like all <code>ModularPipelineBlocks</code>, <code>AutoPipelineBlocks</code> are definitions/specifications, not runnable pipelines. You need to convert them into a <code>ModularPipeline</code> to actually execute them. For information on creating and running pipelines, see the <a href="modular_pipeline.md">Modular Pipeline guide</a>.</p></div> <p data-svelte-h="svelte-1vl4ct9">For example, you might want to support text-to-image and image-to-image tasks. Instead of creating two separate pipelines, you can create an <code>AutoPipelineBlocks</code> that automatically chooses the workflow based on whether an <code>image</code> input is provided.</p> <p data-svelte-h="svelte-6r1dwm">Let’s see an example. We’ll use the helper function from the <a href="./pipeline_block.md">PipelineBlock guide</a> to create our blocks:</p> <p data-svelte-h="svelte-1doi555"><strong>Helper Function</strong></p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> diffusers.modular_pipelines <span class="hljs-keyword">import</span> PipelineBlock, InputParam, OutputParam
	<span class="hljs-keyword">import</span> torch

	<span class="hljs-keyword">def</span> <span class="hljs-title function_">make_block</span>(<span class="hljs-params">inputs=[], intermediate_inputs=[], intermediate_outputs=[], block_fn=<span class="hljs-literal">None</span>, description=<span class="hljs-literal">None</span></span>):
	<span class="hljs-keyword">class</span> <span class="hljs-title class_">TestBlock</span>(<span class="hljs-title class_ inherited__">PipelineBlock</span>):
	model_name = <span class="hljs-string">"test"</span>

	<span class="hljs-meta"> @property</span>
	<span class="hljs-keyword">def</span> <span class="hljs-title function_">inputs</span>(<span class="hljs-params">self</span>):
	<span class="hljs-keyword">return</span> inputs

	<span class="hljs-meta"> @property</span>
	<span class="hljs-keyword">def</span> <span class="hljs-title function_">intermediate_inputs</span>(<span class="hljs-params">self</span>):
	<span class="hljs-keyword">return</span> intermediate_inputs

	<span class="hljs-meta"> @property</span>
	<span class="hljs-keyword">def</span> <span class="hljs-title function_">intermediate_outputs</span>(<span class="hljs-params">self</span>):
	<span class="hljs-keyword">return</span> intermediate_outputs

	<span class="hljs-meta"> @property</span>
	<span class="hljs-keyword">def</span> <span class="hljs-title function_">description</span>(<span class="hljs-params">self</span>):
	<span class="hljs-keyword">return</span> description <span class="hljs-keyword">if</span> description <span class="hljs-keyword">is</span> <span class="hljs-keyword">not</span> <span class="hljs-literal">None</span> <span class="hljs-keyword">else</span> <span class="hljs-string">""</span>

	<span class="hljs-keyword">def</span> <span class="hljs-title function_">__call__</span>(<span class="hljs-params">self, components, state</span>):
	block_state = self.get_block_state(state)
	<span class="hljs-keyword">if</span> block_fn <span class="hljs-keyword">is</span> <span class="hljs-keyword">not</span> <span class="hljs-literal">None</span>:
	block_state = block_fn(block_state, state)
	self.set_block_state(state, block_state)
	<span class="hljs-keyword">return</span> components, state

	<span class="hljs-keyword">return</span> TestBlock<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-j4ivop">Now let’s create a dummy <code>AutoPipelineBlocks</code> that includes dummy text-to-image, image-to-image, and inpaint pipelines.</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> diffusers.modular_pipelines <span class="hljs-keyword">import</span> AutoPipelineBlocks

	<span class="hljs-comment"># These are dummy blocks and we only focus on "inputs" for our purpose</span>
	inputs = [InputParam(name=<span class="hljs-string">"prompt"</span>)]
	<span class="hljs-comment"># block_fn prints out which workflow is running so we can see the execution order at runtime</span>
	block_fn = <span class="hljs-keyword">lambda</span> x, y: <span class="hljs-built_in">print</span>(<span class="hljs-string">"running the text-to-image workflow"</span>)
	block_t2i_cls = make_block(inputs=inputs, block_fn=block_fn, description=<span class="hljs-string">"I'm a text-to-image workflow!"</span>)

	inputs = [InputParam(name=<span class="hljs-string">"prompt"</span>), InputParam(name=<span class="hljs-string">"image"</span>)]
	block_fn = <span class="hljs-keyword">lambda</span> x, y: <span class="hljs-built_in">print</span>(<span class="hljs-string">"running the image-to-image workflow"</span>)
	block_i2i_cls = make_block(inputs=inputs, block_fn=block_fn, description=<span class="hljs-string">"I'm a image-to-image workflow!"</span>)

	inputs = [InputParam(name=<span class="hljs-string">"prompt"</span>), InputParam(name=<span class="hljs-string">"image"</span>), InputParam(name=<span class="hljs-string">"mask"</span>)]
	block_fn = <span class="hljs-keyword">lambda</span> x, y: <span class="hljs-built_in">print</span>(<span class="hljs-string">"running the inpaint workflow"</span>)
	block_inpaint_cls = make_block(inputs=inputs, block_fn=block_fn, description=<span class="hljs-string">"I'm a inpaint workflow!"</span>)

	<span class="hljs-keyword">class</span> <span class="hljs-title class_">AutoImageBlocks</span>(<span class="hljs-title class_ inherited__">AutoPipelineBlocks</span>):
	<span class="hljs-comment"># List of sub-block classes to choose from</span>
	block_classes = [block_inpaint_cls, block_i2i_cls, block_t2i_cls]
	<span class="hljs-comment"># Names for each block in the same order</span>
	block_names = [<span class="hljs-string">"inpaint"</span>, <span class="hljs-string">"img2img"</span>, <span class="hljs-string">"text2img"</span>]
	<span class="hljs-comment"># Trigger inputs that determine which block to run</span>
	<span class="hljs-comment"># - "mask" triggers inpaint workflow</span>
	<span class="hljs-comment"># - "image" triggers img2img workflow (but only if mask is not provided) </span>
	<span class="hljs-comment"># - if none of above, runs the text2img workflow (default)</span>
	block_trigger_inputs = [<span class="hljs-string">"mask"</span>, <span class="hljs-string">"image"</span>, <span class="hljs-literal">None</span>]
	<span class="hljs-comment"># Description is extremely important for AutoPipelineBlocks</span>
	<span class="hljs-meta"> @property</span>
	<span class="hljs-keyword">def</span> <span class="hljs-title function_">description</span>(<span class="hljs-params">self</span>):
	<span class="hljs-keyword">return</span> (
	<span class="hljs-string">"Pipeline generates images given different types of conditions!\n"</span>
	+ <span class="hljs-string">"This is an auto pipeline block that works for text2img, img2img and inpainting tasks.\n"</span>
	+ <span class="hljs-string">" - inpaint workflow is run when `mask` is provided.\n"</span>
	+ <span class="hljs-string">" - img2img workflow is run when `image` is provided (but only when `mask` is not provided).\n"</span>
	+ <span class="hljs-string">" - text2img workflow is run when neither `image` nor `mask` is provided.\n"</span>
	)

	<span class="hljs-comment"># Create the blocks</span>
	auto_blocks = AutoImageBlocks()
	<span class="hljs-comment"># convert to pipeline</span>
	auto_pipeline = auto_blocks.init_pipeline()<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-ab78gz">Now we have created an <code>AutoPipelineBlocks</code> that contains 3 sub-blocks. Notice the warning message at the top - this automatically appears in every <code>ModularPipelineBlocks</code> that contains <code>AutoPipelineBlocks</code> to remind end users that dynamic block selection happens at runtime.</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->AutoImageBlocks(
	Class: AutoPipelineBlocks

	====================================================================================================
	This pipeline contains blocks that are selected at runtime based on inputs.
	Trigger Inputs: [<span class="hljs-string">'mask'</span>, <span class="hljs-string">'image'</span>]
	====================================================================================================


	Description: Pipeline generates images given different types of conditions!
	This <span class="hljs-keyword">is</span> an auto pipeline block that works <span class="hljs-keyword">for</span> text2img, img2img <span class="hljs-keyword">and</span> inpainting tasks.
	- inpaint workflow <span class="hljs-keyword">is</span> run when `mask` <span class="hljs-keyword">is</span> provided.
	- img2img workflow <span class="hljs-keyword">is</span> run when `image` <span class="hljs-keyword">is</span> provided (but only when `mask` <span class="hljs-keyword">is</span> <span class="hljs-keyword">not</span> provided).
	- text2img workflow <span class="hljs-keyword">is</span> run when neither `image` nor `mask` <span class="hljs-keyword">is</span> provided.



	Sub-Blocks:
	• inpaint [trigger: mask] (TestBlock)
	Description: I<span class="hljs-string">'m a inpaint workflow!

	• img2img [trigger: image] (TestBlock)
	Description: I'</span>m a image-to-image workflow!

	• text2img [default] (TestBlock)
	Description: I<span class="hljs-string">'m a text-to-image workflow!

	)</span><!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-gdbz8g">Check out the documentation with <code>print(auto_pipeline.doc)</code>:</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-meta">>>> </span><span class="hljs-built_in">print</span>(auto_pipeline.doc)
	<span class="hljs-keyword">class</span> <span class="hljs-title class_">AutoImageBlocks</span>

	Pipeline generates images given different types of conditions!
	This <span class="hljs-keyword">is</span> an auto pipeline block that works <span class="hljs-keyword">for</span> text2img, img2img <span class="hljs-keyword">and</span> inpainting tasks.
	- inpaint workflow <span class="hljs-keyword">is</span> run when `mask` <span class="hljs-keyword">is</span> provided.
	- img2img workflow <span class="hljs-keyword">is</span> run when `image` <span class="hljs-keyword">is</span> provided (but only when `mask` <span class="hljs-keyword">is</span> <span class="hljs-keyword">not</span> provided).
	- text2img workflow <span class="hljs-keyword">is</span> run when neither `image` nor `mask` <span class="hljs-keyword">is</span> provided.

	Inputs:

	prompt (`<span class="hljs-literal">None</span>`, optional):

	image (`<span class="hljs-literal">None</span>`, optional):

	mask (`<span class="hljs-literal">None</span>`, optional):<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1hyz85d">There is a fundamental trade-off of AutoPipelineBlocks: it trades clarity for convenience. While it is really easy for packaging multiple workflows, it can become confusing without proper documentation. e.g. if we just throw a pipeline at you and tell you that it contains 3 sub-blocks and takes 3 inputs <code>prompt</code>, <code>image</code> and <code>mask</code>, and ask you to run an image-to-image workflow: if you don’t have any prior knowledge on how these pipelines work, you would be pretty clueless, right?</p> <p data-svelte-h="svelte-w9jep0">This pipeline we just made though, has a docstring that shows all available inputs and workflows and explains how to use each with different inputs. So it’s really helpful for users. For example, it’s clear that you need to pass <code>image</code> to run img2img. This is why the description field is absolutely critical for AutoPipelineBlocks. We highly recommend you to explain the conditional logic very well for each <code>AutoPipelineBlocks</code> you would make. We also recommend to always test individual pipelines first before packaging them into AutoPipelineBlocks.</p> <p data-svelte-h="svelte-1ohmy0h">Let’s run this auto pipeline with different inputs to see if the conditional logic works as described. Remember that we have added <code>print</code> in each <code>PipelineBlock</code>’s <code>__call__</code> method to print out its workflow name, so it should be easy to tell which one is running:</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-meta">>>> </span>_ = auto_pipeline(image=<span class="hljs-string">"image"</span>, mask=<span class="hljs-string">"mask"</span>)
	running the inpaint workflow
	<span class="hljs-meta">>>> </span>_ = auto_pipeline(image=<span class="hljs-string">"image"</span>)
	running the image-to-image workflow
	<span class="hljs-meta">>>> </span>_ = auto_pipeline(prompt=<span class="hljs-string">"prompt"</span>)
	running the text-to-image workflow
	<span class="hljs-meta">>>> </span>_ = auto_pipeline(image=<span class="hljs-string">"prompt"</span>, mask=<span class="hljs-string">"mask"</span>)
	running the inpaint workflow<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-jdpirs">However, even with documentation, it can become very confusing when AutoPipelineBlocks are combined with other blocks. The complexity grows quickly when you have nested AutoPipelineBlocks or use them as sub-blocks in larger pipelines.</p> <p data-svelte-h="svelte-1p46svu">Let’s make another <code>AutoPipelineBlocks</code> - this one only contains one block, and it does not include <code>None</code> in its <code>block_trigger_inputs</code> (which corresponds to the default block to run when none of the trigger inputs are provided). This means this block will be skipped if the trigger input (<code>ip_adapter_image</code>) is not provided at runtime.</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-keyword">from</span> diffusers.modular_pipelines <span class="hljs-keyword">import</span> SequentialPipelineBlocks, InsertableDict
	inputs = [InputParam(name=<span class="hljs-string">"ip_adapter_image"</span>)]
	block_fn = <span class="hljs-keyword">lambda</span> x, y: <span class="hljs-built_in">print</span>(<span class="hljs-string">"running the ip-adapter workflow"</span>)
	block_ipa_cls = make_block(inputs=inputs, block_fn=block_fn, description=<span class="hljs-string">"I'm a IP-adapter workflow!"</span>)

	<span class="hljs-keyword">class</span> <span class="hljs-title class_">AutoIPAdapter</span>(<span class="hljs-title class_ inherited__">AutoPipelineBlocks</span>):
	block_classes = [block_ipa_cls]
	block_names = [<span class="hljs-string">"ip-adapter"</span>]
	block_trigger_inputs = [<span class="hljs-string">"ip_adapter_image"</span>]
	<span class="hljs-meta"> @property</span>
	<span class="hljs-keyword">def</span> <span class="hljs-title function_">description</span>(<span class="hljs-params">self</span>):
	<span class="hljs-keyword">return</span> <span class="hljs-string">"Run IP Adapter step if `ip_adapter_image` is provided."</span><!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-s6ib93">Now let’s combine these 2 auto blocks together into a <code>SequentialPipelineBlocks</code>:</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START -->auto_ipa_blocks = AutoIPAdapter()
	blocks_dict = InsertableDict()
	blocks_dict[<span class="hljs-string">"ip-adapter"</span>] = auto_ipa_blocks
	blocks_dict[<span class="hljs-string">"image-generation"</span>] = auto_blocks
	all_blocks = SequentialPipelineBlocks.from_blocks_dict(blocks_dict)
	pipeline = all_blocks.init_pipeline()<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-ec72dx">Let’s take a look: now things get more confusing. In this particular example, you could still try to explain the conditional logic in the <code>description</code> field here - there are only 4 possible execution paths so it’s doable. However, since this is a <code>SequentialPipelineBlocks</code> that could contain many more blocks, the complexity can quickly get out of hand as the number of blocks increases.</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-meta">>>> </span>all_blocks
	SequentialPipelineBlocks(
	Class: ModularPipelineBlocks

	====================================================================================================
	This pipeline contains blocks that are selected at runtime based on inputs.
	Trigger Inputs: [<span class="hljs-string">'image'</span>, <span class="hljs-string">'mask'</span>, <span class="hljs-string">'ip_adapter_image'</span>]
	Use `get_execution_blocks()` <span class="hljs-keyword">with</span> <span class="hljs-built_in">input</span> names to see selected blocks (e.g. `get_execution_blocks(<span class="hljs-string">'image'</span>)`).
	====================================================================================================


	Description:


	Sub-Blocks:
	[<span class="hljs-number">0</span>] ip-adapter (AutoIPAdapter)
	Description: Run IP Adapter step <span class="hljs-keyword">if</span> `ip_adapter_image` <span class="hljs-keyword">is</span> provided.


	[<span class="hljs-number">1</span>] image-generation (AutoImageBlocks)
	Description: Pipeline generates images given different types of conditions!
	This <span class="hljs-keyword">is</span> an auto pipeline block that works <span class="hljs-keyword">for</span> text2img, img2img <span class="hljs-keyword">and</span> inpainting tasks.
	- inpaint workflow <span class="hljs-keyword">is</span> run when `mask` <span class="hljs-keyword">is</span> provided.
	- img2img workflow <span class="hljs-keyword">is</span> run when `image` <span class="hljs-keyword">is</span> provided (but only when `mask` <span class="hljs-keyword">is</span> <span class="hljs-keyword">not</span> provided).
	- text2img workflow <span class="hljs-keyword">is</span> run when neither `image` nor `mask` <span class="hljs-keyword">is</span> provided.


	)
	<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-93ywld">This is when the <code>get_execution_blocks()</code> method comes in handy - it basically extracts a <code>SequentialPipelineBlocks</code> that only contains the blocks that are actually run based on your inputs.</p> <p data-svelte-h="svelte-duj019">Let’s try some examples:</p> <p data-svelte-h="svelte-5e2q2p"><code>mask</code>: we expect it to skip the first ip-adapter since <code>ip_adapter_image</code> is not provided, and then run the inpaint for the second block.</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-meta">>>> </span>all_blocks.get_execution_blocks(<span class="hljs-string">'mask'</span>)
	SequentialPipelineBlocks(
	Class: ModularPipelineBlocks

	Description:


	Sub-Blocks:
	[<span class="hljs-number">0</span>] image-generation (TestBlock)
	Description: I<span class="hljs-string">'m a inpaint workflow!

	)</span><!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-1ekzn1m">Let’s also actually run the pipeline to confirm:</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-meta">>>> </span>_ = pipeline(mask=<span class="hljs-string">"mask"</span>)
	skipping auto block: AutoIPAdapter
	running the inpaint workflow<!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-183uoi5">Try a few more:</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg class="" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg> <div class="absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0"><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent; "></div> Copied</div></button></div> <pre class=""><!-- HTML_TAG_START --><span class="hljs-built_in">print</span>(<span class="hljs-string">f"inputs: ip_adapter_image:"</span>)
	blocks_select = all_blocks.get_execution_blocks(<span class="hljs-string">'ip_adapter_image'</span>)
	<span class="hljs-built_in">print</span>(<span class="hljs-string">f"expected_execution_blocks: <span class="hljs-subst">{blocks_select}</span>"</span>)
	<span class="hljs-built_in">print</span>(<span class="hljs-string">f"actual execution blocks:"</span>)
	_ = pipeline(ip_adapter_image=<span class="hljs-string">"ip_adapter_image"</span>, prompt=<span class="hljs-string">"prompt"</span>)
	<span class="hljs-comment"># expect to see ip-adapter + text2img</span>

	<span class="hljs-built_in">print</span>(<span class="hljs-string">f"inputs: image:"</span>)
	blocks_select = all_blocks.get_execution_blocks(<span class="hljs-string">'image'</span>)
	<span class="hljs-built_in">print</span>(<span class="hljs-string">f"expected_execution_blocks: <span class="hljs-subst">{blocks_select}</span>"</span>)
	<span class="hljs-built_in">print</span>(<span class="hljs-string">f"actual execution blocks:"</span>)
	_ = pipeline(image=<span class="hljs-string">"image"</span>, prompt=<span class="hljs-string">"prompt"</span>)
	<span class="hljs-comment"># expect to see img2img</span>

	<span class="hljs-built_in">print</span>(<span class="hljs-string">f"inputs: prompt:"</span>)
	blocks_select = all_blocks.get_execution_blocks(<span class="hljs-string">'prompt'</span>)
	<span class="hljs-built_in">print</span>(<span class="hljs-string">f"expected_execution_blocks: <span class="hljs-subst">{blocks_select}</span>"</span>)
	<span class="hljs-built_in">print</span>(<span class="hljs-string">f"actual execution blocks:"</span>)
	_ = pipeline(prompt=<span class="hljs-string">"prompt"</span>)
	<span class="hljs-comment"># expect to see text2img (prompt is not a trigger input so fallback to default)</span>

	<span class="hljs-built_in">print</span>(<span class="hljs-string">f"inputs: mask + ip_adapter_image:"</span>)
	blocks_select = all_blocks.get_execution_blocks(<span class="hljs-string">'mask'</span>,<span class="hljs-string">'ip_adapter_image'</span>)
	<span class="hljs-built_in">print</span>(<span class="hljs-string">f"expected_execution_blocks: <span class="hljs-subst">{blocks_select}</span>"</span>)
	<span class="hljs-built_in">print</span>(<span class="hljs-string">f"actual execution blocks:"</span>)
	_ = pipeline(mask=<span class="hljs-string">"mask"</span>, ip_adapter_image=<span class="hljs-string">"ip_adapter_image"</span>)
	<span class="hljs-comment"># expect to see ip-adapter + inpaint</span><!-- HTML_TAG_END --></pre></div> <p data-svelte-h="svelte-9lxtc7">In summary, <code>AutoPipelineBlocks</code> is a good tool for packaging multiple workflows into a single, convenient interface and it can greatly simplify the user experience. However, always provide clear descriptions explaining the conditional logic, test individual pipelines first before combining them, and use <code>get_execution_blocks()</code> to understand runtime behavior in complex compositions.</p> <a class="!text-gray-400 !no-underline text-sm flex items-center not-prose mt-4" href="https://github.com/huggingface/diffusers/blob/main/docs/source/en/modular_diffusers/auto_pipeline_blocks.md" target="_blank"><span data-svelte-h="svelte-1kd6by1"><</span> <span data-svelte-h="svelte-x0xyl0">></span> <span data-svelte-h="svelte-1dajgef"><span class="underline ml-1.5">Update</span> on GitHub</span></a> <p></p>

	<script>
	{
	__sveltekit_9fqn6m = {
	assets: "/docs/diffusers/pr_11797/en",
	base: "/docs/diffusers/pr_11797/en",
	env: {}
	};

	const element = document.currentScript.parentElement;

	const data = [null,null];

	Promise.all([
	import("/docs/diffusers/pr_11797/en/_app/immutable/entry/start.2e1ee9a7.js"),
	import("/docs/diffusers/pr_11797/en/_app/immutable/entry/app.87c95e21.js")
	]).then(([kit, app]) => {
	kit.start(app, element, {
	node_ids: [0, 227],
	data,
	form: null,
	error: null
	});
	});
	}
	</script>

Xet Storage Details

Size:: 42.2 kB
Xet hash:: 88d85c5311e509cb340e9f707c5269664ff151238331f421aaab6c46ed37eead

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.