diffusers/pr_12411/en/api/modular_diffusers/pipeline.md

Pipeline

ModularPipeline[[diffusers.ModularPipeline]]

diffusers.ModularPipeline[[diffusers.ModularPipeline]]

Source

Base class for all Modular pipelines.

> This is an experimental feature and is likely to change in the future.

from_pretraineddiffusers.ModularPipeline.from_pretrainedhttps://github.com/huggingface/diffusers/blob/vr_12411/src/diffusers/modular_pipelines/modular_pipeline.py#L1651[{"name": "pretrained_model_name_or_path", "val": ": typing.Union[str, os.PathLike, NoneType]"}, {"name": "trust_remote_code", "val": ": typing.Optional[bool] = None"}, {"name": "components_manager", "val": ": typing.Optional[diffusers.modular_pipelines.components_manager.ComponentsManager] = None"}, {"name": "collection", "val": ": typing.Optional[str] = None"}, {"name": "**kwargs", "val": ""}]- pretrained_model_name_or_path (str or os.PathLike, optional) -- Path to a pretrained pipeline configuration. It will first try to load config from modular_model_index.json, then fallback to model_index.json for compatibility with standard non-modular repositories. If the pretrained_model_name_or_path does not contain any pipeline config, it will be set to None during initialization.

• trust_remote_code (bool, optional) -- Whether to trust remote code when loading the pipeline, need to be set to True if you want to create pipeline blocks based on the custom code in pretrained_model_name_or_path
• components_manager (ComponentsManager, optional) -- ComponentsManager instance for managing multiple component cross different pipelines and apply offloading strategies.
• collection (str, optional) --` Collection name for organizing components in the ComponentsManager.0

Load a ModularPipeline from a huggingface hub repo.

Parameters:

blocks : ModularPipelineBlocks, the blocks to be used in the pipeline

get_component_spec[[diffusers.ModularPipeline.get_component_spec]]

Source

Returns:

• a copy of the ComponentSpec object for the given component name

load_components[[diffusers.ModularPipeline.load_components]]

Source

Load selected components from specs.

Parameters:

names : List of component names to load. If None, will load all components with default_creation_method == "from_pretrained". If provided as a list or string, will load only the specified components.

• **kwargs : additional kwargs to be passed to from_pretrained().Can be: - a single value to be applied to all components to be loaded, e.g. torch_dtype=torch.bfloat16 - a dict, e.g. torch_dtype={"unet": torch.bfloat16, "default": torch.float32} - if potentially override ComponentSpec if passed a different loading field in kwargs, e.g. pretrained_model_name_or_path, variant, revision, etc. - if potentially override ComponentSpec if passed a different loading field in kwargs, e.g. pretrained_model_name_or_path, variant, revision, etc.

register_components[[diffusers.ModularPipeline.register_components]]

Source

Register components with their corresponding specifications.

This method is responsible for:

1. Sets component objects as attributes on the loader (e.g., self.unet = unet)
2. Updates the config dict, which will be saved as modular_model_index.json during save_pretrained (only for from_pretrained components)
3. Adds components to the component manager if one is attached (only for from_pretrained components)

This method is called when:

• Components are first initialized in init:
  • from_pretrained components not loaded during init so they are registered as None;
  • non from_pretrained components are created during init and registered as the object itself
• Components are updated with the update_components() method: e.g. loader.update_components(unet=unet) or loader.update_components(guider=guider_spec)
• (from_pretrained) Components are loaded with the load_components() method: e.g. loader.load_components(names=["unet"]) or loader.load_components() to load all default components

Notes:

• When registering None for a component, it sets attribute to None but still syncs specs with the config dict, which will be saved as modular_model_index.json during save_pretrained
• component_specs are updated to match the new component outside of this method, e.g. in update_components() method

Parameters:

• **kwargs : Keyword arguments where keys are component names and values are component objects. E.g., register_components(unet=unet_model, text_encoder=encoder_model)

save_pretrained[[diffusers.ModularPipeline.save_pretrained]]

Source

Save the pipeline to a directory. It does not save components, you need to save them separately.

Parameters:

save_directory (str or os.PathLike) : Path to the directory where the pipeline will be saved.

push_to_hub (bool, optional) : Whether to push the pipeline to the huggingface hub.

• **kwargs : Additional arguments passed to save_config() method

to[[diffusers.ModularPipeline.to]]

Source

Performs Pipeline dtype and/or device conversion. A torch.dtype and torch.device are inferred from the arguments of self.to(*args, **kwargs).

> If the pipeline already has the correct torch.dtype and torch.device, then it is returned as is. Otherwise, > the returned pipeline is a copy of self with the desired torch.dtype and torch.device.

Here are the ways to call to:

• to(dtype, silence_dtype_warnings=False) → DiffusionPipeline to return a pipeline with the specified dtype
• to(device, silence_dtype_warnings=False) → DiffusionPipeline to return a pipeline with the specified device
• to(device=None, dtype=None, silence_dtype_warnings=False) → DiffusionPipeline to return a pipeline with the specified device and dtype

Parameters:

dtype (torch.dtype, optional) : Returns a pipeline with the specified dtype

device (torch.Device, optional) : Returns a pipeline with the specified device

silence_dtype_warnings (str, optional, defaults to False) : Whether to omit warnings if the target dtype is not compatible with the target device.

Returns:

[DiffusionPipeline](/docs/diffusers/pr_12411/en/api/pipelines/overview#diffusers.DiffusionPipeline)

The pipeline converted to specified dtype and/or dtype.

update_components[[diffusers.ModularPipeline.update_components]]

Source

Update components and configuration values and specs after the pipeline has been instantiated.

This method allows you to:

1. Replace existing components with new ones (e.g., updating self.unet or self.text_encoder)
2. Update configuration values (e.g., changing self.requires_safety_checker flag)

In addition to updating the components and configuration values as pipeline attributes, the method also updates:

• the corresponding specs in _component_specs and _config_specs
• the config dict, which will be saved as modular_model_index.json during save_pretrained

Examples:

# Update multiple components at once
pipeline.update_components(unet=new_unet_model, text_encoder=new_text_encoder)

# Update configuration values
pipeline.update_components(requires_safety_checker=False)

# Update both components and configs together
pipeline.update_components(unet=new_unet_model, requires_safety_checker=False)

# Update with ComponentSpec objects (from_config only)
pipeline.update_components(
    guider=ComponentSpec(
        name="guider",
        type_hint=ClassifierFreeGuidance,
        config={"guidance_scale": 5.0},
        default_creation_method="from_config",
    )
)

Notes:

• Components with trained weights must be created using ComponentSpec.load(). If the component has not been shared in huggingface hub and you don't have loading specs, you can upload it using push_to_hub()
• ConfigMixin objects without weights (e.g., schedulers, guiders) can be passed directly
• ComponentSpec objects with default_creation_method="from_pretrained" are not supported in update_components()

Parameters:

• **kwargs : Component objects, ComponentSpec objects, or configuration values to update: - Component objects: Only supports components we can extract specs using ComponentSpec.from_component() method i.e. components created with ComponentSpec.load() or ConfigMixin subclasses that aren't nn.Modules (e.g., unet=new_unet, text_encoder=new_encoder) - ComponentSpec objects: Only supports default_creation_method == "from_config", will call create() method to create a new component (e.g., guider=ComponentSpec(name="guider", type_hint=ClassifierFreeGuidance, config={...}, default_creation_method="from_config")) - Configuration values: Simple values to update configuration settings (e.g., requires_safety_checker=False)