diffusers/pr_12509/en/api/activations.md

Activation functions

Customized activation functions for supporting various models in 🤗 Diffusers.

GELU[[diffusers.models.activations.GELU]]

class diffusers.models.activations.GELUdiffusers.models.activations.GELUhttps://github.com/huggingface/diffusers/blob/vr_12509/src/diffusers/models/activations.py#L65[{"name": "dim_in", "val": ": int"}, {"name": "dim_out", "val": ": int"}, {"name": "approximate", "val": ": str = 'none'"}, {"name": "bias", "val": ": bool = True"}]- dim_in (int) -- The number of channels in the input.

• dim_out (int) -- The number of channels in the output.
• approximate (str, optional, defaults to "none") -- If "tanh", use tanh approximation.
• bias (bool, defaults to True) -- Whether to use a bias in the linear layer.0

GELU activation function with tanh approximation support with approximate="tanh".

GEGLU[[diffusers.models.activations.GEGLU]]

class diffusers.models.activations.GEGLUdiffusers.models.activations.GEGLUhttps://github.com/huggingface/diffusers/blob/vr_12509/src/diffusers/models/activations.py#L93[{"name": "dim_in", "val": ": int"}, {"name": "dim_out", "val": ": int"}, {"name": "bias", "val": ": bool = True"}]- dim_in (int) -- The number of channels in the input.

• dim_out (int) -- The number of channels in the output.
• bias (bool, defaults to True) -- Whether to use a bias in the linear layer.0

A variant of the gated linear unit activation function.

ApproximateGELU[[diffusers.models.activations.ApproximateGELU]]

class diffusers.models.activations.ApproximateGELUdiffusers.models.activations.ApproximateGELUhttps://github.com/huggingface/diffusers/blob/vr_12509/src/diffusers/models/activations.py#L149[{"name": "dim_in", "val": ": int"}, {"name": "dim_out", "val": ": int"}, {"name": "bias", "val": ": bool = True"}]- dim_in (int) -- The number of channels in the input.

• dim_out (int) -- The number of channels in the output.
• bias (bool, defaults to True) -- Whether to use a bias in the linear layer.0

The approximate form of the Gaussian Error Linear Unit (GELU). For more details, see section 2 of this paper.

SwiGLU[[diffusers.models.activations.SwiGLU]]

class diffusers.models.activations.SwiGLUdiffusers.models.activations.SwiGLUhttps://github.com/huggingface/diffusers/blob/vr_12509/src/diffusers/models/activations.py#L126[{"name": "dim_in", "val": ": int"}, {"name": "dim_out", "val": ": int"}, {"name": "bias", "val": ": bool = True"}]- dim_in (int) -- The number of channels in the input.

• dim_out (int) -- The number of channels in the output.
• bias (bool, defaults to True) -- Whether to use a bias in the linear layer.0

A variant of the gated linear unit activation function. It's similar to GEGLU but uses SiLU / Swish instead of GeLU.

FP32SiLU[[diffusers.models.activations.FP32SiLU]]

class diffusers.models.activations.FP32SiLUdiffusers.models.activations.FP32SiLUhttps://github.com/huggingface/diffusers/blob/vr_12509/src/diffusers/models/activations.py#L53[]

SiLU activation function with input upcasted to torch.float32.

LinearActivation[[diffusers.models.activations.LinearActivation]]

class diffusers.models.activations.LinearActivationdiffusers.models.activations.LinearActivationhttps://github.com/huggingface/diffusers/blob/vr_12509/src/diffusers/models/activations.py#L169[{"name": "dim_in", "val": ": int"}, {"name": "dim_out", "val": ": int"}, {"name": "bias", "val": ": bool = True"}, {"name": "activation", "val": ": str = 'silu'"}]