models.decoder
class
Decoder(keras.src.layers.layer.Layer):
Decoder layer in a Transformer model architecture.
This layer implements the decoder component of the Transformer model, which is responsible for generating the output sequence based on the encoded input sequence and previously generated output tokens.
Parameters:
- dropout_rate (float): Dropout rate applied to the outputs of each sub-layer. Default is 0.2.
- num_heads (int): Number of attention heads. Default is 32.
- head_dims (int): Dimensionality of each attention head. Default is 40.
- fc_dim_factor (int): Factor controlling the dimensionality of the fully connected layers. Default is 5.
- input_len (int): Length of the input sequence. Default is 64.
References: - Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems 30 (2017).
Example:
>>> decoder = Decoder()
>>> output = decoder(keras.ops.ones((1, 10, 1280)
>>> print(output)
Decoder( dropout_rate=0.2, num_heads=32, head_dims=40, fc_dim_factor=5, input_len=64)
Initializes the Decoder layer.
Args:
- dropout_rate (float): Dropout rate applied to the outputs of each sub-layer. Default is 0.2.
- num_heads (int): Number of attention heads. Default is 32.
- head_dims (int): Dimensionality of each attention head. Default is 40.
- fc_dim_factor (int): Factor controlling the dimensionality of the fully connected layers. Default is 5.
- input_len (int): Length of the input sequence. Default is 64.
def
call(self, inputs):
Executes the forward pass of the Decoder layer.
Args:
- inputs: Input tensor.
Returns:
- keras.Tensor: Output tensor.
Inherited Members
- keras.src.layers.layer.Layer
- get_build_config
- build_from_config
- add_variable
- add_weight
- trainable
- variables
- trainable_variables
- non_trainable_variables
- weights
- trainable_weights
- non_trainable_weights
- metrics
- metrics_variables
- get_weights
- set_weights
- dtype
- compute_dtype
- variable_dtype
- input_dtype
- supports_masking
- stateless_call
- add_loss
- losses
- save_own_variables
- load_own_variables
- count_params
- get_config
- keras.src.ops.operation.Operation
- from_config
- input
- output