brainpy.nn.nodes.ANN.GroupNorm

class brainpy.nn.nodes.ANN.GroupNorm(num_groups=None, group_size=None, epsilon=1e-05, use_bias=True, use_scale=True, beta_init=ZeroInit, gamma_init=OneInit(value=1.0), axis=None, **kwargs)

Group normalization layer.

This layer divides channels into groups and normalizes the features within each group. Its computation is also independent of the batch size. The feature size must be multiple of the group size.

The shape of the data should be (b, d1, d2, …, c), where d denotes the batch size and c denotes the feature (channel) size. The d and c axis should be excluded in parameter axis.

Parameters

• num_groups (int) – the number of groups. It should be a factor of the number of features.

• group_size (int) – the group size. It should equal to int(num_features / num_groups). Either num_groups or group_size should be specified.

• epsilon (float) – a value added to the denominator for numerical stability. Default: 1e-5

• use_bias (bool) – whether to translate data in refactoring. Default: True

• use_scale (bool) – whether to scale data in refactoring. Default: True

• beta_init (brainpy.init.Initializer) – an initializer generating the original translation matrix

• gamma_init (brainpy.init.Initializer) – an initializer generating the original scaling matrix

• axis (int, tuple, list) – axes where the data will be normalized. Besides the batch axis, the channel axis should be also excluded, since it will be automatically added to axis.

__init__(num_groups=None, group_size=None, epsilon=1e-05, use_bias=True, use_scale=True, beta_init=ZeroInit, gamma_init=OneInit(value=1.0), axis=None, **kwargs)

Methods

__init__([num_groups, group_size, epsilon, ...])
copy([name, shallow])	Returns a copy of the Node.
feedback(ff_output, **shared_kwargs)	The feedback computation function of a node.
forward(ff, **shared_kwargs)	The feedforward computation function of a node.
init_fb_conn()	Initialize the feedback connections.
init_fb_output([num_batch])	Set the initial node feedback state.
init_ff_conn()	Initialize the feedforward connections.
init_state([num_batch])	Set the initial node state.
initialize([num_batch])	Initialize the node.
load_states(filename[, verbose])	Load the model states.
nodes([method, level, include_self])	Collect all children nodes.
offline_fit(targets, ffs[, fbs])	Offline training interface.
online_fit(target, ff[, fb])	Online training fitting interface.
online_init()	Online training initialization interface.
register_implicit_nodes(nodes)
register_implicit_vars(variables)
save_states(filename[, variables])	Save the model states.
set_fb_output(state)	Safely set the feedback state of the node.
set_feedback_shapes(fb_shapes)
set_feedforward_shapes(feedforward_shapes)
set_output_shape(shape)
set_state(state)	Safely set the state of the node.
train_vars([method, level, include_self])	The shortcut for retrieving all trainable variables.
unique_name([name, type_])	Get the unique name for this object.
vars([method, level, include_self])	Collect all variables in this node and the children nodes.

Attributes

data_pass	Offline fitting method.
fb_output	rtype Optional[TypeVar(Tensor, JaxArray, ndarray)]
feedback_shapes	Output data size.
feedforward_shapes	Input data size.
is_feedback_input_supported
is_feedback_supported
is_initialized	rtype bool
name
output_shape	Output data size.
state	Node current internal state.
trainable	Returns if the Node can be trained.