brainpy.dyn.layers.GroupNorm#

class brainpy.dyn.layers.GroupNorm(num_groups, num_channels, epsilon=1e-05, affine=True, bias_init=ZeroInit, scale_init=OneInit(value=1.0), mode=TrainingMode, name=None)[source]#

Group normalization layer.

\[y =\]

rac{x - mathrm{E}[x]}{ sqrt{mathrm{Var}[x] + epsilon}} * gamma + eta

This layer divides channels into groups and normalizes the features within each group. Its computation is also independent of the batch size. The feature size must be multiple of the group size.

The shape of the data should be (b, d1, d2, …, c), where d denotes the batch size and c denotes the feature (channel) size.

num_groups: int

The number of groups. It should be a factor of the number of channels.

num_channels: int

The number of channels expected in input.

epsilon: float

a value added to the denominator for numerical stability. Default: 1e-5

affine: bool

A boolean value that when set to True, this module has learnable per-channel affine parameters initialized to ones (for weights) and zeros (for biases). Default: True.

bias_init: Initializer

An initializer generating the original translation matrix

scale_init: Initializer

An initializer generating the original scaling matrix

>>> import brainpy as bp
>>> import brainpy.math as bm
>>> input = bm.random.randn(20, 10, 10, 6)
>>> # Separate 6 channels into 3 groups
>>> m = bp.layers.GroupNorm(3, 6)
>>> # Separate 6 channels into 6 groups (equivalent with InstanceNorm)
>>> m = bp.layers.GroupNorm(6, 6)
>>> # Put all 6 channels into a single group (equivalent with LayerNorm)
>>> m = bp.layers.GroupNorm(1, 6)
>>> # Activating the module
>>> output = m(input)
__init__(num_groups, num_channels, epsilon=1e-05, affine=True, bias_init=ZeroInit, scale_init=OneInit(value=1.0), mode=TrainingMode, name=None)[source]#

Methods

__init__(num_groups, num_channels[, ...])

clear_input()

get_delay_data(identifier, delay_step, *indices)

Get delay data according to the provided delay steps.

load_states(filename[, verbose])

Load the model states.

nodes([method, level, include_self])

Collect all children nodes.

offline_fit(target, fit_record)

offline_init()

online_fit(target, fit_record)

online_init()

register_delay(identifier, delay_step, ...)

Register delay variable.

register_implicit_nodes(*nodes, **named_nodes)

register_implicit_vars(*variables, ...)

reset([batch_size])

Reset function which reset the whole variables in the model.

reset_local_delays([nodes])

Reset local delay variables.

reset_state([batch_size])

Reset function which reset the states in the model.

save_states(filename[, variables])

Save the model states.

train_vars([method, level, include_self])

The shortcut for retrieving all trainable variables.

unique_name([name, type_])

Get the unique name for this object.

update(sha, x)

The function to specify the updating rule.

update_local_delays([nodes])

Update local delay variables.

vars([method, level, include_self])

Collect all variables in this node and the children nodes.

Attributes

global_delay_data

mode

Mode of the model, which is useful to control the multiple behaviors of the model.

name

Name of the model.