brainpy.optimizers.Adam#

class brainpy.optimizers.Adam(lr, train_vars=None, beta1=0.9, beta2=0.999, eps=1e-08, name=None)[source]#

Optimizer that implements the Adam algorithm.

Adam 6 - a stochastic gradient descent method (SGD) that computes individual adaptive learning rates for different parameters from estimates of first- and second-order moments of the gradients.

Parameters

  • beta1 (optional, float) – A positive scalar value for beta_1, the exponential decay rate for the first moment estimates (default 0.9).

  • beta2 (optional, float) – A positive scalar value for beta_2, the exponential decay rate for the second moment estimates (default 0.999).

  • eps (optional, float) – A positive scalar value for epsilon, a small constant for numerical stability (default 1e-8).

  • name (optional, str) – The optimizer name.

References

6

Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.

__init__(lr, train_vars=None, beta1=0.9, beta2=0.999, eps=1e-08, name=None)[source]#

Methods

__init__(lr[, train_vars, beta1, beta2, ...])

check_grads(grads)

load_states(filename[, verbose])

Load the model states.

nodes([method, level, include_self])

Collect all children nodes.

register_implicit_nodes(nodes)

register_implicit_vars(variables)

register_vars([train_vars])

save_states(filename[, variables])

Save the model states.

train_vars([method, level, include_self])

The shortcut for retrieving all trainable variables.

unique_name([name, type_])

Get the unique name for this object.

update(grads)

vars([method, level, include_self])

Collect all variables in this node and the children nodes.

Attributes

name