brainpy.optim.Adam#

class brainpy.optim.Adam(lr, train_vars=None, beta1=0.9, beta2=0.999, eps=1e-08, weight_decay=None, name=None)[source]#

Optimizer that implements the Adam algorithm.

Adam 6 - a stochastic gradient descent method (SGD) that computes individual adaptive learning rates for different parameters from estimates of first- and second-order moments of the gradients.

Parameters

  • lr (float, Scheduler) – learning rate.

  • beta1 (optional, float) – A positive scalar value for beta_1, the exponential decay rate for the first moment estimates (default 0.9).

  • beta2 (optional, float) – A positive scalar value for beta_2, the exponential decay rate for the second moment estimates (default 0.999).

  • eps (optional, float) – A positive scalar value for epsilon, a small constant for numerical stability (default 1e-8).

  • name (optional, str) – The optimizer name.

References

6

Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.

__init__(lr, train_vars=None, beta1=0.9, beta2=0.999, eps=1e-08, weight_decay=None, name=None)[source]#

Methods

__init__(lr[, train_vars, beta1, beta2, ...])

check_grads(grads)

cpu()

Move all variable into the CPU device.

cuda()

Move all variables into the GPU device.

load_state_dict(state_dict[, warn])

Copy parameters and buffers from state_dict into this module and its descendants.

load_states(filename[, verbose])

Load the model states.

nodes([method, level, include_self])

Collect all children nodes.

register_implicit_nodes(*nodes[, node_cls])

register_implicit_vars(*variables, ...)

register_train_vars([train_vars])

register_vars([train_vars])

save_states(filename[, variables])

Save the model states.

state_dict()

Returns a dictionary containing a whole state of the module.

to(device)

Moves all variables into the given device.

tpu()

Move all variables into the TPU device.

train_vars([method, level, include_self])

The shortcut for retrieving all trainable variables.

tree_flatten()

Flattens the object as a PyTree.

tree_unflatten(aux, dynamic_values)

New in version 2.3.1.

unique_name([name, type_])

Get the unique name for this object.

update(grads)

vars([method, level, include_self, ...])

Collect all variables in this node and the children nodes.

Attributes

name

Name of the model.