brainpy.optimizers.MomentumNesterov#

class brainpy.optimizers.MomentumNesterov(lr, train_vars=None, momentum=0.9, name=None)[source]#

Nesterov accelerated gradient optimizer 2.

\[\begin{split}\begin{align} \begin{split} v_t &= \gamma v_{t-1} + \eta \nabla_\theta J( \theta - \gamma v_{t-1} ) \\ \theta &= \theta - v_t \end{split} \end{align}\end{split}\]

References

2

Nesterov, Y. (1983). A method for unconstrained convex minimization problem with the rate of convergence o(1/k2). Doklady ANSSSR (translated as Soviet.Math.Docl.), vol. 269, pp. 543– 547.

__init__(lr, train_vars=None, momentum=0.9, name=None)[source]#

Methods

__init__(lr[, train_vars, momentum, name])

check_grads(grads)

load_states(filename[, verbose])

Load the model states.

nodes([method, level, include_self])

Collect all children nodes.

register_implicit_nodes(*nodes, **named_nodes)

register_implicit_vars(*variables, ...)

register_vars([train_vars])

save_states(filename[, variables])

Save the model states.

train_vars([method, level, include_self])

The shortcut for retrieving all trainable variables.

unique_name([name, type_])

Get the unique name for this object.

update(grads)

vars([method, level, include_self])

Collect all variables in this node and the children nodes.

Attributes

name

Name of the model.