brainpy.optimizers.MomentumNesterov
brainpy.optimizers.MomentumNesterov#
- class brainpy.optimizers.MomentumNesterov(lr, train_vars=None, momentum=0.9, name=None)[source]#
Nesterov accelerated gradient optimizer 2.
\[\begin{split}\begin{align} \begin{split} v_t &= \gamma v_{t-1} + \eta \nabla_\theta J( \theta - \gamma v_{t-1} ) \\ \theta &= \theta - v_t \end{split} \end{align}\end{split}\]References
- 2
Nesterov, Y. (1983). A method for unconstrained convex minimization problem with the rate of convergence o(1/k2). Doklady ANSSSR (translated as Soviet.Math.Docl.), vol. 269, pp. 543– 547.
Methods
__init__
(lr[, train_vars, momentum, name])check_grads
(grads)load_states
(filename[, verbose])Load the model states.
nodes
([method, level, include_self])Collect all children nodes.
register_implicit_nodes
(nodes)register_implicit_vars
(variables)register_vars
([train_vars])save_states
(filename[, variables])Save the model states.
train_vars
([method, level, include_self])The shortcut for retrieving all trainable variables.
unique_name
([name, type_])Get the unique name for this object.
update
(grads)vars
([method, level, include_self])Collect all variables in this node and the children nodes.
Attributes
name