Reservoir

Reservoir#

class brainpy.dyn.Reservoir(input_shape, num_out, leaky_rate=0.3, activation='tanh', activation_type='internal', Win_initializer=Normal(scale=0.1, rng=RandomState(Array((), dtype=key<fry>) overlaying: [ 216744582 1008666480])), Wrec_initializer=Normal(scale=0.1, rng=RandomState(Array((), dtype=key<fry>) overlaying: [ 216744582 1008666480])), b_initializer=ZeroInit, in_connectivity=0.1, rec_connectivity=0.1, comp_type='dense', spectral_radius=None, noise_in=0.0, noise_rec=0.0, noise_type='normal', mode=None, name=None)[source]#

Reservoir node, a pool of leaky-integrator neurons with random recurrent connections [1].

Parameters:

  • input_shape (Union[int, Tuple[int]]) – The input shape.

  • num_out (int) – The number of reservoir nodes.

  • Win_initializer (Union[Initializer, Callable, TypeVar(ArrayType, Array, Variable, TrainVar, Array, ndarray)]) – The initialization method for the feedforward connections.

  • Wrec_initializer (Union[Initializer, Callable, TypeVar(ArrayType, Array, Variable, TrainVar, Array, ndarray)]) – The initialization method for the recurrent connections.

  • b_initializer (Union[Callable, TypeVar(ArrayType, Array, Variable, TrainVar, Array, ndarray), Initializer, None]) – The initialization method for the bias.

  • leaky_rate (float) – A float between 0 and 1.

  • activation (Union[str, Callable]) –

    Reservoir activation function.

    • If a str, should be a brainpy.math.activations function name.

    • If a callable, should be an element-wise operator.

  • activation_type (str) –

    • If “internal” (default), then leaky integration happens on states transformed by the activation function:

    \[r[n+1] = (1 - \alpha) \cdot r[t] + \alpha \cdot f(W_{ff} \cdot u[n] + W_{fb} \cdot b[n] + W_{rec} \cdot r[t])\]

    • If “external”, then leaky integration happens on internal states of each neuron, stored in an internal_state parameter (\(x\) in the equation below). A neuron internal state is the value of its state before applying the activation function \(f\):

      \[\begin{split}x[n+1] &= (1 - \alpha) \cdot x[t] + \alpha \cdot f(W_{ff} \cdot u[n] + W_{rec} \cdot r[t] + W_{fb} \cdot b[n]) \\ r[n+1] &= f(x[n+1])\end{split}\]

  • in_connectivity (float) – Connectivity of input neurons, i.e. ratio of input neurons connected to reservoir neurons. Must be in [0, 1], by default 0.1

  • rec_connectivity (float) – Connectivity of recurrent weights matrix, i.e. ratio of reservoir neurons connected to other reservoir neurons, including themselves. Must be in [0, 1], by default 0.1

  • comp_type (str) –

    The connectivity type, can be “dense” or “sparse”, “jit”.

    • "dense" means the connectivity matrix is a dense matrix.

    • "sparse" means the connectivity matrix is a CSR sparse matrix.

  • spectral_radius (Optional[float]) – Spectral radius of recurrent weight matrix, by default None.

  • noise_rec (float) – Gain of noise applied to reservoir internal states, by default 0.0

  • noise_in (float) – Gain of noise applied to feedforward signals, by default 0.0

  • noise_type (str) – Distribution of noise. Must be a random variable generator distribution (see brainpy.math.random.RandomState), by default “normal”.

References

update(x)[source]#

Feedforward output.