brainpy.losses module#

This module implements several loss functions.

Comparison#

 cross_entropy_loss(predicts, targets[, ...]) This criterion combines LogSoftmax and NLLLoss in one single class. cross_entropy_sparse(predicts, targets) Computes the softmax cross-entropy loss. cross_entropy_sigmoid(predicts, targets) Computes the sigmoid cross-entropy loss. l1_loos(logits, targets[, reduction]) Creates a criterion that measures the mean absolute error (MAE) between each element in the logits $$x$$ and targets $$y$$. l2_loss(predicts, targets) Computes the L2 loss. huber_loss(predicts, targets[, delta]) Huber loss. mean_absolute_error(x, y[, axis]) Computes the mean absolute error between x and y. mean_squared_error(predicts, targets[, axis]) Computes the mean squared error between x and y. mean_squared_log_error(predicts, targets[, axis]) Computes the mean squared logarithmic error between y_true and y_pred. binary_logistic_loss(predicts, targets) Binary logistic loss. multiclass_logistic_loss(label, logits) Multiclass logistic loss. sigmoid_binary_cross_entropy(logits, labels) Computes sigmoid cross entropy given logits and multiple class labels. softmax_cross_entropy(logits, labels) Computes the softmax cross entropy between sets of logits and labels. log_cosh_loss(predicts, targets) Calculates the log-cosh loss for a set of predictions. ctc_loss_with_forward_probs(logits, ...[, ...]) Computes CTC loss and CTC forward-probabilities. The CTC loss is a loss function based on log-likelihoods of the model that introduces a special blank symbol $$\phi$$ to represent variable-length output sequences. Forward probabilities returned by this function, as auxiliary results, are grouped into two part: blank alpha-probability and non-blank alpha probability. Those are defined as follows: .. math:: alpha_{mathrm{BLANK}}(t, n) = sum_{pi_{1:t-1}} p(pi_t = phi | pi_{1:t-1}, y_{1:n-1}, cdots), \ alpha_{mathrm{LABEL}}(t, n) = sum_{pi_{1:t-1}} p(pi_t = y_n | pi_{1:t-1}, y_{1:n-1}, cdots). Here, $$\pi$$ denotes the alignment sequence in the reference [Graves et al, 2006] that is blank-inserted representations of labels. The return values are the logarithms of the above probabilities. .. rubric:: References. ctc_loss(logits, logit_paddings, labels, ...) Computes CTC loss. See docstring for ctc_loss_with_forward_probs for details. :type logits: TypeVar(Array, JaxArray, Variable, TrainVar, Array, ndarray) :param logits: (B, T, K)-array containing logits of each class where B denotes the batch size, T denotes the max time frames in logits, and K denotes the number of classes including a class for blanks. :type logit_paddings: TypeVar(Array, JaxArray, Variable, TrainVar, Array, ndarray) :param logit_paddings: (B, T)-array. Padding indicators for logits. Each element must be either 1.0 or 0.0, and logitpaddings[b, t] == 1.0 denotes that logits[b, t, :] are padded values. :type labels: TypeVar(Array, JaxArray, Variable, TrainVar, Array, ndarray) :param labels: (B, N)-array containing reference integer labels where N denotes the max time frames in the label sequence. :type label_paddings: TypeVar(Array, JaxArray, Variable, TrainVar, Array, ndarray) :param label_paddings: (B, N)-array. Padding indicators for labels. Each element must be either 1.0 or 0.0, and labelpaddings[b, n] == 1.0 denotes that labels[b, n] is a padded label. In the current implementation, labels must be right-padded, i.e. each row labelpaddings[b, :] must be repetition of zeroes, followed by repetition of ones. :type blank_id: int :param blank_id: Id for blank token. logits[b, :, blank_id] are used as probabilities of blank symbols. :type log_epsilon: float :param log_epsilon: Numerically-stable approximation of log(+0).

Regularization#

 l2_norm(x[, axis]) Computes the L2 loss. mean_absolute(outputs[, axis]) Computes the mean absolute error between x and y. mean_square(predicts[, axis]) log_cosh(errors) Calculates the log-cosh loss for a set of predictions. smooth_labels`(labels, alpha) Apply label smoothing.