Inverted dropout

Keeps each unit with probability p and rescales survivors by 1/p, so the expected activation is unchanged and inference needs no adjustment – training = FALSE is exactly the identity. Note p is the KEEP probability, following Srivastava et al.; libraries that take the drop probability instead will scale every activation by p^2 if the two conventions are mixed.

Usage

morie_esl_dropout(X, p = 0.5, training = TRUE, seed = 0L)

Arguments

X

Activations, n by d.

p

Keep probability in (0, 1].

training

When FALSE the input is returned unchanged.

seed

Seed for the mask.

Value

List with output, mask, kept_fraction, scale.

References

Srivastava, N., et al. (2014). Dropout: A simple way to prevent neural networks from overfitting. JMLR 15, 1929-1958.

Examples

sort(unique(round(as.numeric(morie_esl_dropout(matrix(1, 20, 5), p = 0.5)$output), 6)))
#> [1] 0 2