Leaky ReLU (Geron Ch 11, morie.fn hmlrel)

max(alpha z, z) with alpha in [0, 1). alpha = 0 is a plain ReLU and is flagged, because that is the dying-unit case.

Usage

morie_geron_leaky_relu(z, alpha = 0.01)

Arguments

z

Numeric vector.

alpha

Leak slope in [0, 1).

Value

List with activation, derivative, n_leaky, alpha, warnings, estimate, n.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
morie_geron_leaky_relu(V)
#> $activation
#> [1] 1 2 3 4 5 6 7 8
#> 
#> $derivative
#> [1] 1 1 1 1 1 1 1 1
#> 
#> $n_leaky
#> [1] 0
#> 
#> $alpha
#> [1] 0.01
#> 
#> $warnings
#> character(0)
#> 
#> $estimate
#> [1] 4.5
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "Leaky ReLU activation"
#>