ELU activation (Geron Ch 11, morie.fn hmelu)

ELU(z) = z for z >= 0, alpha * (exp(z) - 1) otherwise. The derivative below zero is a + alpha, which is why alpha = 1 is the only C1 member of the family.

Usage

morie_geron_elu(z, alpha = 1)

Arguments

z

Numeric vector of pre-activations.

alpha

Positive saturation scale.

Value

List with a, activation, derivative, saturation, alpha, is_c1, estimate, n.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
morie_geron_elu(V)
#> $a
#> [1] 1 2 3 4 5 6 7 8
#> 
#> $activation
#> [1] 1 2 3 4 5 6 7 8
#> 
#> $derivative
#> [1] 1 1 1 1 1 1 1 1
#> 
#> $saturation
#> [1] -1
#> 
#> $alpha
#> [1] 1
#> 
#> $is_c1
#> [1] TRUE
#> 
#> $estimate
#> [1] 4.5
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "ELU(z) = z if z >= 0 else alpha*(exp(z) - 1)"
#>