Batch normalization, Ioffe-Szegedy running form (Geron Ch 11, morie.fn grbn) — morie_geron_batch_normalization

Same normalisation as morie_geron_batch_normalization() but grbn's running-statistic convention: defaults are zeros/ones, the update is optional (only when momentum is given), and it uses the BIASED batch variance rather than the unbiased one.

Usage

morie_geron_batch_normalization_paper(
  X,
  gamma,
  beta,
  eps = 1e-05,
  momentum = NULL,
  running_mean = NULL,
  running_var = NULL
)

Arguments

X: Mini-batch (m, d).
gamma, beta: Scale and shift.
eps: Variance floor.
momentum: Optional EMA factor in [0, 1).
running_mean, running_var: Optional statistics to update.

Value

List with Y, x_hat, batch_mean, batch_var, running_mean, running_var.

Examples

morie_geron_batch_normalization_paper(X = c(1, 2, 3, 4, 5, 6, 7, 8), gamma = 0.5, beta = 0.5)
#> $Y
#>             [,1]
#> [1,] -0.26376189
#> [2,] -0.04554421
#> [3,]  0.17267348
#> [4,]  0.39089116
#> [5,]  0.60910884
#> [6,]  0.82732652
#> [7,]  1.04554421
#> [8,]  1.26376189
#> 
#> $x_hat
#>            [,1]
#> [1,] -1.5275238
#> [2,] -1.0910884
#> [3,] -0.6546530
#> [4,] -0.2182177
#> [5,]  0.2182177
#> [6,]  0.6546530
#> [7,]  1.0910884
#> [8,]  1.5275238
#> 
#> $batch_mean
#> [1] 4.5
#> 
#> $batch_var
#> [1] 5.25
#> 
#> $running_mean
#> [1] 0
#> 
#> $running_var
#> [1] 1
#> 
#> $eps
#> [1] 1e-05
#> 
#> $estimate
#> [1] 0.5
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "Batch normalization (Ioffe & Szegedy 2015)"
#>