He (Kaiming) initialization (Geron Ch 11, morie.fn hmhei) — morie_geron_he_init

RNG substitute: LCG uniform stream mapped through qnorm (normal) or an affine map (uniform); not bit-portable to numpy's PCG64. Exact anchors use the deterministic targets (var_target/std_target/limit/shape); the draw itself is checked only statistically.

Usage

morie_geron_he_init_hmhei(
  fan_in,
  seed = 0,
  fan_out = NULL,
  distribution = "normal"
)

Arguments

fan_in, fan_out: Layer widths.
seed: Seed.
distribution: "normal" or "uniform".

Value

List with W, std_target, var_target, limit, empirical_std, estimate, n, method.

Examples

morie_geron_he_init_hmhei(fan_in = 5L)
#> $W
#>             [,1]        [,2]       [,3]        [,4]       [,5]
#> [1,] -0.45474063 -0.37131721  0.5778049  0.27450507 -0.1864412
#> [2,]  0.19620754 -0.25444665  0.2267694  0.01233165  0.1305984
#> [3,]  0.52492665  0.40215208  0.1829774  0.75955182 -1.5623800
#> [4,]  0.02339013  0.34581003 -0.3106037 -0.18653867  0.2346362
#> [5,]  0.03942896  0.06623773 -0.1597194 -0.43223168 -1.4192071
#> 
#> $std_target
#> [1] 0.6324555
#> 
#> $var_target
#> [1] 0.4
#> 
#> $limit
#> [1] 1.095445
#> 
#> $empirical_std
#> [1] 0.525975
#> 
#> $fan_in
#> [1] 5
#> 
#> $fan_out
#> [1] 5
#> 
#> $distribution
#> [1] "normal"
#> 
#> $estimate
#> [1] 0.6324555
#> 
#> $n
#> [1] 25
#> 
#> $method
#> [1] "He (Kaiming) initialization"
#>