Skip to contents

Glorot (Xavier) initialization (Geron Ch 11, morie.fn hmxav)

Source: R/geron_ml2_native.R
morie_geron_glorot_init.Rd

Var(W) = 2/(fan_in + fan_out). The uniform arm draws the LCG stream draw-for-draw; the normal arm draws a SECOND stream seeded seed + 7919 and combines the two by Box-Muller, exactly as the Python module does. The reshape is row-major, so byrow = TRUE.

Usage

morie_geron_glorot_init(fan_in, fan_out, seed = 0, distribution = "uniform")

Arguments

fan_in, fan_out

Positive layer widths.

seed

LCG seed.

distribution

"uniform" or "normal".

Value

List with W, limit, std, variance, fan_in, fan_out, distribution, estimate, n.

Examples

morie_geron_glorot_init(fan_in = 5L, fan_out = 5L)
#> $W
#>             [,1]        [,2]       [,3]        [,4]       [,5]
#> [1,] -0.40888174 -0.34304267  0.4950196  0.26005828 -0.1795866
#> [2,]  0.18870335 -0.24209875  0.2169422  0.01204978  0.1267202
#> [3,]  0.45968516  0.36803504  0.1763431  0.59661920 -0.7641408
#> [4,]  0.02285174  0.32181840 -0.2917515 -0.17967775  0.2241348
#> [5,]  0.03850523  0.06460971 -0.1544354 -0.39168057 -0.7553598
#> 
#> $limit
#> [1] 0.7745967
#> 
#> $std
#> [1] 0.4472136
#> 
#> $variance
#> [1] 0.2
#> 
#> $fan_in
#> [1] 5
#> 
#> $fan_out
#> [1] 5
#> 
#> $distribution
#> [1] "uniform"
#> 
#> $estimate
#> [1] 0.2
#> 
#> $n
#> [1] 25
#> 
#> $method
#> [1] "Glorot uniform initialization, Var(W) = 2/(fan_in + fan_out)"
#>