Glorot (Xavier) initialization (Geron Ch 11, morie.fn grxvi)

Var(W) = 2/(fan_in + fan_out); the uniform limit is sqrt(6/(in+out)). The normal branch consumes an even LCG budget, builds the cos half and the sin half SEPARATELY, and concatenates cos-block then sin-block before truncating – NOT the interleaved pair order used by grhei / grgrp. achieved_variance is the ddof = 0 variance. The weight matrix is reshaped ROW-MAJOR into (fan_in, fan_out).

Usage

morie_geron_glorot_xavier_init(
  fan_in,
  fan_out,
  distribution = "normal",
  seed = 42
)

Arguments

fan_in, fan_out

Positive widths.

distribution

"normal" or "uniform".

seed

LCG seed.

Value

List with weights, target_variance, achieved_variance, scale, distribution.

Examples

morie_geron_glorot_xavier_init(fan_in = 5L, fan_out = 5L)
#> $weights
#>            [,1]        [,2]         [,3]        [,4]       [,5]
#> [1,]  0.6312570  0.08044232  0.617683565 0.417280672  0.2321507
#> [2,] -0.2519980  0.27133148  0.382706433 0.843244059  0.1361076
#> [3,]  0.2066151 -0.02753292  0.066468199 0.390253143  0.4618434
#> [4,]  0.1004745  0.38432263 -0.007829657 0.000511853 -0.3219401
#> [5,]  0.2454108 -0.27346961 -0.113365041 0.442804090  0.4881747
#> 
#> $target_variance
#> [1] 0.2
#> 
#> $achieved_variance
#> [1] 0.08468917
#> 
#> $scale
#> [1] 0.4472136
#> 
#> $distribution
#> [1] "normal"
#> 
#> $estimate
#>            [,1]        [,2]         [,3]        [,4]       [,5]
#> [1,]  0.6312570  0.08044232  0.617683565 0.417280672  0.2321507
#> [2,] -0.2519980  0.27133148  0.382706433 0.843244059  0.1361076
#> [3,]  0.2066151 -0.02753292  0.066468199 0.390253143  0.4618434
#> [4,]  0.1004745  0.38432263 -0.007829657 0.000511853 -0.3219401
#> [5,]  0.2454108 -0.27346961 -0.113365041 0.442804090  0.4881747
#> 
#> $n
#> [1] 25
#>