Skip to contents

Layer norm used by the transformer blocks (morie.fn grteb)

Source: R/geron_train_native.R
morie_geron_block_layer_norm.Rd

Layer norm used by the transformer blocks (morie.fn grteb)

Usage

morie_geron_block_layer_norm(X, gamma = NULL, beta = NULL, eps = 1e-05)

Arguments

X

(T, d) matrix.

gamma, beta

Optional length-d scale and shift.

eps

Positive variance floor.

Value

Normalized matrix.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
morie_geron_block_layer_norm(V)
#>           [,1]      [,2]      [,3]       [,4]      [,5]     [,6]     [,7]
#> [1,] -1.527524 -1.091088 -0.654653 -0.2182177 0.2182177 0.654653 1.091088
#>          [,8]
#> [1,] 1.527524