Simple RNN forward pass over a sequence

h_t = tanh(W_x x_t + W_h h_{t-1} + b), unrolled step by step.

Usage

morie_geron_simple_rnn(X, Wx, Wh, b = NULL, h0 = NULL)

Arguments

X

Sequence (T, n_inputs).

Wx

Input weights (n_inputs, n_units).

Wh

Recurrent weights (n_units, n_units).

b

Optional bias; default zeros.

h0

Optional initial hidden state; default zeros.

Value

list with H, h_T, grads, jacobian_gain, n_units, estimate, n, method.

Examples

morie_geron_simple_rnn(matrix(c(1, 0), 2, 1), matrix(1, 1, 1),
    matrix(1, 1, 1))
#> $H
#>           [,1]
#> [1,] 0.7615942
#> [2,] 0.6420150
#> 
#> $h_T
#> [1] 0.642015
#> 
#> $grads
#>           [,1]
#> [1,] 0.4199743
#> [2,] 0.5878168
#> 
#> $jacobian_gain
#> [1] 0.246868
#> 
#> $n_units
#> [1] 1
#> 
#> $estimate
#> [1] 0.642015
#> 
#> $n
#> [1] 2
#> 
#> $method
#> [1] "Simple (Elman) RNN unrolled with tanh states from hmtanh"
#>