GRU cell forward pass (Geron Ch 15, morie.fn grgruc)

PORTED AS THE PYTHON HAS IT: this module uses the tied-gate convention h = (1 - z) h_prev + z h_tilde (rather than the more common z h_prev + (1 - z) h_tilde), the gates act on the concatenation [h, x] and the candidate on [r * h, x], and there are no biases.

Usage

morie_geron_gru_cell(x_t, h_prev, Wz, Wr, W)

Arguments

x_t

Input vector.

h_prev

Previous hidden state.

Wz, Wr, W

(H, H + n) weight matrices.

Value

List with h, z, r, h_tilde, update_fraction.

Examples

x_t <- c(0.5, -0.1)
h_prev <- c(0.2, 0.3)
Wz <- matrix(0.1, 2, 4); Wr <- matrix(0.1, 2, 4); W <- matrix(0.1, 2, 4)
morie_geron_gru_cell(x_t, h_prev, Wz, Wr, W)
#> $h
#> [1] 0.1300017 0.1777532
#> 
#> $z
#> [1] 0.5224848 0.5224848
#> 
#> $r
#> [1] 0.5224848 0.5224848
#> 
#> $h_tilde
#> [1] 0.06602804 0.06602804
#> 
#> $update_fraction
#> [1] 0.5224848
#> 
#> $estimate
#> [1] 0.1300017 0.1777532
#> 
#> $n
#> [1] 2
#>