LSTM vocabulary softmax and Affect-LM (Kamath Eq 6.27-6.28)

Usage

morie_kamath_ch6_lstm_softmax_word(U, f, c_t_1, b)

morie_kamath_ch6_affect_lm(U, V, f, g, c, e, beta, b)

Arguments

U: Vocabulary projection.
f: Hidden map or vector.
c_t_1: Previous cell state.
b: Bias.
V: Affect projection.
g: Affect hidden map.
c: Cell state.
e: Affect category.
beta: Affect strength.

Value

List with p, logits, argmax.

Details

One softmax core serves both Python modules exactly as km104 imports km103's _softmax_logits/_hidden: km103 -> morie_kamath_ch6_lstm_softmax_word (softmax(U f(c) + b)), km104 -> morie_kamath_ch6_affect_lm (the same, shifted by beta V g(e)). argmax is 0-BASED.

Examples

morie_kamath_ch6_lstm_softmax_word(U = 5L, f = 5L, c_t_1 = c(1, 2, 3, 4, 5, 6, 7, 8), b = 5L)
#> $p
#> [1] 1
#> 
#> $logits
#> [1] 30
#> 
#> $argmax
#> [1] 0
#> 
#> $estimate
#> [1] 1
#> 
#> $n
#> [1] 1
#> 
#> $method
#> [1] "LSTM vocabulary softmax (Kamath Eq 6.27)"
#>