Binary sentiment head (Geron Ch 16, morie.fn grsnt)

p = sigmoid(w . pool(E[ids]) + b); mean pooling loses order, max acts as a keyword detector. Token ids are 0-based.

Usage

morie_geron_sentiment_binary(
  token_ids,
  E,
  w,
  b = 0,
  pooling = "mean",
  threshold = 0.5
)

Arguments

token_ids

0-based ids.

E

(V, d) embedding table.

w

Length-d weights.

b

Finite bias.

pooling

"mean", "max" or "sum".

threshold

Label cut in [0, 1].

Value

List with probability, label, logit, pooled, token_contributions.

Examples

set.seed(6)
E <- matrix(rnorm(20), 5, 4)
morie_geron_sentiment_binary(c(0L, 3L, 2L), E, w = c(0.5, -0.2, 0.1, 0.3),
                             b = 0.1)
#> $probability
#> [1] 0.6070639
#> 
#> $label
#> [1] 1
#> 
#> $logit
#> [1] 0.434987
#> 
#> $pooled
#> [1]  0.9551538  0.3838400  0.6708304 -0.4430165
#> 
#> $token_contributions
#> [1] 0.25036461 0.73098196 0.02361431
#> 
#> $estimate
#> [1] 0.6070639
#> 
#> $n
#> [1] 3
#>