Skip-gram log-likelihood with the full softmax (Kamath Ch 2)

sum log softmax(u_o . v_c) over the (center, context) pairs. Indices are 0-BASED.

Usage

morie_kamath_word2vec_skipgram(center_indices, context_indices, V, U)

Arguments

center_indices, context_indices

0-based vocabulary indices.

V, U

Input and output embedding matrices.

Value

List with log_likelihood, per_pair, probabilities.

Examples

morie_kamath_word2vec_skipgram(c(0, 1), c(1, 0), diag(2), diag(2))
#> $log_likelihood
#> [1] -2.626523
#> 
#> $mean_log_likelihood
#> [1] -1.313262
#> 
#> $per_pair
#> [1] -1.313262 -1.313262
#> 
#> $probabilities
#> [1] 0.2689414 0.2689414
#> 
#> $estimate
#> [1] -1.313262
#> 
#> $vocab_size
#> [1] 2
#> 
#> $n
#> [1] 2
#> 
#> $method
#> [1] "Skip-gram log-likelihood with the full softmax"
#>