GloVe weighted least-squares cost (Kamath Ch 2) — morie_kamath_glove

sum f(X_ij) (w_i . w~_j + b_i + b~_j - log X_ij)^2 over the NON-ZERO counts only.

Usage

morie_kamath_glove_cost(X, W, W_tilde, b, b_tilde, x_max = 100, alpha = 0.75)

morie_kamath_glove_weight(x, x_max, alpha)

Arguments

X: Co-occurrence counts.
W, W_tilde: Word and context vectors.
b, b_tilde: Biases.
x_max, alpha: Weighting parameters.
x: Counts to weight.

Value

List with estimate, weights, residuals, n_nonzero.

Examples

morie_kamath_glove_cost(matrix(c(2, 1, 0, 3), 2), matrix(c(1,
    0), 2), matrix(c(1, 1), 2), c(0, 0), c(0, 0), x_max = 2,
    alpha = 0.5)
#> $estimate
#> [1] 1.301108
#> 
#> $cost
#> [1] 1.301108
#> 
#> $weights
#>           [,1] [,2]
#> [1,] 1.0000000    0
#> [2,] 0.7071068    1
#> 
#> $residuals
#>           [,1]      [,2]
#> [1,] 0.3068528  0.000000
#> [2,] 0.0000000 -1.098612
#> 
#> $n_nonzero
#> [1] 3
#> 
#> $n
#> [1] 4
#> 
#> $method
#> [1] "GloVe weighted least-squares cost"
#>