Word embeddings: dense vector representations learned per token — morie_geron_word

Embedding table as a lookup (row i is the token's vector), LCG-init or a supplied pretrained matrix, plus the cosine-similarity matrix.

Usage

morie_geron_word_embeddings(vocab, d = 8, E = NULL, seed = 0)

Arguments

vocab: Character vector of distinct tokens.
d: Embedding width (ignored if E supplied).
E: Optional pretrained (V,d) matrix.
seed: LCG seed.

Value

list with E, lookup, index, vocab, norms, similarity, n_params, d, estimate, n, method. lookup is a function(token_or_char_vector).

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
morie_geron_word_embeddings(V)
#> $E
#>              [,1]        [,2]        [,3]         [,4]        [,5]        [,6]
#> [1,] -0.186628126 -0.15657684  0.22594449  0.118699822 -0.08196968  0.08613090
#> [2,]  0.005499949  0.05783960  0.20981661  0.167984246  0.08048923  0.27231816
#> [3,]  0.146889330 -0.13316575 -0.08201130  0.102303067  0.01757515  0.02949016
#> [4,] -0.344772981 -0.16586227  0.34192863  0.079363724 -0.03316687 -0.25908921
#> [5,] -0.091330893 -0.08708083  0.34733859  0.088599159  0.07746301 -0.23169209
#> [6,] -0.194292629 -0.29319199  0.06743904  0.223238200 -0.29540628 -0.23727957
#> [7,] -0.145488579  0.01254693 -0.25574730 -0.240866747  0.28328567 -0.03591670
#> [8,] -0.057496010 -0.01257849 -0.32525712 -0.009907236 -0.23855410  0.24690949
#>             [,7]        [,8]
#> [1,] -0.11050245  0.09902013
#> [2,] -0.34878096  0.01043035
#> [3,] -0.07048977 -0.17877690
#> [4,] -0.11094707 -0.33788974
#> [5,]  0.33700718  0.04588478
#> [6,] -0.28622900 -0.04812053
#> [7,] -0.32612410  0.08114409
#> [8,] -0.24907851 -0.03136765
#> 
#> $lookup
#> function (token) 
#> {
#>     if (length(token) > 1) {
#>         return(M[index[token] + 1, , drop = FALSE])
#>     }
#>     M[index[[token]] + 1, ]
#> }
#> <environment: 0x562159db0100>
#> 
#> $index
#> $index$`1`
#> [1] 0
#> 
#> $index$`2`
#> [1] 1
#> 
#> $index$`3`
#> [1] 2
#> 
#> $index$`4`
#> [1] 3
#> 
#> $index$`5`
#> [1] 4
#> 
#> $index$`6`
#> [1] 5
#> 
#> $index$`7`
#> [1] 6
#> 
#> $index$`8`
#> [1] 7
#> 
#> 
#> $vocab
#> [1] "1" "2" "3" "4" "5" "6" "7" "8"
#> 
#> $norms
#> [1] 0.4007995 0.5272666 0.3075872 0.6813945 0.5654903 0.6371056 0.5824345
#> [8] 0.5387492
#> 
#> $similarity
#>             [,1]        [,2]        [,3]        [,4]        [,5]        [,6]
#> [1,]  1.00000000  0.53800261 -0.17653588  0.49870270  0.26769952  0.60522314
#> [2,]  0.53800261  1.00000000  0.15568894  0.09889921 -0.30755036  0.13264997
#> [3,] -0.17653588  0.15568894  1.00000000  0.05498811 -0.33731052  0.22657369
#> [4,]  0.49870270  0.09889921  0.05498811  1.00000000  0.45752429  0.63503754
#> [5,]  0.26769952 -0.30755036 -0.33731052  0.45752429  1.00000000  0.05524285
#> [6,]  0.60522314  0.13264997  0.22657369  0.63503754  0.05524285  1.00000000
#> [7,] -0.18604307  0.10881402 -0.07986326 -0.12550595 -0.52797258 -0.08664054
#> [8,]  0.01515456  0.22400269  0.27242844 -0.29397317 -0.88155475  0.21966831
#>             [,7]        [,8]
#> [1,] -0.18604307  0.01515456
#> [2,]  0.10881402  0.22400269
#> [3,] -0.07986326  0.27242844
#> [4,] -0.12550595 -0.29397317
#> [5,] -0.52797258 -0.88155475
#> [6,] -0.08664054  0.21966831
#> [7,]  1.00000000  0.30598937
#> [8,]  0.30598937  1.00000000
#> 
#> $n_params
#> [1] 64
#> 
#> $d
#> [1] 8
#> 
#> $estimate
#> [1] 64
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "Embedding lookup table with cosine-similarity geometry"
#>