Image-text matching head (Kamath Ch 9) — morie_kamath_image_text

p = sigmoid(w . [I; T] + b), or a caller-supplied fusion.

Usage

morie_kamath_image_text_matching(image_emb, text_emb, W, b, fuse = NULL)

Arguments

image_emb, text_emb: Embeddings.
W: Head weights.
b: Bias.
fuse: Optional fusion function.

Value

List with estimate, logit, match, fused.

Examples

morie_kamath_image_text_matching(image_emb = list(a = 1, b = 2),
  text_emb = matrix(c(1, 2, 3, 4, 5, 6), nrow = 2), W = c(1, 2, 3, 4, 5, 6, 7, 8), b = 5L)
#> $estimate
#> [1] 1
#> 
#> $probability
#> [1] 1
#> 
#> $logit
#> [1] 143
#> 
#> $match
#> [1] TRUE
#> 
#> $fused
#> [1] 1 2 1 2 3 4 5 6
#> 
#> $fusion
#> [1] "concatenation [I; T]"
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "Image-text matching head p = sigmoid(w.fused + b)"
#>