SimVLM masked-token loss with visual conditioning (Kamath Eq 9.10) — morie_kamath_ch9_simvlm_mlm • rmorie

Delegates the masked-token arithmetic to morie_kamath_mlm_loss (Ch 2, km022) exactly as the Python module imports it. x_m is 0-BASED.

Usage

morie_kamath_ch9_simvlm_mlm(theta, x, v, x_m)

Arguments

theta: Optional model theta(x, v).
x: Token probabilities.
v: Image-region features.
x_m: 0-based masked positions.

Value

List with estimate, per_position, n_image_regions.

Examples

morie_kamath_ch9_simvlm_mlm(NULL, c(0.5, 0.25, 0.5), matrix(c(1,
    2), 1), c(0, 2))
#> $estimate
#> [1] 0.6931472
#> 
#> $per_position
#> [1] 0.6931472 0.6931472
#> 
#> $positions_scored
#> [1] 0 2
#> 
#> $n_image_regions
#> [1] 1
#> 
#> $n
#> [1] 3
#> 
#> $method
#> [1] "SimVLM MLM loss with visual conditioning (Kamath Eq 9.10)"
#>