Autoregressive response losses (Kamath Eq 9.14, 9.17)

Usage

morie_kamath_ch9_mmllm_autoregressive(R, I, theta = NULL)

morie_kamath_ch9_itg_loss(x, y)

Arguments

R: Response token probabilities.
I: Visual context.
theta: Optional model.
x: Visual contexts, or NULL.
y: List of response probabilities.

Value

List with estimate, mean_nll, per_token.

Details

One per-sequence NLL serves both Python modules exactly as km142 calls km145: km145 -> morie_kamath_ch9_mmllm_autoregressive (-sum log p over one response), km142 -> morie_kamath_ch9_itg_loss (that sum over the (X, Y) corpus).

Examples

morie_kamath_ch9_mmllm_autoregressive(c(0.5, 0.25), NULL)
#> $estimate
#> [1] 2.079442
#> 
#> $mean_nll
#> [1] 1.039721
#> 
#> $per_token
#> [1] 0.6931472 1.3862944
#> 
#> $sequence_probability
#> [1] 0.125
#> 
#> $n
#> [1] 2
#> 
#> $method
#> [1] "MMLLM autoregressive response loss (Kamath Eq 9.17)"
#>