Flamingo factorized likelihood and dataset mix (Kamath Eq 9.21-9.22)

Usage

morie_kamath_ch9_flamingo_factorized(y, x = NULL, L = NULL, model = NULL)

morie_kamath_ch9_flamingo_dataset_mix(D_m, lambda_m, x = NULL, y = NULL)

Arguments

y: Per-token conditionals.
x: Context.
L: Optional length.
model: Optional model (y, x).
D_m: List of datasets (each a list of probability vectors).
lambda_m: Dataset weights.

Value

List with estimate (the product), log_prob, nll.

Details

One per-sequence NLL serves both Python modules exactly as km150 calls km149: km149 -> morie_kamath_ch9_flamingo_factorized (product of per-token conditionals, with its log), km150 -> morie_kamath_ch9_flamingo_dataset_mix (sum_m lambda_m times the mean per-sequence NLL of dataset m).

Examples

morie_kamath_ch9_flamingo_factorized(c(0.5, 0.25))
#> $estimate
#> [1] 0.125
#> 
#> $log_prob
#> [1] -2.079442
#> 
#> $nll
#> [1] 2.079442
#> 
#> $per_token
#> [1] 0.50 0.25
#> 
#> $n
#> [1] 2
#> 
#> $method
#> [1] "Flamingo factorized text likelihood (Kamath Eq 9.21)"
#>