
Equalising log-probability-ratio regulariser (Kamath Eq 6.22)
Source:R/kamath_llm2_native.R
morie_kamath_ch6_log_prob_ratio_attr.Rdlam * MEAN log P(a_i)/P(a_j) over the K word pairs; the unweighted SUM is reported alongside.
Examples
morie_kamath_ch6_log_prob_ratio_attr(c(0.6, 0.4), c(0.4, 0.6),
lam = 2)
#> $estimate
#> [1] -5.551115e-17
#>
#> $unweighted
#> [1] -5.551115e-17
#>
#> $per_pair
#> [1] 0.4054651 -0.4054651
#>
#> $lam
#> [1] 2
#>
#> $K
#> [1] 2
#>
#> $n
#> [1] 2
#>
#> $method
#> [1] "equalising log-probability-ratio regulariser (Eq 6.22)"
#>