Prefix-tuning objective (Kamath Eq 3.12)

A SUM of log p_phi(z_i | h_<i) over the target index set. Y_idx is 0-BASED, as in Python.

Usage

morie_kamath_ch3_prefix_tuning_obj(phi, x, y, h, Y_idx = NULL)

Arguments

phi

Function (z_i, h_prefix) -> probability.

x

Source.

y

Target tokens.

h

Per-position activations.

Y_idx

Optional 0-based positions.

Value

List with estimate, per_position, positions_scored.

Examples

morie_kamath_ch3_prefix_tuning_obj(function(z, hp) 0.5, "s:",
    list("a", "b"), list(0, 1))
#> $estimate
#> [1] -1.386294
#> 
#> $per_position
#> [1] -0.6931472 -0.6931472
#> 
#> $positions_scored
#> [1] 0 1
#> 
#> $prompt
#> [1] "s:"
#> 
#> $n
#> [1] 2
#> 
#> $method
#> [1] "prefix-tuning objective (Kamath Eq 3.12)"
#>