Bradley-Terry reward-model NLL over preference pairs (Kamath Ch 5) — morie_kamath_reward_model_training

mean log(1 + exp(-(r_w - r_l))), with the pair accuracy reported.

Usage

morie_kamath_reward_model_training_loss(scores_w, scores_l)

Arguments

scores_w, scores_l: Chosen and rejected scores.

Value

List with estimate, per_pair, margins, accuracy.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
morie_kamath_reward_model_training_loss(V, V)
#> $estimate
#> [1] 0.6931472
#> 
#> $loss
#> [1] 0.6931472
#> 
#> $per_pair
#> [1] 0.6931472 0.6931472 0.6931472 0.6931472 0.6931472 0.6931472 0.6931472
#> [8] 0.6931472
#> 
#> $margins
#> [1] 0 0 0 0 0 0 0 0
#> 
#> $mean_margin
#> [1] 0
#> 
#> $accuracy
#> [1] 0
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "Bradley-Terry reward-model NLL over preference pairs"
#>