sum_s s * softmax(judge logits)_s over the rubric.
Usage
morie_kamath_g_eval(x, y, rubric, model)
Arguments
- x, y
Source and candidate.
- rubric
Score points or a list
carrying scores.
- model
Judge returning one logit per point.
Value
List with estimate, probabilities, score_points.
Examples
morie_kamath_g_eval("x", "y", c(1, 2, 3), function(x, y, r) c(0,
1, 0))
#> $estimate
#> [1] 2
#>
#> $probabilities
#> [1] 0.2119416 0.5761169 0.2119416
#>
#> $score_points
#> [1] 1 2 3
#>
#> $logits
#> [1] 0 1 0
#>
#> $n
#> [1] 3
#>
#> $method
#> [1] "G-Eval probability-weighted rubric score"
#>
