Verbalizer class probabilities (Kamath Ch 3)

Sums the softmax mass of each class's answer tokens; the leftover mass is reported rather than hidden, and a token may verbalise only one class.

Usage

morie_kamath_verbalizer_mapping(logits, vocab, verbalizer_map)

Arguments

logits

Vocabulary logits.

vocab

Token vector.

verbalizer_map

Named list class -> answer tokens.

Value

List with probabilities, normalized, prediction.

Examples

morie_kamath_verbalizer_mapping(c(1, 0, 2), c("gr", "te", "ok"),
    list(pos = c("gr", "ok"), neg = c("te")))
#> $probabilities
#>        pos        neg 
#> 0.90996943 0.09003057 
#> 
#> $normalized
#>        pos        neg 
#> 0.90996943 0.09003057 
#> 
#> $prediction
#> [1] "pos"
#> 
#> $mass_on_labels
#> [1] 1
#> 
#> $mass_outside
#> [1] 1.110223e-16
#> 
#> $answer_tokens
#> $answer_tokens$pos
#> [1] "gr" "ok"
#> 
#> $answer_tokens$neg
#> [1] "te"
#> 
#> 
#> $estimate
#> [1] 0.9099694
#> 
#> $n
#> [1] 2
#> 
#> $method
#> [1] "Verbalizer class probability = summed answer-token mass"
#>