Temperature-scaled softmax (Kamath Ch 8)

softmax(z / T), max-shifted; T <= 0 is greedy decoding and is refused.

Usage

morie_kamath_temperature_sampling(logits, T)

Arguments

logits

Logits.

T

Temperature.

Value

List with probabilities, entropy, max_entropy, argmax.

Examples

morie_kamath_temperature_sampling(logits = c(1, 2, 3, 4, 5, 6, 7, 8), T = 5L)
#> $probabilities
#> [1] 0.05600833 0.06840873 0.08355462 0.10205384 0.12464884 0.15224644 0.18595422
#> [8] 0.22712499
#> 
#> $estimate
#> [1] 0.227125
#> 
#> $entropy
#> [1] 1.980833
#> 
#> $max_entropy
#> [1] 2.079442
#> 
#> $argmax
#> [1] 7
#> 
#> $temperature
#> [1] 5
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "Temperature-scaled softmax"
#>