Formula: PPL = exp(-(1/N) sum_i log p(x_i)); cross-entropy H = -(1/N) sum_i log2 p(x_i)
Usage
Perplex(log_probs, N = NULL)
Arguments
- log_probs
See Usage.
- N
See Usage.
Value
List with perplexity, cross_entropy_nats, cross_entropy_bits, N.
References
Brown, Della Pietra, Mercer, Della Pietra and Lai (1992), An estimate of
an upper bound for the entropy of English, Computational Linguistics 18:31-40. Not
held locally; perplexity as the exponentiated per-token cross-entropy is the standard
published definition.
Examples
Perplex(log(c(0.25, 0.25, 0.25, 0.25)))$perplexity # uniform over 4 tokens: 4
#> [1] 4
