Surprisal (self-information) of one or more outcomes

$$I(x) = -\log p(x)$$ Shannon (1948), Section 6; Cover & Thomas (2006), p. 14, where the entropy is the expectation of -log p(X).

Usage

Surpris(p, x, base = 2)

Arguments

p: Non-negative pmf; closed to unit sum here.
x: Outcome index or indices, 0-based to match the Python arm.
base: Log base; 2 gives bits.

Value

Named list with estimate (mean surprisal), values, entropy, n, method.

References

Shannon (1948), Section 6.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
M <- matrix(c(1, 2, 3, 4, 5, 6), nrow = 2)
Surpris(V, M)
#> $estimate
#> [1] 3.120057
#> 
#> $values
#> [1] 4.169925 3.584963 3.169925 2.847997 2.584963 2.362570
#> 
#> $entropy
#> [1] 2.794209
#> 
#> $n
#> [1] 6
#> 
#> $method
#> [1] "Surprisal -log p(x) -- Shannon (1948) Sec. 6"
#>