Kozachenko-Leonenko k-nearest-neighbour differential entropy

$$\hat H = -\psi(k) + \psi(n) + \log c_d + (d/n) \sum \log \epsilon_i$$ with \(\epsilon_i\) the distance to the k-th nearest neighbour.

Usage

morie_knn_entropy(x, k = 3L)

Arguments

x

Numeric vector or matrix, one row per observation.

k

Neighbour rank.

Value

A list with entropy, k, dimension, distance_concentration, gaussian_reference.

Details

The \(-\psi(k)\) term is what makes this an estimator rather than a plug-in: the count in a fixed-radius ball is Poisson and \(E[\log N] \ne \log E[N]\), so using \(\log k\) leaves a bias that does not vanish with n.

Differential entropy is not scale invariant – rescaling by \(a\) adds \(d \log a\) – so it is not comparable across units and may be negative.

References

Kozachenko LF, Leonenko NN (1987). Kraskov A, Stogbauer H, Grassberger P (2004) Phys Rev E 69:066138.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
morie_knn_entropy(V)
#> $entropy
#> [1] 2.580518
#> 
#> $k
#> [1] 3
#> 
#> $dimension
#> [1] 1
#> 
#> $neighbour_distances
#> 1 2 3 4 5 6 7 8 
#> 3 2 2 2 2 2 2 3 
#> 
#> $distance_concentration
#> [1] 0.2057378
#> 
#> $gaussian_reference
#> [1] 2.314818
#> 
#> $negentropy
#> [1] -0.2656995
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "Kozachenko-Leonenko k-NN differential entropy"
#>