Jensen-Shannon divergence

Symmetric, always finite (unlike Kullback-Leibler), bounded above by log 2, and equal to log 2 exactly when the two distributions have disjoint support – all four checked in the tests. Its square root is a metric (Endres and Schindelin 2003). The divergence is Lin's (1991), verified against the DOI; the label "Jensen-Zhang (1986)" sometimes attached to it corresponds to no traceable paper.

Usage

Jzdiff(y, p, q)

Arguments

y

Support labels; only the length is used.

p

Non-negative weights, normalised internally.

q

Non-negative weights of the same length.

Value

List with estimate, divergence, distance, entropy_p, entropy_q, entropy_m, n, method.

Details

Formula: JS(P, Q) = H(M) - (H(P) + H(Q))/2 with M = (P + Q)/2.

References

Lin (1991), Divergence measures based on the Shannon entropy, IEEE Transactions on Information Theory 37(1):145-151, doi:10.1109/18.61115 ; Endres and Schindelin (2003), IEEE Transactions on Information Theory 49(7):1858-1860. doi:10.1109/TIT.2003.813506

Examples

Jzdiff(y = 5L, p = 0.5, q = 0.5)
#> $estimate
#> [1] 0
#> 
#> $divergence
#> [1] 0
#> 
#> $distance
#> [1] 0
#> 
#> $entropy_p
#> [1] 0
#> 
#> $entropy_q
#> [1] 0
#> 
#> $entropy_m
#> [1] 0
#> 
#> $n
#> [1] 1
#> 
#> $method
#> [1] "JS = H(M) - (H(P) + H(Q))/2 with M = (P+Q)/2, Lin (1991) eq. (3.6)"
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"