Rank statistics for simulation-based calibration, and a uniformity test

The SHAPE of the departure names the fault: U means the posterior is too narrow, a hump too wide, a slope biased. The bin count must divide L + 1 exactly or the expected counts are unequal.

Usage

Sbcrank(prior_draw, post_draws, bins = NULL)

Arguments

prior_draw

One prior draw per replication.

post_draws

Matrix of posterior draws, one row per replication.

bins

Number of histogram bins; must divide L + 1.

Value

List with rank, histogram, expected, statistic, p_value, df, bins, J, L.

Details

Formula: rank_j = #{ l : theta^{(l)}_j < theta_j^prior }; under calibration rank ~ Uniform{0, ..., L}; chi^2 = sum_b (O_b - E_b)^2 / E_b on (bins - 1) df

References

Talts, Betancourt, Simpson, Vehtari & Gelman (2018), arXiv:1804.06788 – the primary source. Bayesian Data Analysis, 3rd edition, predates simulation-based calibration and does not cover it.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
Sbcrank(V, V)
#> $rank
#> [1] 0 0 0 0 0 0 0 0
#> 
#> $histogram
#> [1] 8 0
#> 
#> $expected
#> [1] 4
#> 
#> $statistic
#> [1] 8
#> 
#> $p_value
#> [1] 0.004677735
#> 
#> $df
#> [1] 1
#> 
#> $bins
#> [1] 2
#> 
#> $J
#> [1] 8
#> 
#> $L
#> [1] 1
#> 
#> $method
#> [1] "Simulation-based calibration ranks (Talts et al. 2018)"
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"