Can two groups' true base rates be ordered under label noise?

Applies the sharp intervals of morie_fairness_base_rate_bounds to two recorded rates. When the intervals are disjoint the order of the true rates holds for every admissible noise pair (Research.P5.compare_decided); when they overlap there are admissible noise pairs that put either group first, so the data cannot settle the order without narrower noise boxes (Research.P5.compare_undecided). The function also reports the largest beta_max (holding alpha_max) at which the order would become decidable, a breakdown value.

Usage

morie_fairness_compare_groups(p_obs_a, p_obs_b, alpha_max, beta_max)

Arguments

p_obs_a, p_obs_b

Recorded base rates of the two groups.

alpha_max

Largest credible false-positive noise rate.

beta_max

Largest credible false-negative noise rate; alpha_max + beta_max must be below 1.

Value

A list with decided (logical), order ("a < b", "b < a" or "undecided"), the two intervals, and breakdown_beta_max (the largest beta_max that would make the comparison decidable at the same alpha_max, NA if none).

Examples

morie_fairness_compare_groups(0.35, 0.55, alpha_max = 0.05, beta_max = 0.20)
#> $decided
#> [1] TRUE
#> 
#> $order
#> [1] "a < b"
#> 
#> $interval_a
#>     lower     upper 
#> 0.3157895 0.4375000 
#> 
#> $interval_b
#>     lower     upper 
#> 0.5263158 0.6875000 
#> 
#> $breakdown_beta_max
#> [1] 0.335
#> 
#> $theorem
#> [1] "Research.P5.compare_decided"
#> 
morie_fairness_compare_groups(0.35, 0.55, alpha_max = 0.05, beta_max = 0.40)
#> $decided
#> [1] FALSE
#> 
#> $order
#> [1] "undecided"
#> 
#> $interval_a
#>     lower     upper 
#> 0.3157895 0.5833333 
#> 
#> $interval_b
#>     lower     upper 
#> 0.5263158 0.9166667 
#> 
#> $breakdown_beta_max
#> [1] 0.335
#> 
#> $theorem
#> [1] "Research.P5.compare_undecided"
#>