Research: the hardest problems, with proofs¶
morie.research carries the research programme on the hardest open problems
in criminology and sociolegal studies. Every function rests on a theorem checked
in Lean 4 with Mathlib (research/lean in the rmorie repository: 0 sorry,
axioms propext, Classical.choice and Quot.sound only), names the
theorems it uses in its theorems field, and says where the empirical
assumption enters. Lean certifies the implication, never the antecedent: no proof
shows that two lists are independent, that a noise box is right, or that
spillover stops at the ring the analyst named.
The R package (rmorie) carries the same functions with the morie_ prefix.
The two arms agree to rounding on the same inputs, and the stochastic ones draw
from the shared Philox stream, so a seed gives the same path in both.
from morie import research as R
R.dark_figure_two_source(400, 250, 80, kappa=2) # Lincoln-Petersen with a dependence box
R.feedback_loop_limit(0.3, 0.2, 10, 10) # the proved limit of the patrol share
R.meta_random_effects([0.20, 0.35, 0.10], [0.010, 0.020, 0.015]) # DerSimonian-Laird, with the truncation flag
Problems and functions¶
Problem |
Theorems ( |
Functions |
|---|---|---|
P1 dark figure |
|
|
P1 Le Cam two-point bound |
|
|
P2 selection in police records |
|
|
P3 interference |
|
|
P4 predictive-policing feedback |
|
|
P5 risk scores under label bias |
|
|
P6 age-crime curve |
|
|
P7 crime concentration |
|
|
P8 deterrence |
|
|
P9 recording as a linear map |
|
|
P10 near-repeat contagion |
|
|
P11 sentencing effects as intervals |
|
|
P12 ecological inference |
|
|
P13 pooling evaluations |
|
|
P13 HKSJ interval |
|
|
P14 judge leniency |
|
|
P14 many-judge slope test |
|
|
P15 disparity decomposition |
|
|
P15 DFL reweighting |
|
|
P16 court backlog |
|
|
P16 disposed-cases mean as a bound |
|
|
P17 incapacitation |
|
|
P17 desistance and replacement |
|
|
P18 selective labels |
|
|
P19 regression to the mean |
|
|
P19 empirical-Bayes shrinkage |
|
|
Reference¶
The research programme: methods for the hardest problems in criminology, each resting on a Lean 4 theorem.
Every function names the theorems in research/lean (Lean 4 + Mathlib, 0 sorry,
standard axioms only) that cover its arithmetic, and says where the empirical
assumption enters. Lean certifies the implication, never the antecedent. The R
package carries the same functions with the morie_ prefix; both arms agree
to rounding on the same inputs, and the stochastic ones share the Philox stream.
- morie.research.nb_matches_zip(pi, lam)[source]¶
A negative binomial with the zero share and mean of any zero-inflated Poisson.
Lean
Research.P7.ZeroInflation.nb_matches_zip.>>> r = nb_matches_zip(0.3, 2) >>> abs(r["nb_zero"] - r["zip_zero"]) < 1e-9 True
- morie.research.gamma_bound(T, n, Gamma=None)[source]¶
A Rosenbaum Gamma bound is not a test of ignorability (Lean
Research.P5.Gamma.gamma_star_kills).>>> gamma_bound(70, 100)["gamma_star"] 2.3333333333333335
- morie.research.blocking_crossover(delta, sigma, n, alpha=0.05, rho=None)[source]¶
The within-pair correlation crossover for blocking (Lean
Research.P3.Blocking.paired_better_iff).>>> blocking_crossover(0.5, 1, 10)["crossover"] > 0 True
- morie.research.adjustment_variances(b1, b0, v1, v0, p)[source]¶
Freedman/Lin asymptotic variances (Lean
Research.P3.Adjustment.adjusted_minus_unadjusted).>>> adjustment_variances(1, 0, 1, 1, 0.8)["adjustment_hurts"] True
- morie.research.budget_optimum(g, d, C, grid=201)[source]¶
Stick versus carrot: the interior optimum on a grid (Lean
Research.P8.Budget.interior_optimum).>>> budget_optimum(math.sqrt, lambda s: 2 * math.log1p(s), 10)["interior"] True
- morie.research.outcome_test(groups)[source]¶
Stop and hit rates of threshold groups (Lean
Research.P2.OutcomeTest.infra_marginality).>>> r = outcome_test({"A": {"q": [0.3, 0.9], "w": [1, 1], "t": 0.5}, "B": {"q": [0.9], "w": [1], "t": 0.2}}) >>> r["A"]["hit_rate"] == r["B"]["hit_rate"] True
- morie.research.spatial_omission(rho, x, Sigma, alpha=0.05, m=1)[source]¶
Lag endogeneity, GLS over OLS, and the family-wise error (Lean
Research.P3.Spatial.lag_regressor_correlated).>>> r = spatial_omission(0.4, [1, 2], [[1, 0.5], [0.5, 1]]) >>> (round(r["var_ols"], 6), round(r["var_gls"], 6)) (0.28, 0.25)
- morie.research.iv_natural_bounds(p11, p10, m1, m0)[source]¶
IV natural bounds (Lean
Research.P8.NaturalBounds.width).>>> round(iv_natural_bounds(0.8, 0.3, 0.6, 0.4)["width"], 12) 0.5
- morie.research.design_sensitivity(mu, mu_gamma=None, upper=10000.0)[source]¶
Design sensitivity as the root of mu_Gamma = mu (Lean
Research.P5.DesignSensitivity.design_sensitivity_exists_unique).>>> round(design_sensitivity(0.7)["gamma_tilde"], 6) 2.333333
- morie.research.moran_trend(W, y, X=None)[source]¶
Moran’s I under a trend and on residuals (Lean
Research.P7.MoranTrend.residual_moran_ratio).>>> W = [[0, 1, 0, 0], [1, 0, 1, 0], [0, 1, 0, 1], [0, 0, 1, 0]] >>> round(moran_trend(W, [1, 2, 3, 4])["I"], 6) 0.333333
- Return type:
- morie.research.exposure_positivity(p, degrees)[source]¶
Exposure positivity under Bernoulli assignment (Lean
Research.P3.Exposure.exposure_prob_tendsto_zero).>>> exposure_positivity(0.5, [1, 2])["ht_weight"] [2.0, 4.0]
- morie.research.permutation_valid(stats, alpha=(0.01, 0.05, 0.1))[source]¶
Exactness of the permutation p-value (Lean
Research.P3.Exposure.permutation_valid).>>> permutation_valid([3, 1, 4, 1, 5], [0.5])["levels"]["valid"] [True]
- Return type:
- morie.research.thinning_identification(lam, gamma, s=(0.0, 0.5, 0.9, 1.0))[source]¶
Thinning a Poisson count (Lean
Research.P1.Thinning.not_identified).>>> thinning_identification(5, 0.3)["recorded_rate"] 1.5
- morie.research.poisson_chernoff(lam, eps)[source]¶
Chernoff tail bound for a Poisson count (Lean
Research.P7.Chernoff.poisson_chernoff).>>> r = poisson_chernoff(20, [0.5]) >>> r["exact"][0] <= r["bound"][0] True
- morie.research.random_walk_exposure(adj)[source]¶
Random-walk exposure is proportional to degree (Lean
Research.P3.RandomWalk.stationary).>>> random_walk_exposure([[0, 1, 1], [1, 0, 1], [1, 1, 0]])["stationary"] True
- Return type:
- morie.research.extremal_index_clusters(m, tau, n=(10, 100, 1000))[source]¶
Extremal index of a block-cluster process (Lean
Research.P10.ExtremalIndex.extremal_index).>>> extremal_index_clusters(4, 2)["theta"] 0.25
- morie.research.tail_ratio(k, q=0.5, alpha=1.0, r=2, counts=None)[source]¶
Tail ratios separating light from power-law tails (Lean
Research.P7.TailShape.nb_pmf_ratio_tendsto_zero).>>> tail_ratio([5], q=0.7, alpha=1.5, r=3)["nb_pmf_ratio"][0] <= tail_ratio([5], q=0.7, alpha=1.5, r=3)["nb_bound"][0] True
- morie.research.calibration_flip(fA, fB, r)[source]¶
Calibration to the record and to the truth cannot both hold (Lean
Research.P5.DataProcessing.calibration_incompatible).>>> calibration_flip(0.05, 0.2, [0.3, 0.5])["incompatible"] [True, False]
- morie.research.bayes_risk_dpi(mu, w, h)[source]¶
Post-processing cannot lower the best squared-error risk (Lean
Research.P5.DataProcessing.bayes_risk_data_processing).>>> bayes_risk_dpi([[0.3, 0.1, 0.05], [0.1, 0.2, 0.25]], [0, 1], [1, 1, 2])["dpi_holds"] True
- Return type:
- morie.research.bellman_value(r, p, gamma, iters=1000, tol=1e-10)[source]¶
Value iteration with the contraction audited (Lean
Research.P4.Bellman.bellman_contracting).>>> r = [[1, 0], [0.5, 0.5], [1, 1]] >>> p = [[[0, 1, 0], [0, 0, 1]], [[0, 1, 0], [0, 1, 0]], [[0, 0, 1], [0, 0, 1]]] >>> bellman_value(r, p, 0.9)["policy"][0] 1
- morie.research.myopic_example(gamma)[source]¶
Greedy versus patient allocation (Lean
Research.P4.Bellman.myopic_suboptimal).>>> myopic_example([0.5, 0.9])["patient_better"] [False, True]
- Return type:
- morie.research.budget_rule(m, tau, kappa)[source]¶
The budget-constrained threshold rule (Lean
Research.P5.BudgetRule.threshold_optimal).>>> budget_rule([1, 1, 1], [0.5, 0.2, -0.1], 1.5)["rule"] [1.0, 0.5, 0.0]
- morie.research.run_length_bound(alpha, J=None)[source]¶
Mean run length of a restarted detector (Lean
Research.P10.Cusum.mean_run_length_ge).>>> run_length_bound(0.01)["bound"] 50.0
- morie.research.attenuation_two_regressor(beta1, beta2, s11, s22, s12=0.0, su2=0.0)[source]¶
Measurement error attenuates only in the one-regressor case.
The one-regressor slope on the observed regressor is
beta1 * s11 / (s11 + su2); with an error-free second regressor the population coefficients areb1 = beta1 (s11 s22 - s12^2) / D*andb2 = beta2 + beta1 su2 s12 / D*withD* = (s11 + su2) s22 - s12^2(LeanResearch.P5.Attenuation.two_regressor_coeffs): the error-free coefficient is biased unlessbeta1,su2ors12is zero, and the bias has the sign ofbeta1 * s12.>>> r = attenuation_two_regressor(1, 0.5, 1, 1, 0.6, 0.5) >>> round(r["b2"] - 0.5, 6) 0.263158
- morie.research.friedman_urn(b0, r0, n=100)[source]¶
The Friedman urn: cross-colour reinforcement pulls the share to one half.
The expected difference after
nsteps isD0 (T0 - 1) / (T0 + n - 1), which solvesE[D_{n+1}] = E[D_n] (1 - 1/T_n)(LeanResearch.P4.Friedman.meanDiff_succ) and tends to zero; the mean share1/2 + E[D_n] / (2 T_n)tends to one half.>>> friedman_urn(9, 1, 3)["mean_diff"] [8.0, 7.2, 6.545454545454546, 6.0]
- morie.research.ruin_probability(p, N, k=None)[source]¶
Absorption probability of a finite-capacity allocation.
With
rho = (1 - p) / pthe probability of ending at zero fromkis(rho^k - rho^N) / (1 - rho^N), the solution ofp_k = p p_{k+1} + (1 - p) p_{k-1}withp_0 = 1,p_N = 0(LeanResearch.P4.Ruin.ruin_harmonic); atp = 1/2it is1 - k/N.>>> ruin_probability(0.5, 4)["prob"] [1.0, 0.75, 0.5, 0.25, 0.0]
- morie.research.cox_overdispersion(w, lam)[source]¶
Any intensity heterogeneity forces overdispersion.
A count whose intensity is a finite mixture has
Var N = E N + Var Lambda(LeanResearch.P7.Cox.total_variance), so the variance is at least the mean, with equality iff the intensity is constant on the support.>>> r = cox_overdispersion([0.7, 0.3], [1, 5]) >>> round(r["variance"] - r["mean"], 6) == round(r["intensity_var"], 6) True
- Return type:
- morie.research.zip_vuong_boundary(counts, lam=None)[source]¶
Vuong’s test of zero-inflated against plain Poisson at the boundary.
At
pi = 0the two pmfs agree at every count (LeanResearch.P7.Cox.zip_zero_eq_poisson), so every pointwise log-likelihood ratio is zero (llr_zero), their standard deviation is zero and the statistic is 0/0.>>> zip_vuong_boundary([0, 0, 1, 2])["sd"] 0.0
- morie.research.nonresponse_region(p_obs, r)[source]¶
The identification region of a proportion under nonresponse.
With reporting rate
rand observed sharep_obsamong reporters the proportion lies in[r p_obs, r p_obs + 1 - r], both ends attained, of width exactly1 - r(LeanResearch.P1.Nonresponse.region_width).>>> nonresponse_region(0.4, [0.106, 0.5])["width"] [0.894, 0.5]
- Return type:
- morie.research.instrumental_inequality(p, tol=1e-08)[source]¶
Pearl’s instrumental inequality on a finite table.
p[y][x][z]isP(y, x | z)with every z-slice summing to one; the model impliessum_y max_z P(y, x | z) <= 1for everyx(LeanResearch.P8.IVInequality.instrumental_inequality).>>> p = [[[0.4, 0.2], [0.3, 0.1]], [[0.1, 0.3], [0.2, 0.4]]] >>> instrumental_inequality(p)["passes"] True
- morie.research.misclassified_rd(r1, r0, prevalence, sensitivity, specificity)[source]¶
Nondifferential misclassification of a binary exposure.
The measured risk difference is
(se + sp - 1)(r1 - r0) pi (1 - pi) / (d1 d0)withd1, d0the measured group shares (LeanResearch.P2.Misclassification.binary_attenuation), so it keeps the sign of the true difference wheneverse + sp >= 1.>>> round(misclassified_rd(0.3, 0.1, 0.4, 0.8, 0.9)["factor"], 6) 0.713073
- morie.research.misclassified_contrast(risks, group, weights=None, high='high', low='low')[source]¶
Misclassification of an ordinal exposure can reverse a contrast.
Levels are mapped to measured groups; the measured contrast of weighted mean risks can be zero or of the opposite sign to the true top-versus-bottom contrast when the map is not monotone (Lean
Research.P2.Misclassification.ordinal_witness,ordinal_reversal).>>> misclassified_contrast([0, 0.5, 1], ["high", "low", "high"])["measured_contrast"] 0.0
- morie.research.sensitivity_amplification(Lambda=None, Delta=None, Gamma=None)[source]¶
Amplification of a sensitivity parameter: one Gamma is a curve of (Lambda, Delta).
Gamma = (Lambda Delta + 1) / (Lambda + Delta); everyGamma >= 1is reached alongDelta = (Lambda Gamma - 1) / (Lambda - Gamma)forLambda > Gamma(LeanResearch.P11.Amplification.level_curve). Give any two of the three.>>> sensitivity_amplification(Gamma=2, Lambda=[3, 4])["Delta"] [5.0, 3.5]
- Return type:
- morie.research.design_effect_direction(rho, M, x=None)[source]¶
The design effect of clustering is direction-specific.
The quadratic form of a regressor is
(1 - rho) sum x^2 + rho sum_c (sum_j x_cj)^2(LeanResearch.P13.Design.quad_form): a cluster-level regressor inflates the naive variance by1 + (M - 1) rho, a within-cluster-balanced one deflates it by1 - rho, andrho >= -1/(M - 1).>>> design_effect_direction(0.1, 20)["deff_cluster_level"] 2.9000000000000004
- morie.research.ht_elephant(t, pi, bound=None)[source]¶
Basu’s elephant: the Horvitz-Thompson total is unbiased with exploding variance.
Mean
sum t_iand variancesum t_i^2 (1 - pi_i) / pi_iunder independent inclusion (LeanResearch.P3.Basu.ht_variance); the largest unit alone exceeds any boundBatpi_e = t_e^2 / (2 (B + t_e^2))(elephant_unbounded).>>> ht_elephant([1, 2, 3, 100], [0.5, 0.5, 0.5, 0.01])["mean"] 106.0
- morie.research.attractiveness_contraction(S, omega, eta, B, n=10)[source]¶
The discrete attractiveness field contracts absent crime.
B' = (1 - omega)[(1 - eta) B + eta S B]withSrow-stochastic contracts the sup norm by1 - omegaper step (LeanResearch.P4.Contraction.decay); withSdoubly stochastic the total is scaled by exactly1 - omega; the uniform feedback mode grows iffeps rho > omega / (1 - omega).>>> S = [[0.5, 0.5, 0], [0.5, 0, 0.5], [0, 0.5, 0.5]] >>> attractiveness_contraction(S, 0.2, 0.3, [5, 1, 0], 2)["total"] [6.0, 4.800000000000001, 3.84]
- morie.research.inspection_game(a, x, steps=1000, dt=0.01)[source]¶
The criminal-citizen-inspector game cycles.
Replicator dynamics for a rock-paper-scissors payoff with stake
ahasx^T A x = 0, the centroid as its only interior rest point andV = x1 x2 x3conserved (LeanResearch.P8.Inspection.conserved): orbits are closed curves around the centroid.>>> inspection_game(1, [1/3, 1/3, 1/3], 5)["field"] [0.0, 0.0, 0.0]
- morie.research.age_crime_panel_cov(y_a, y_b, n_perm=2000, seed=1)[source]¶
Sample cross-age covariance with an exact one-type permutation null (
one_type_cov_zero).
- morie.research.feedback_loop_envelope(r0, b0, a=1.0, n=100, t=0.25)[source]¶
Drift envelope of the discovery urn: variance at most a/(4(r0+b0)) (
Research.P4.Envelope.var_le).Examples
>>> e = feedback_loop_envelope(1, 1, 1, n=40) >>> (e["variance_bound"], round(e["variance_n"], 12), round(e["mean_n"], 12)) (0.125, 0.079365079365, 0.5)
- morie.research.rank_persistence_null(n, k, observed=None)[source]¶
Hypergeometric null for the overlap of two top-k sets of n places (
Research.P7.Persistence.overlap_prob).Examples
>>> r = rank_persistence_null(158, 16, observed=9) >>> (round(r["mean"], 12), r["p_value"] < 1e-4, round(sum(r["probability"]), 12)) (1.620253164557, True, 1.0)
- morie.research.spillover_ring_bound(outer_share, gamma_max, gamma=None)[source]¶
Ring mis-specification bound: |E gap| <= gamma_max * mean(outer_share) (
Research.P3.Ring.expected_gap_le).Examples
>>> r = spillover_ring_bound([0.1, 0.3, 0.2], gamma_max=2) >>> (round(r["mean_share"], 12), round(r["bound"], 12)) (0.2, 0.4)
- morie.research.hotspot_threshold(A0, omega, eps, D, gamma, eta, z=1.0, k=None)[source]¶
Hot-spot threshold of the Short-Brantingham model (
Research.P4.Hotspot).The uniform steady state is B = eps D gamma / omega, A = A0 + B, rho = gamma / A (
steady_iff); the mode-k determinant is P k^2 - Q k + R with P = eta D^2 / z, Q = D (3 eps D rho - omega - eta A / z), R = omega A, and the trace is negative, so a mode grows iff the determinant is negative (unstable_iff). Q <= 0 means no hot spots at any wavelength (no_hotspots_below_threshold); Q > 0 with Q^2 > 4 P R makes the critical mode k* = Q / (2 P) grow (hotspots_above_threshold).Examples
>>> hotspot_threshold(A0=1, omega=1, eps=2, D=1, gamma=1, eta=0.01)["unstable"] True >>> hotspot_threshold(A0=1, omega=1, eps=0, D=1, gamma=1, eta=0.01)["unstable"] False
- morie.research.dark_figure_comparable(VA, RA, RB, lam=1.0)[source]¶
The dark figure from a survey covering only the offence set A (
Research.P1.Comparable).The survey identifies VA and so r_A = RA / VA (
rate_on_A); the overall rate is at most (RA + RB) / (VA + RB) (total_rate_le) and has no positive lower bound without an assumption on B. Under r_B >= lam r_A the total count lies in [VA + RB, VA + RB/(lam r_A)] (total_interval) and the dark figure in [VA - RA, VA - RA + RB (1/(lam r_A) - 1)] (dark_interval), an interval of width RB (1 - lam r_A)/(lam r_A) (dark_figure_width).Examples
>>> r = dark_figure_comparable(VA=1000, RA=400, RB=300, lam=0.5) >>> (r["rate_A"], r["total"], r["dark_figure"], r["width"]) (0.4, (1300.0, 2500.0), (600.0, 1800.0), 1200.0)
- morie.research.age_crime_type_cov(w, rate_a, rate_b)[source]¶
Cross-age covariance of a type mixture (
cross_moment_eq,two_type_cov).Examples
>>> age_crime_type_cov([0.5, 0.5], [1, 3], [3, 1])["cross_cov"] -1.0
- Return type:
- morie.research.concentration_null_bound(x, n_sim=2000, seed=1)[source]¶
The Gini against its coefficient-of-variation bound and a simulated Poisson null (
gini_le_cv).Examples
>>> r = concentration_null_bound([0, 0, 1, 2, 9, 0, 3], n_sim=200) >>> round(r["gini"], 12), round(r["cv_bound"], 12), r["gini"] <= r["cv_bound"] (0.666666666667, 0.988826464946, True)
- morie.research.contagion_moment_witness(statistic=None)[source]¶
Two count laws with equal mean and variance but different zero mass (
second_moments_do_not_identify).Examples
>>> w = contagion_moment_witness(lambda m, v: v / m) >>> w["dispersion"], w["zero_probability"] ((1.0, 1.0), (0.5, 0.3333333333333333))
- Return type:
- morie.research.dark_figure_no_interaction(cells, interaction=None)[source]¶
The never-listed cell under no three-way interaction (
missing_cell_identified).Examples
>>> r = dark_figure_no_interaction({"n100": 120, "n010": 80, "n001": 60, "n110": 30, "n101": 24, "n011": 16, "n111": 8}) >>> round(r["missing"], 12), round(r["total"], 12) (400.0, 738.0)
- morie.research.deterrence_logcost(p, s, c, y=None, rho=None)[source]¶
The expected-cost response on the log design and the discount it implies (
logcost_linear).Examples
>>> p, s, c = [0.2, 0.5, 0.8, 0.4], [2, 5, 3, 8], [1, 4, 2, 6] >>> y = [2 - 0.8 * (math.log(pi) + math.log(si) - 0.1 * ci) for pi, si, ci in zip(p, s, c)] >>> round(deterrence_logcost(p, s, c, y)["implied_discount"], 10) 0.1
- Return type:
- morie.research.rif_gap(yA, yB, tau, wA=None, wB=None)[source]¶
The gap of two groups’ RIF means as quantile gap plus recentering (
rif_gap).
- morie.research.rif_quantile(y, tau, weights=None, q=None, f=None)[source]¶
Recentered influence function of a quantile (
rif_quantile_avg).Examples
>>> r = rif_quantile([1, 2, 3, 4, 5, 6, 7, 8], 0.5, q=4, f=0.125) >>> round(r["mean_rif"], 12), round(r["recentering"], 12) (4.0, 0.0)
- morie.research.shrinkage_factor_error(y, noise_variance, weights=None, total_variance=None)[source]¶
Error of the plug-in shrinkage factor when the total mass is estimated (
bhat_error_bound).Examples
>>> r = shrinkage_factor_error([3, 5, 9, 2, 7, 4], noise_variance=5, total_variance=6.0) >>> round(r["B_hat"], 12), round(r["error"], 12) (0.882352941176, 0.049019607843)
- morie.research.veil_of_darkness(day, night, reference=None, exposure_shift=None)[source]¶
The veil-of-darkness odds ratio as the relative visible-race factor (
odds_ratio_identifies_bias).Examples
>>> r = veil_of_darkness({"white": 400, "black": 200}, {"white": 300, "black": 100}) >>> round(r["bias_ratio"]["black"], 12), r["sign"]["black"] (1.5, 'higher')
- morie.research.wald_iv(y, p, z)[source]¶
The Wald instrumental-variable ratio (
wald_identifies,wald_undefined).Examples
>>> wald_iv([1, 3, 2, 4], [0, 1, 0, 1], [0, 1, 0, 1])["beta"] 2.0
- Return type:
- morie.research.age_crime_aggregate(shares, curves, ages=None)[source]¶
Aggregate age-crime curve of a mixture of latent types (
Research.P6.aggregate_not_identifying).Examples
>>> a = age_crime_aggregate([0.5, 0.5], [[1, 3], [2, 2], [3, 1]], ages=[10, 11, 12]) >>> list(a["aggregate"]) [2.0, 2.0, 2.0]
- morie.research.backlog_censoring(disposed, pending_ages)[source]¶
The disposed-cases mean as a bound: what the pending cases imply.
Research.P16Censoring.true_mean_ge/lower_bound_sub/bias_lower/disposed_understates/no_upper_bound.Examples
>>> b = backlog_censoring([30, 45, 60, 90, 120], [100, 150, 200]) >>> (b["disposed_mean"], b["lower_bound"], b["bias_lower"], b["understates"], b["upper_bound"]) (69.0, 99.375, 30.375, True, inf)
- Return type:
- morie.research.backlog_published(received, days, pending, mean_days, band_shares=None, band_floors=None)[source]¶
Published court statistics read through Little’s law (
Research.P16.little_target,Research.P16Censoring.disposed_understates,band_floor_le_mean).R parity:
morie_backlog_published.Examples
>>> r = backlog_published(21265, 30, 148904, 175, band_shares=[[0.808, 0.105, 0.034, 0.053]], ... band_floors=[0, 304, 456, 548])[0] >>> (round(r["implied_mean_days"], 4), round(r["ratio"], 6), round(r["band_floor_bound"], 3)) (210.0691, 1.200395, 76.468)
- Return type:
- morie.research.bounds_confidence(lower, upper, se_lower, se_upper, level=0.95)[source]¶
Imbens-Manski confidence interval for a partially identified parameter.
The cutoff
csolvesPhi(c + Delta) - Phi(-c) = 1 - alphawithDelta = (upper - lower)/max(se)it is non-increasing in Delta (Research.P11.im_cutoff_antitone), lies between the one- and two-sided quantiles (im_cutoff_between), and the two-sided quantile over-covers any region of positive width (two_sided_overcovers).Examples
>>> r = bounds_confidence(lower=0.10, upper=0.35, se_lower=0.03, se_upper=0.04) >>> round(r["cutoff"], 6) 1.644854
- Return type:
- morie.research.cheeger_bound(adjacency, S, exhaustive=False)[source]¶
Conductance of a set and the Cheeger bound
lambda_2 <= E(f)/D(f) <= 2 h(S).Examples
>>> A = [[0, 1, 1, 0, 0, 0], [1, 0, 1, 0, 0, 0], [1, 1, 0, 1, 0, 0], ... [0, 0, 1, 0, 1, 1], [0, 0, 0, 1, 0, 1], [0, 0, 0, 1, 1, 0]] >>> r = cheeger_bound(A, S=[0, 1, 2], exhaustive=True) >>> (round(r["conductance"], 12), round(r["rayleigh_test"], 12), r["bound_holds"], r["argmin_set"]) (0.142857142857, 0.285714285714, True, [0, 1, 2])
- Return type:
- morie.research.collider_arrest(a, e, pi_bg)[source]¶
Collider bias from conditioning on arrest (
collider_or_eq_background,collider_or_lt_one).Examples
>>> r = collider_arrest(a=0.3, e=0.2, pi_bg=0.1) >>> (round(r["arrestee_or"], 12), r["population_or"]) (0.1, 1)
- Return type:
- morie.research.concentration_decompose(x)[source]¶
Decompose crime concentration into crime-free places and the rest (
gini_zero_decomposition).Examples
>>> r = concentration_decompose([0, 0, 0, 1, 9]) >>> (r["zero_share"], round(r["gini_positive"], 12), round(r["identity_check"], 15)) (0.6, 0.4, 0.0)
- Return type:
- morie.research.concentration_dispersion(x)[source]¶
Dispersion of place counts against the Poisson null and its mixtures.
Examples
>>> r = concentration_dispersion([0, 0, 1, 3, 0, 2]) >>> (r["mean_count"], round(r["dispersion_index"], 12)) (1.0, 1.6)
- Return type:
- morie.research.concentration_distinct_growth(place)[source]¶
Growth of the number of distinct places under Polya allocation (
expectedDistinct_bounds).Examples
>>> r = concentration_distinct_growth([1, 2, 1, 3, 2, 1, 4, 1]) >>> (r["n"], r["distinct"][-1], round(r["M_hat"], 6)) (8, 4, 2.500624)
- Return type:
- morie.research.concentration_gini(x)[source]¶
Gini coefficient of a count vector (mean-absolute-difference form).
Examples
>>> round(concentration_gini([0, 0, 0, 1, 9]), 12) 0.76
- Return type:
- morie.research.contagion_branching(n, mu=1.0, generations=10)[source]¶
Branching-ratio arithmetic of a self-exciting (Hawkes) crime process.
Examples
>>> r = contagion_branching(n=0.4, mu=2) >>> (round(r["expected_cluster_size"], 12), round(r["stationary_rate"], 12), r["endogeneity_share"]) (1.666666666667, 3.333333333333, 0.4) >>> contagion_branching(n=1.1, mu=2)["expected_cluster_size"] inf
- Return type:
- morie.research.contagion_extinction(p, tol=1e-14, max_iter=100000)[source]¶
Extinction probability of a near-repeat chain: the smallest fixed point of the generating function.
Iterates
s_0 = 0,s_{n+1} = f(s_n)(Research.P10.iter_tendsto,extinction_fixed,extinction_le_fixed); mean offspring below one gives certain extinction (subcritical_extinction_one), above one a survival probability strictly positive (supercritical_extinction_lt_one).Examples
>>> r = contagion_extinction([0.3, 0.3, 0.4]) >>> (round(r["extinction"], 9), r["regime"]) (0.75, 'supercritical') >>> round(contagion_extinction([0.5, 0.3, 0.2])["extinction"], 9) 1.0
- Return type:
- morie.research.contaminated_bounds(q, p)[source]¶
Contaminated-sample bounds for a recorded proportion (
Research.P11.clean_bounds,clean_width,clean_informative).Examples
>>> b = contaminated_bounds([0.05, 0.5, 0.97], 0.1) >>> [round(v, 12) for v in b["lower"]] [0.0, 0.444444444444, 0.966666666667]
- morie.research.court_backlog(arrivals, dispositions, horizon=None, target_backlog=None)[source]¶
Little’s law on a docket: time-average pending cases = filing rate x mean disposition time.
Examples
>>> r = court_backlog([0, 1, 2, 4, 5, 7], [3, 2.5, 6, 5, 9, 10], horizon=10, target_backlog=1) >>> (round(r["average_backlog"], 12), round(r["required_mean_time"], 12)) (1.65, 1.666666666667)
- Return type:
- morie.research.dark_figure_bounds(v_obs, r, alpha_max, beta_max)[source]¶
Sharp bounds on a true victimisation rate and its dark figure.
max(r, (v_obs - a)/(1 - a)) <= v <= v_obs/(1 - b)with the boxesalpha in [0, a],beta in [0, b],a + b < 1(Research.P1.true_rate_bounds,dark_figure_bounds). Returns a frame withv_obs, r, v_lower, v_upper, dark_lower, dark_upper, ratio_upper.Examples
>>> b = dark_figure_bounds(0.06, 0.02, alpha_max=0.01, beta_max=0.30) >>> round(float(b["v_upper"][0]), 10) 0.0857142857
- morie.research.dark_figure_breakdown(v_obs, threshold)[source]¶
Breakdown analysis for a dark-figure conclusion.
“The true rate is below
threshold” survives every admissible noise pair while the under-reporting box stays below1 - v_obs/threshold(Research.P1.conclusion_holds_below_breakdown) and fails at or above it (conclusion_fails_above_breakdown).Examples
>>> round(dark_figure_breakdown(0.06, threshold=0.10)["breakdown_beta_max"], 12) 0.4
- Return type:
- morie.research.dark_figure_hierarchy(offences_per_incident)[source]¶
Incident versus offence counting: the hierarchy-rule arithmetic.
With N incidents carrying k_i >= 1 offences, bounded by K, the offence count lies in [N, KN] and equals N(1 + mean extra) (
Research.P1.offence_count_bounds,offence_count_eq); it is not a function of N (category_not_identified).Examples
>>> r = dark_figure_hierarchy([1, 1, 2, 1, 3, 1, 1, 2]) >>> (r["incidents"], r["offences"], r["mean_extra"]) (8, 12, 0.5)
- Return type:
- morie.research.dark_figure_three_list(counts, candidate_missing=None)[source]¶
Three-list capture-recapture: what is and is not identified.
The saturated model reproduces any positive eight-cell table (
Research.P1.three_list_saturated_fits), so the missing cell is free (missing_cell_unconstrained); setting the three-way interaction to zero givesm000 = m111 m100 m010 m001 / (m110 m101 m011). Pairwise Petersen and Chapman estimates respect the floor (petersen_ge_floor,chapman_ge_floor).Examples
>>> r = dark_figure_three_list({"100": 120, "010": 90, "001": 70, "110": 40, "101": 30, "011": 25, "111": 15}) >>> (r["observed"], round(r["missing_no_three_way"], 6)) (390.0, 378.0)
- Return type:
- morie.research.dark_figure_two_source(n1, n2, m, kappa=1.0, chapman=False)[source]¶
Two-source (capture-recapture) count with a dependence box.
Under independence the true count is the Lincoln-Petersen ratio
n1 n2 / m. With the dependence factor only known to lie in[1/kappa, kappa]the true count lies in[n1 n2 / (kappa m), kappa n1 n2 / m], both ends attainable (Research.P1.TwoSource.petersen_bounds).Examples
>>> dark_figure_two_source(400, 250, 80)["point"] 1250.0 >>> r = dark_figure_two_source(400, 250, 80, kappa=2) >>> (r["lower"], r["upper"]) (625.0, 2500.0)
- Return type:
- morie.research.detection_rate_shift(detected, total, n, d, q)[source]¶
Detection-rate arithmetic of downgrade-and-caution (
Research.P9.detection_rate_rises).Examples
>>> r = detection_rate_shift(detected=2000, total=10000, n=1500, d=0.2, q=0.5) >>> (r["rate_before"], r["rate_after"], r["rise"]) (0.2, 0.26, 0.06)
- Return type:
- morie.research.deterrence_design_check(p, s, c)[source]¶
Which deterrence partial effects a design can identify (
Research.P8.constant_dimension_not_identified).Examples
>>> r = deterrence_design_check(p=[0.1, 0.3, 0.5], s=[2, 2, 2], c=[30, 30, 30]) >>> (r["rank"], r["identified"]) (2, {'certainty': True, 'severity': False, 'celerity': False})
- Return type:
- morie.research.deterrence_response(x, benefit, sanction, p)[source]¶
Direction of deterrence without convexity (
certainty_monotone,severity_monotone,aggregate_monotone).Examples
>>> r = deterrence_response([0, 1, 2, 3], [0, 2, 3, 3.5], [0, 1, 3, 6], p=[0.1, 0.5, 1]) >>> list(r["x_opt"]) [3.0, 2.0, 1.0]
- morie.research.dfl_reweight(group, x, y, weights=None)[source]¶
DiNardo-Fortin-Lemieux reweighting: composition and structure without a linear model.
Research.P15Reweight.reweighting_matches/reweighted_mass/counterfactual_outcome/decomposition.Examples
>>> g = [True] * 6 + [False] * 6 >>> x = ["a", "a", "a", "a", "b", "b", "a", "a", "b", "b", "b", "b"] >>> y = [10, 12, 11, 13, 20, 22, 8, 9, 15, 16, 14, 17] >>> r = dfl_reweight(g, x, y) >>> tuple(round(r[k], 12) for k in ("mean_1", "mean_0", "counterfactual", "structure", "composition")) (14.666666666667, 13.166666666667, 10.833333333333, 3.833333333333, -2.333333333333) >>> (r["psi"]["x"], r["psi"]["psi"], r["max_composition_gap"]) (['a', 'b'], [2.0, 0.5], 0.0)
- Return type:
- morie.research.disparity_benchmark(pop, contact, force, reference, exposure_error_factor=None)[source]¶
Disparity benchmarks: the product identity and the exposure-offset shift (
Research.P2.benchmark_product,benchmark_not_additive,offset_shift,disparity_ratio_shift).Examples
>>> d = disparity_benchmark({"A": 100, "B": 100}, {"A": 30, "B": 10}, {"A": 12, "B": 2}, reference="B") >>> ([round(v, 12) for v in d["resident_disparity"]], [round(v, 12) for v in d["product_check"]]) ([6.0, 1.0], [0.0, 0.0])
- morie.research.disparity_decomposition(y, X, group, reference, names=None, shift=1.0)[source]¶
Oaxaca-Blinder decomposition of a gap in means, with both references and the interaction.
Xholds the covariates (one column each, no constant: it is added),groupthe two-level label per row,referencethe level whose coefficients price the explained part (group B).Examples
>>> y = [10, 12, 13, 15, 9, 8, 11, 7]; X = [[1, 0], [2, 1], [3, 1], [4, 0], [1, 1], [2, 0], [2, 0], [3, 1]] >>> r = disparity_decomposition(y, X, ["A"] * 4 + ["B"] * 4, reference="B", names=["x1", "x2"]) >>> (round(r["gap"], 12), [round(abs(v), 12) for v in r["identity_checks"].values()]) (3.75, [0.0, 0.0, 0.0, 0.0]) >>> ({k: round(v, 12) for k, v in r["twofold_B"].items()}, round(r["threefold"]["interaction"], 12)) ({'explained': -0.5, 'unexplained': 4.25}, 1.3)
- Return type:
- morie.research.disparity_exposure_bounds(y, m, gamma=1.0, reference=None)[source]¶
Bounds on a police-outcome disparity when exposure is measured by a proxy (
Research.P2.rate_bounds,disparity_bounds,disparity_sign_identified).yandmare dicts keyed by group.Examples
>>> d = disparity_exposure_bounds({"A": 300, "B": 100}, {"A": 1000, "B": 1000}, gamma=1.5, reference="B") >>> ([round(v, 12) for v in d["ratio_proxy"]], [round(v, 12) for v in d["ratio_lower"]], list(d["direction_identified"])) ([3.0, 1.0], [1.333333333333, 0.444444444444], [True, False])
- morie.research.ecological_bounds(p, q, weights=None)[source]¶
Duncan-Davis bounds: what neighbourhood marginals say about an individual rate.
max(0, (p+q-1)/p) <= P(y | x) <= min(1, q/p), both ends attained (Research.P12.dd_bounds,Cells.ends_attained); the complement rate follows fromq = p r + (1-p) r'(dd_complement); the aggregate rate inherits them_g p_g-weighted mean of the intervals (dd_aggregate_bounds).Examples
>>> r = ecological_bounds([0.2, 0.5, 0.8], [0.1, 0.3, 0.6], weights=[1000, 2000, 500]) >>> ([round(float(v), 12) for v in r["neighbourhoods"]["upper"]], round(r["aggregate"]["upper"], 12)) ([0.5, 0.6, 0.75], 0.625)
- Return type:
- morie.research.ecological_decompose(x, y, group)[source]¶
Between/within decomposition of a correlation across groups.
Population (divide by n) conventions throughout, matching the Lean definitions.
Examples
>>> r = ecological_decompose([0, 2, 1, 3], [1, 3, 0, 2], ["a", "a", "b", "b"]) >>> (round(r["corr_individual"], 12), r["corr_ecological"], r["sign_reversed"]) (0.6, -1.0, True)
- Return type:
- morie.research.fairness_base_rate_bounds(p_obs, alpha_max, beta_max)[source]¶
Sharp bounds on a true base rate from a noisy recorded one (
Research.P5.true_base_rate_bounds).Examples
>>> b = fairness_base_rate_bounds([0.35, 0.55], alpha_max=0.10, beta_max=0.20) >>> [round(v, 12) for v in b["upper"]] [0.4375, 0.6875]
- morie.research.fairness_compare_groups(p_obs_a, p_obs_b, alpha_max, beta_max)[source]¶
Can two groups’ true base rates be ordered under label noise? (
compare_decided,compare_undecided)Examples
>>> fairness_compare_groups(0.35, 0.55, alpha_max=0.05, beta_max=0.20)["order"] 'a < b' >>> fairness_compare_groups(0.35, 0.55, alpha_max=0.05, beta_max=0.40)["order"] 'undecided'
- Return type:
- morie.research.fairness_implied_fpr(p, ppv, fnr)[source]¶
Chouldechova’s identity
fpr = p/(1-p) (1-ppv)/ppv (1-fnr)(Research.P5.Table.chouldechova).Examples
>>> r = fairness_rates(120, 60, 40, 280) >>> round(fairness_implied_fpr(r["p"], r["ppv"], r["fnr"]) - r["fpr"], 15) 0.0
- Return type:
- morie.research.fairness_rates(tp, fp, fn, tn)[source]¶
Group-wise rates from a confusion table.
Examples
>>> r = fairness_rates(tp=120, fp=60, fn=40, tn=280) >>> (round(r["p"], 12), round(r["ppv"], 12), round(r["fpr"], 12), r["fnr"]) (0.32, 0.666666666667, 0.176470588235, 0.25)
- Return type:
- morie.research.fairness_true_rate(p_obs, alpha, beta)[source]¶
Recover a true base rate when the noise rates are known (
Research.P5.trueRate_observedRate).Examples
>>> round(float(fairness_true_rate(0.4 * 0.8 + 0.6 * 0.1, alpha=0.1, beta=0.2)[0]), 12) 0.4
- morie.research.feedback_loop_bound(lam_a, lam_b, c_a0, c_b0, n_steps=1000)[source]¶
Proved upper bound on how far the naive loop can move in N steps (
naiveShare_rate_bound).Examples
>>> round(feedback_loop_bound(0.3, 0.2, 10, 10, n_steps=4)["bound"], 12) 0.004926706191
- Return type:
- morie.research.feedback_loop_limit(lam_a, lam_b, c_a0, c_b0, update='naive', rho=0.0)[source]¶
Proved limit of the two-region feedback loop, with the theorem that proves it.
Examples
>>> feedback_loop_limit(0.3, 0.2, 10, 10)["share_a"] 1 >>> feedback_loop_limit(0.3, 0.2, 10, 10, "corrected")["share_a"] 0.6 >>> round(feedback_loop_limit(0.3, 0.2, 10, 10, rho=0.5)["cap"], 12) 0.75
- Return type:
- morie.research.feedback_loop_meanfield(lam_a, lam_b, c_a0, c_b0, n_steps=100, update='naive', rho=0.0)[source]¶
Mean-field feedback-loop recursion for two regions.
Returns a frame with
step, share_a, c_a, c_b;frame.attrs["limit"]holds the proved limit (seefeedback_loop_limit()).Examples
>>> mf = feedback_loop_meanfield(0.3, 0.2, 7, 3, n_steps=1) >>> x = 0.7; drift = x * (1 - x) * 0.1 / (10 + 0.3 * x + 0.2 * (1 - x)) >>> round(float(mf["share_a"][1] - mf["share_a"][0]) - drift, 15) 0.0
- morie.research.feedback_loop_sim(lam_a, lam_b, c_a0, c_b0, n_steps=1000, update='naive', rho=0.0, n_sims=100, seed=0)[source]¶
Stochastic urn simulation of the two-region feedback loop on the shared Philox stream.
Examples
>>> s = feedback_loop_sim(0.3, 0.2, 10, 10, n_steps=50, n_sims=3, seed=1) >>> (s["share_a"].shape, s["limit"]["theorem"]) ((3, 51), 'Research.P4.naiveShare_tendsto_one')
- Return type:
- morie.research.feedback_loop_urn_law(n_steps)[source]¶
Exact law of the stochastic two-region urn with equal rates (
polya_uniform).Examples
>>> law = feedback_loop_urn_law(10) >>> (round(float(law["prob"][0]), 12), law.attrs["prob_middle"] >= 0.25) (0.090909090909, True)
- morie.research.hazard_selection(s, h, l, survive_high, survive_low)[source]¶
Built-in selection in period-by-period hazard ratios (
survivor_hazard_mono,hr2_gt_one_of_depletion).Examples
>>> r = hazard_selection(s=0.5, h=0.5, l=0.1, survive_high=(0.5, 0.8), survive_low=(0.9, 0.9)) >>> round(r["period2_hazard_ratio"], 12) 1.186851211073
- Return type:
- morie.research.hotspot_shrinkage(y, noise_variance, weights=None, B=None)[source]¶
Empirical-Bayes shrinkage of hot-spot counts: the expected size of the fall.
Research.P19Shrinkage.loss_min/bstar_mem/predicted_fall.Examples
>>> s = hotspot_shrinkage([40, 12, 9, 25, 7, 31, 5, 18], noise_variance=18.375) >>> (round(s["B"], 12), round(s["predicted_fall"][0], 12), round(s["shrunk"][0], 12), s["total_variance"]) (0.132686449284, 2.869344465757, 37.130655534243, 138.484375)
- Return type:
- morie.research.incapacitation(lam, q, S, shares=None)[source]¶
Crime rate, prevented share and the marginal sentence year under the incapacitation model.
Examples
>>> r = incapacitation(lam=[2, 10], q=0.1, S=1, shares=[0.8, 0.2]) >>> ([round(float(v), 12) for v in r["rate"]], {k: round(v, 12) for k, v in r.attrs["aggregate"].items()}) ([1.666666666667, 5.0], {'free_rate': 3.6, 'incapacitated_rate': 2.333333333333, 'prevented_share': 0.351851851852})
- morie.research.incapacitation_career(lambda_path, t0, S, replacement=0.0)[source]¶
Incapacitation under desistance (a non-increasing rate path) and replacement.
Research.P17Replacement.prevented_le_const/prevented_ge_const/later_sentence_prevents_less/replaced_antitone/prevented_net_le.Examples
>>> r = incapacitation_career([12, 10, 8, 6, 5, 4, 3, 2, 2, 1], t0=2, S=3, replacement=0.25) >>> (r["prevented"], r["prevented_net"], r["upper"], r["lower"], r["later"]) (19.0, 14.25, 24.0, 12.0, 15.0)
- Return type:
- morie.research.interracial_rates(offences, population)[source]¶
Interracial offending rates against the random-mixing null (
rate_per_offender_group,dyad_ratio).Examples
>>> d = interracial_rates({"A_on_B": 120, "B_on_A": 200, "A_on_A": 900, "B_on_B": 300}, {"A": 80000, "B": 20000}) >>> [round(v, 12) for v in d["rate_per_pair_exposure"]] [0.0075, 0.0125, 0.0140625, 0.075]
- morie.research.judge_iv(z, d, y, weights=None, defier_share=(0, 0.05, 0.1), defier_effect=(0,))[source]¶
Observational Wald ratio, the type shares under monotonicity, and the defier sensitivity.
Examples
>>> z = [0, 0, 0, 0, 1, 1, 1, 1]; d = [0, 0, 1, 0, 1, 1, 1, 0]; y = [1, 2, 5, 1, 6, 4, 5, 2] >>> r = judge_iv(z, d, y) >>> (round(r["first_stage"], 12), round(r["itt"], 12), round(r["wald"], 12)) (0.5, 2.0, 4.0)
- Return type:
- morie.research.judge_iv_population(d0, d1, y0, y1, weights=None)[source]¶
The exact decomposition of a binary-instrument design on a known population.
Examples
>>> r = judge_iv_population([0, 0, 1, 0, 1], [1, 1, 1, 0, 0], [0, 0, 1, 0, 0], [1, 0, 1, 1, 1]) >>> ({k: round(v, 12) for k, v in r["shares"].items()}, round(r["wald"], 12)) ({'complier': 0.4, 'defier': 0.2, 'always': 0.2, 'never': 0.2}, 0.0)
- Return type:
- morie.research.judge_slope_test(judge, d, y, weights=None, lo=None, hi=None)[source]¶
The many-judge slope test of monotonicity (Frandsen, Lefgren & Leslie 2023).
Research.P14Slope.propensity_mono/outcome_diff/slope_bound/violation_refutes_monotonicity.Examples
>>> judge = ["A"] * 6 + ["B"] * 6 + ["C"] * 6 >>> d = [0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0] >>> y = [1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 1] >>> s = judge_slope_test(judge, d, y) >>> (s["judges"]["judge"], [round(p, 12) for p in s["judges"]["propensity"]], s["pairs"]["violation"], s["violations"]) (['A', 'B', 'C'], [0.333333333333, 0.666666666667, 0.833333333333], [False, False, True], 1) >>> [round(v, 12) for v in s["pairs"]["late"]] [0.5, 1.0, 2.0]
- Return type:
- morie.research.judge_monotonicity(judge, d, group, min_n=30, adjust='holm', alpha=0.05)[source]¶
Do judges’ leniency rankings hold inside every subgroup? (
Research.P14Slope.subgroup_propensity_mono,subgroup_reversal_refutes,crossing_refutes_both).R parity:
morie_judge_monotonicity(Holm adjustment, pooled one-sided two-proportion test).Examples
>>> r = judge_monotonicity(["a"] * 4 + ["b"] * 4, [1, 1, 1, 0, 0, 0, 0, 1], ["g"] * 8, min_n=1) >>> (r["summary"]["comparisons"], r["summary"]["significant"]) (1, 0)
- Return type:
- morie.research.logit_rescale(beta, omitted_var, error_var=3.289868133696453)[source]¶
Rescaling of logit coefficients across nested models (
Research.P5.reduced_coefficientand kin).Examples
>>> r = logit_rescale(beta=0.8, omitted_var=1) >>> round(r["rescale"], 12) 0.875724044222
- Return type:
- morie.research.logit_separation(y, x, intercept=True, method='auto')[source]¶
Detect separation in a logistic regression and show why the fit cannot converge.
The exact check is the linear programme: maximise
sum_i s_i x_i . dsubject to0 <= s_i x_i . d <= 1; a positive optimum is a separating direction (complete when every margin is positive, quasi-complete when some are zero).method="glm"inspects an iteratively reweighted fit for fitted probabilities at 0 or 1 instead, a heuristic.Examples
>>> x = [[1, 0.2], [1, -1], [1, 0.5], [0, 0.1], [0, -0.4], [0, 1.2], [0, 0.3]] >>> r = logit_separation([1, 1, 1, 0, 0, 0, 0], x) >>> (r["separation"], all(v < 0 for v in r["loglik_along"].values()), r["loglik_along"]["t=100"] > -1e-6) ('complete', True, True)
- Return type:
- morie.research.meta_dl_bias(variances, n_draws=2000, seed=0)[source]¶
Size of the DerSimonian-Laird truncation bias under homogeneity (shared Philox stream).
Examples
>>> b = meta_dl_bias([0.010, 0.020, 0.015, 0.030, 0.012], n_draws=500, seed=1) >>> (b["mean_tau2"] > 0, 0 < b["share_positive"] < 1) (True, True)
- Return type:
- morie.research.meta_hksj(estimates, variances, tau2='DL', level=0.95)[source]¶
Hartung-Knapp-Sidik-Jonkman interval with DerSimonian-Laird or REML heterogeneity.
Research.P13HKSJ.hksj_wider_iff(wider than Wald iffq >= 1),Q_eq_zero_iff(degenerate iff every site equals the pooled value),hksj_equal_weights(equal weights: the one-sample t variance).Examples
>>> h = meta_hksj([-0.25, -0.10, -0.40, 0.05, -0.30], [0.010, 0.020, 0.015, 0.030, 0.012]) >>> (round(h["tau2"], 12), round(h["q"], 12), round(h["hksj"]["se"], 12), round(h["wald"]["se"], 12), h["wider_than_wald"]) (0.006967741935, 1.086177559643, 0.070056783817, 0.067220197281, True) >>> r = meta_hksj([-0.25, -0.10, -0.40, 0.05, -0.30], [0.010, 0.020, 0.015, 0.030, 0.012], tau2="REML") >>> (round(r["tau2"], 12), round(r["q"], 12)) (0.002908035914, 1.271198964824)
- Return type:
- morie.research.meta_random_effects(estimates, variances, level=0.95)[source]¶
Fixed-effect and DerSimonian-Laird random-effects pooling, with what the truncation does.
Examples
>>> m = meta_random_effects([-0.25, -0.10, -0.40, 0.05, -0.30], [0.010, 0.020, 0.015, 0.030, 0.012]) >>> (round(m["tau2"], 12), m["variance_ratio"] >= 1, m["truncated"]) (0.006967741935, True, False)
- Return type:
- morie.research.probability_of_necessity(p_treated, p_control)[source]¶
Probability of necessity bounds (
Research.P8.necessity_bounds,necessity_of_monotone).Examples
>>> r = probability_of_necessity(p_treated=0.6, p_control=0.4) >>> ({k: round(v, 12) for k, v in r["necessary_share_bounds"].items()}, round(r["pn_monotone"], 12)) ({'lower': 0.2, 'upper': 0.6}, 0.333333333333)
- Return type:
- morie.research.ranking_resolution(estimate, half_width)[source]¶
Resolution of a ranking built from noisy risk scores (
rank_reversal_exists,rank_stable_of_gap).Examples
>>> r = ranking_resolution([0.2, 0.35, 0.8], half_width=[0.1, 0.1, 0.05]) >>> (r["n_pairs"], round(r["share_unidentified"], 12), r["resolution"]) (3, 0.333333333333, 0.2)
- Return type:
- morie.research.recording_map(M, counts, names=None)[source]¶
Recorded counts from true counts through a recording matrix.
M[i, j]is the share of true category-j offences recorded as category i.Examples
>>> r = recording_map([[0.7, 0], [0.3, 1]], [100, 400], names=["robbery", "theft"]) >>> (r["recorded"], r["regime"], r["recorded_total"]) ({'robbery': 70.0, 'theft': 430.0}, 'reclassification', 500.0)
- Return type:
- morie.research.recording_regimes(recorded, cuffed_share=0.1, true_total=None, names=None)[source]¶
Two recording regimes from one recorded series (
Research.P9.regimes_not_identified,second_source_separates).The identity map on true counts equal to the series and the map
(1 - eps) Ionrecorded / (1 - eps)both reproduce the series; only a second source that fixes the true total tells them apart.Examples
>>> two = recording_regimes([70, 430], cuffed_share=0.2) >>> (two["reclassification"]["true_total"], two["cuffing"]["true_total"], two["identified_from_series"]) (500.0, 625.0, False) >>> recording_regimes([70, 430], true_total=550)["second_source"]["regime"] 'cuffing'
- Return type:
- morie.research.regression_to_mean(x1, x2, threshold, weights=None)[source]¶
Observed, mirror and symmetrised change on the places selected for a high first-period count.
The symmetrised change is the same statistic on the data plus its period-swapped copy, an exchangeable population, so it is non-positive by
Research.P19.selected_change_nonpos.Examples
>>> r = regression_to_mean([5, 1, 7, 2, 9, 3], [3, 2, 6, 2, 4, 5], threshold=4) >>> (r["n_selected"], round(r["selected_change"], 12), r["symmetrised_change"] <= 0) (3, -2.666666666667, True)
- Return type:
- morie.research.relative_risk_from_or(odds_ratio, base_rate=None, exposed_share=None)[source]¶
Relative risk from an odds ratio: the interval arrest-only data allow (
rr_between,or_overstates).Examples
>>> relative_risk_from_or(3)["rr_bounds"] {'lower': 1, 'upper': 3.0} >>> round(relative_risk_from_or(3, base_rate=0.2, exposed_share=0.3)["risks"]["relative_risk"], 9) 2.333333333
- Return type:
- morie.research.selective_labels(y, released, rule_released, weights=None)[source]¶
What a release rule’s failure rate can be known from the cases a judge released.
Examples
>>> y = [0, 1, 0, 1, 1, 0]; released = [True, True, True, True, False, False] >>> r = selective_labels(y, released, [True, True, True, False, False, False]) >>> (r["identified"], round(r["rule_rate"], 12)) (True, 0.333333333333) >>> r2 = selective_labels(y, released, [True, True, True, True, True, False]) >>> ({k: round(v, 12) for k, v in r2["bounds"].items()}, round(r2["width"], 12)) ({'lower': 0.4, 'upper': 0.6}, 0.2)
- Return type:
- morie.research.selective_labels_judges(y, released, rule_released, weights=None, names=None)[source]¶
Selective labels across judges ordered from strictest to most lenient (
Research.P18.chain_width_antitone,chain_contraction,chain_width_zero_iff).releasedholds one release indicator per judge (a list of columns, or an n x k matrix). With nested release sets the interval narrows with leniency; the most lenient judge gives the sharpest interval, a point exactly when the rule’s set lies inside it.Examples
>>> y = [0, 1, 0, 1, 1, 0] >>> judges = [[True, True, False, False, False, False], [True, True, True, True, False, False]] >>> r = selective_labels_judges(y, judges, [True, True, True, False, False, False]) >>> ([round(w, 12) for w in r["widths"]], r["nested"], r["identified"], round(r["rule_rate"], 12)) ([0.333333333333, 0.0], True, True, 0.333333333333)
- Return type:
- morie.research.sentence_effect_bounds(y, z, weights=None, contrast=None)[source]¶
Worst-case identification bounds for a binary outcome under two sentences.
P(y=1, z=t) <= P[y(t)=1] <= P(y=1, z=t) + P(z != t), both ends attained (Research.P11.Pop.outcome_bounds) the contrast interval has width one and contains zero (ate_width_one,ate_contains_zero).Examples
>>> b = sentence_effect_bounds([1, 0, 1, 0, 1, 1], ["a", "a", "a", "b", "b", "b"]) >>> (round(b["ate_width"], 12), round(b["ate_bounds"]["lower"], 12), round(b["ate_bounds"]["upper"], 12)) (1.0, -0.5, 0.5)
- Return type:
- morie.research.sentence_effect_mtr(y, z, weights=None, contrast=None, direction='non-decreasing')[source]¶
Monotone-treatment-response bounds (
Research.P11.Pop.mtr_lower,mtr_upper,mtr_upper_attained).Examples
>>> r = sentence_effect_mtr([1, 0, 1, 0, 1, 1], ["a", "a", "a", "b", "b", "b"]) >>> (r["bounds"]["lower"], round(r["bounds"]["upper"], 12)) (0, 0.5)
- Return type:
- morie.research.sentence_effect_mts(y, z, weights=None, contrast=None)[source]¶
Monotone-treatment-selection bounds for a sentencing contrast (Manski & Pepper 2000).
The observed mean of the harsher group bounds
E[y(b)]from above and the lighter group’s boundsE[y(a)]from below (Research.P11.Pop.mts_mean_b_le,mts_mean_a_ge); the naive difference overstates the effect (mts_ate_le_naive) with MTR as well the contrast lies in[0, naive](mtr_mts_bounds).Examples
>>> r = sentence_effect_mts([1, 0, 1, 0, 1, 1], ["a", "a", "a", "b", "b", "b"]) >>> ({k: round(v, 12) for k, v in r["ate_bounds_mts"].items()}, r["ate_bounds_mtr_mts"]) ({'lower': -0.5, 'upper': 0.0}, {'lower': 0, 'upper': 0.0})
- Return type:
- morie.research.shrinkage_loss(theta, noise, weights=None, B=None)[source]¶
Loss of the shrinkage estimator against a known truth, next to the closed form of
loss_eq.Research.P19Shrinkage.loss_eq/loss_min/loss_bstar_eq/loss_bstar_le_raw.Examples
>>> l = shrinkage_loss([10, 20, 30, 40], [3, -3, -3, 3], B=0.5) >>> (l["loss"], l["closed_form"], round(l["B_star"], 12), round(l["loss_star"], 12), l["loss_raw"], l["noise_law"]) (134.0, 134.0, 0.067164179104, 33.582089552239, 36.0, True)
- Return type:
- morie.research.spillover_effects(y, exposure, stratum, weights=None)[source]¶
Direct, spillover and net effects under a stated exposure mapping (stratified exposure adjustment).
Examples
>>> y = [5, 4.6, 4, 6, 5.6, 5]; e = [0, 1, 2, 0, 1, 2]; s = ["a", "a", "a", "b", "b", "b"] >>> r = spillover_effects(y, e, s) >>> (round(r["spillover"], 12), round(r["direct"], 12), r["positivity"]) (-0.4, -0.6, True)
- Return type:
- morie.research.spillover_exposure(treated, edges)[source]¶
Three-level exposure from a treatment on a place network (edges are 1-based pairs, as in R).
Examples
>>> e = spillover_exposure([1, 0, 0, 0, 1], [[1, 2], [2, 3], [3, 4], [4, 5]]) >>> (list(e["exposure"]), list(e["treated_neighbours"])) ([2, 1, 0, 1, 2], [0, 1, 0, 1, 0])
- morie.research.spillover_exposure_probs(n, edges, n_treated, n_draws=5000, exact_max=20000, joint=False, seed=0)[source]¶
Exposure probabilities of a completely randomised deployment (exact enumeration, else Philox Monte Carlo).
Examples
>>> pr = spillover_exposure_probs(4, [[1, 2], [2, 3], [3, 4]], 1) >>> [round(float(v), 12) for v in pr[:, 2]] [0.25, 0.25, 0.25, 0.25]
- morie.research.spillover_ht(y, exposure, probs, joint=None)[source]¶
Horvitz-Thompson totals and contrasts under a randomised deployment (
ht_unbiased,ht_variance).Examples
>>> pr = spillover_exposure_probs(6, [[1, 2], [2, 3], [3, 4], [4, 5], [5, 6]], 2, joint=True) >>> e = spillover_exposure([0, 1, 0, 0, 0, 1], [[1, 2], [2, 3], [3, 4], [4, 5], [5, 6]])["exposure"] >>> r = spillover_ht([5, 3.5, 4.5, 5, 4.5, 3.5], list(e), pr["marginal"], pr["joint"]) >>> (round(r["total_effect"], 12), sorted(r.keys())[:3]) (-0.666666666667, ['direct', 'means', 'se'])
- Return type:
- morie.research.spillover_ht_variance(y_pot, pi, pij, form='ht')[source]¶
Design variance of the Horvitz-Thompson total (
ht_variance; SYG form,syg_eq_ht).Examples
>>> pr = spillover_exposure_probs(6, [[1, 2], [2, 3], [3, 4], [4, 5], [5, 6]], 2, joint=True) >>> v = spillover_ht_variance([1] * 6, pr["marginal"][:, 2], pr["joint"][:, :, 2], form="both") >>> (v["fixed_size"], round(v["level_count"], 12)) (True, 2.0)
- morie.research.two_point_bound(p, q, theta_p, theta_q, estimator=None)[source]¶
Le Cam’s two-point lower bound on a finite sample space.
Examples
>>> p = [0.0625, 0.25, 0.375, 0.25, 0.0625] >>> q = [0.0256, 0.1536, 0.3456, 0.3456, 0.1296] >>> b = two_point_bound(p, q, theta_p=2, theta_q=3, estimator=[0, 1.25, 2.5, 3.75, 5]) >>> (round(b["tv"], 12), round(b["bound"], 12), round(b["minimax_risk"], 12), b["satisfied"], round(b["sum_min"], 12)) (0.1627, 0.41865, 1.125, True, 0.8373)
- Return type: