MRM empirical callables (OTIS / TPS / SIU)
Source:vignettes/mrm-empirical-callables.Rmd
mrm-empirical-callables.RmdOverview
This vignette documents the mrm_otis_*(),
mrm_tps_*(), and mrm_siu_*() empirical
callables. Each is a one-line entry point to one analysis. The OTIS and
TPS examples run on the small reference samples returned by
morie_sample(); the SIU examples run on the corpus bundled
in rmoriedata. The vignette needs no network access.
For the full datasets:
- OTIS ->
morie_load_dataset("otisb01")(downloads from data.ontario.ca on first call, then reads the local cache) - TPS ->
morie_fetch_tps("Assault")(ArcGIS REST) - SIU ->
morie_fetch_siu()(the rmoriedata corpus by default; a live fetch withoptions(morie.siu.allow_fetch = TRUE))
See vignette("mrm-dataset-fetchers") for the dataset
side.
library(rmorie)
b01 <- morie_sample("otis_b01")
b09 <- morie_sample("otis_b09")
tps <- morie_sample("tps_assault")
# Some events carry latitude and longitude 0, i.e. no usable
# location; drop them before any spatial analysis.
sum(tps$LAT_WGS84 == 0 | tps$LONG_WGS84 == 0)
#> [1] 21
tps <- tps[tps$LAT_WGS84 != 0 & tps$LONG_WGS84 != 0, ]Left in, those points stretch the grid used by
mrm_tps_moran_clustering() across the Atlantic, put almost
every Toronto event in one cell, and add steps of thousands of
kilometres to mrm_tps_levy_scaling().
OTIS suite
Placement-count concentration on b09
The b09 long-format file publishes per (fiscal year x placement-count band x gender) counts of individuals in segregation. The callable expands the banded counts using midpoints and returns Hill-MLE Pareto exponent, Gini coefficient, mean placements per individual, and the top-k% concentration share.
mrm_otis_placement_concentration(b09)
#> year n_individuals n_placements mean_per_individual gini hill_alpha
#> 1 2023 12647 55421 4.382146 0.5331 1.6890
#> 2 2024 10881 47123 4.330760 0.5862 1.7565
#> 3 2025 9608 46893 4.880620 0.6057 1.7230
#> 4 pooled 33136 149437 4.509808 0.5748 1.7198
#> top_pct_share
#> 1 0.2932
#> 2 0.3351
#> 3 0.3215
#> 4 0.3180The values are computed within fiscal year: OTIS
UniqueIndividual_ID has format YYYY-XXXXX-SG
and is randomly reassigned every fiscal year, so cross-year tracking is
invalid by design.
Segregation-duration KM on b01
NumberConsecutiveDays_Segregation is the duration in
days of each placement (no censoring – all durations are observed). The
callable reports per stratum the mean and median duration,
q75_days (the 75th percentile: the duration that a quarter
of placements exceed), the percentage of placements above the UN Mandela
15-day cutoff, and the durations at which the empirical survival
function falls to 50, 25, 10, 5 and 1 percent.
mrm_otis_seg_duration_km(b01)
#> stratum n mean_days median_days q75_days pct_above_mandela
#> 1 pooled 1000 3.03 2 3 0.7
#> median_among_above_mandela days_at_S50 days_at_S25 days_at_S10 days_at_S05
#> 1 54 2 3 5 7
#> days_at_S01
#> 1 12.01
mrm_otis_seg_duration_km(b01, group_cols = "MentalHealth_Alert")
#> stratum n mean_days median_days q75_days pct_above_mandela
#> 1 No 499 2.76 2 3 0.6
#> 2 Yes 501 3.29 2 4 0.8
#> median_among_above_mandela days_at_S50 days_at_S25 days_at_S10 days_at_S05
#> 1 54.0 2 3 4 6
#> 2 41.5 2 4 5 8
#> days_at_S01
#> 1 11.02
#> 2 13.00This callable replaces the misreading of YYYY-XXXXX-SG
as a persistent person identifier, which produces a spurious cross-year
“time-to-readmission” artifact.
Mortification co-occurrence (alert columns)
The three b01 alert flags (MentalHealth_Alert,
SuicideRisk_Alert, SuicideWatch_Alert)
co-occur to a degree well above independence. The substantive figure is
MentalHealth x SuicideRisk Cramer’s V.
mrm_otis_mortification_cooccurrence(b01)
#> alert_a alert_b n chi2 df p_value cramers_v
#> 1 MentalHealth_Alert SuicideRisk_Alert 1000 33.25 1 8.12e-09 0.1823
#> 2 MentalHealth_Alert SuicideWatch_Alert 1000 12.09 1 5.08e-04 0.1099
#> 3 SuicideRisk_Alert SuicideWatch_Alert 1000 470.37 1 2.66e-104 0.6858Region locality
mrm_otis_region_locality() cross-tabulates the region at
the time of placement against the most recent region. The diagonal share
is the fraction of placements whose region did not change:
res <- mrm_otis_region_locality(b01)
print(res$table)
#>
#> Central Eastern Northern Toronto Western
#> Central 391 1 1 7 11
#> Eastern 6 112 0 3 2
#> Northern 0 0 92 0 2
#> Toronto 3 2 1 155 2
#> Western 6 2 1 2 198
cat("diagonal share:", res$diagonal_share, " V:", res$cramers_v, "\n")
#> diagonal share: 0.948 V: 0.9348Mandela classification
mrm_classify_mandela() supports three denominators:
mrm_classify_mandela(b01, denominator = "row") # per-placement
#> year denominator n_mandela rate pct n_broader_rc rate_broader
#> 1 2023 362 0 0.000000000 0.00 0 0.000000000
#> 2 2024 337 3 0.008902077 0.89 3 0.008902077
#> 3 2025 301 4 0.013289037 1.33 4 0.013289037
#> 4 pooled 1000 7 0.007000000 0.70 7 0.007000000
mrm_classify_mandela(b01, denominator = "individual_any") # per-person
#> year denominator n_mandela rate pct n_broader_rc rate_broader
#> 1 2023 354 0 0.000000000 0.00 0 0.000000000
#> 2 2024 329 3 0.009118541 0.91 3 0.009118541
#> 3 2025 289 4 0.013840830 1.38 4 0.013840830
#> 4 pooled 972 7 0.007201646 0.72 7 0.007201646
mrm_classify_mandela(b01, denominator = "individual_cumulative")
#> year denominator n_mandela rate pct n_broader_rc rate_broader
#> 1 2023 354 0 0.000000000 0.00 0 0.000000000
#> 2 2024 329 3 0.009118541 0.91 3 0.009118541
#> 3 2025 289 5 0.017301038 1.73 5 0.017301038
#> 4 pooled 972 8 0.008230453 0.82 8 0.008230453The per-individual rates by fiscal year come from table c11, which
morie_sample("otis_c11") returns whole.
morie_otis_analyze_c11_mandela_classification() classifies
the individuals in each year as at or below 15 days (solitary) or above
(torture, in the function’s labels):
c11 <- morie_sample("otis_c11")
m11 <- morie_otis_analyze_c11_mandela_classification(c11)
rows <- lapply(m11$tables[[1]]$rows, function(r) unlist(lapply(r, as.character)))
tab11 <- as.data.frame(do.call(rbind, rows))
names(tab11) <- m11$tables[[1]]$headers
tab11[tab11$Type == "Segregation", ]
#> Year Type Solitary N Solitary % Torture N Torture % Total
#> 1 2023 Segregation 11071 87.5% 1576 12.5% 12647
#> 3 2024 Segregation 9084 83.5% 1797 16.5% 10881
#> 5 2025 Segregation 7626 79.4% 1982 20.6% 9608TPS suite
Levy-flight Hill exponent on inter-event step lengths
Treats consecutive events in chronological order as a single stream
and computes the haversine inter-event step length (km). Returns the
Hill-MLE exponent restricted to steps above
min_step_km.
mrm_tps_levy_scaling(tps)
#> $n_events
#> [1] 979
#>
#> $n_dropped_coords
#> [1] 0
#>
#> $n_steps_tail
#> [1] 974
#>
#> $min_step_km
#> [1] 0.5
#>
#> $hill_alpha
#> [1] 1.3336Moran’s I + DBSCAN clustering
Grids the WGS84 extent into a coarse raster, counts events per cell, and computes the global Moran’s I via a rook-contiguity matrix. Also runs DBSCAN on the raw lat/long points (rescaled to km) for cluster counts.
mrm_tps_moran_clustering(tps, grid_resolution = 20L)
#> $morans_I
#> [1] 0.593565
#>
#> $morans_z
#> [1] 17.08
#>
#> $dbscan_n_clusters
#> [1] 20
#>
#> $dbscan_n_noise
#> [1] 727
#>
#> $dbscan_largest
#> [1] 82
#>
#> $n_events
#> [1] 979
#>
#> $n_dropped_coords
#> [1] 0For the full Assault layer, read the file that
morie_fetch_tps("Assault") writes
(utils::read.csv() on the returned path), drop the zero
coordinates, and pass the data frame.
Neighbourhood inter-event recurrence
For each HOOD_158 neighbourhood, sorts events
chronologically and computes the gap (in days) between consecutive
events.
head(mrm_tps_neighbourhood_recurrence_km(tps))
#> hood n_events n_gaps mean_gap_days median_gap_days p25_gap_days p75_gap_days
#> 1 001 17 16 267.06 213.5 101.00 356.75
#> 2 002 12 11 447.91 360.0 74.00 606.00
#> 3 003 5 4 887.75 869.0 557.75 1199.00
#> 4 004 3 2 1781.00 1781.0 1065.50 2496.50
#> 5 005 4 3 776.00 676.0 537.00 965.00
#> 6 006 5 4 845.00 731.0 289.50 1286.50SIU suite
The SIU callables operate on the SIU.csv file that
morie_fetch_siu() writes. By default it writes the
panel-reviewed corpus bundled in rmoriedata (English and French
reports); lang = "en" keeps the English rows.
siu_path <- morie_fetch_siu(lang = "en")
#> SIU: wrote the panel-reviewed corpus from rmoriedata (2309 rows) to /tmp/Rtmpm3faUk/morie/siu/SIU.csv. For a live re-fetch set options(morie.siu.allow_fetch = TRUE).
siu <- utils::read.csv(siu_path)
dim(siu)
#> [1] 2309 66
res <- mrm_siu_case_to_decision_km(siu)
print(res$pooled)
#> stratum n n_censored median_days mean_days p25_days p75_days max_days
#> 1 pooled 2295 5 121 237.44 118 280 12857
head(res$by_service[order(-res$by_service$n), ])
#> stratum n n_censored median_days mean_days
#> 30 Toronto Police Service 499 2 121.0 282.44
#> 18 Ontario Provincial Police 447 1 121.0 203.18
#> 20 Peel Regional Police 199 2 121.0 257.47
#> 15 Niagara Regional Police Service 102 0 126.5 219.97
#> 19 Ottawa Police Service 98 0 120.5 207.72
#> 10 Hamilton Police Service 96 0 139.5 265.16
#> p25_days p75_days max_days
#> 30 118 296 12857
#> 18 118 267 5944
#> 20 119 261 4998
#> 15 119 312 900
#> 19 118 303 857
#> 10 119 302 4552
head(mrm_siu_per_service_rate(siu))
#> service year n_cases
#> 1 Toronto Police Service 1982 1
#> 2 Durham Regional Police Service 2005 2
#> 3 Niagara Regional Police Service 2005 1
#> 4 Ontario Provincial Police 2005 1
#> 5 Owen Sound Police Service 2005 1
#> 6 Thunder Bay Police Service 2005 1mrm_siu_case_to_decision_km() measures the days from
incident to the Director’s decision; open cases are treated as censored.
The max_days column shows that some incident dates in the
corpus are decades before the decision, so check extreme rows before
using the mean.