Package index • rmoriebricklayer

Fetch open data

Resilient downloaders with libcurl + Wayback fallback for pulling public datasets over flaky endpoints.

bricklayer_fetch(): Fetch a URL to disk with an Internet Archive fallback (C++/libcurl)
bricklayer_fetch_siu(): Fetch an Ontario SIU director's report by drid
download_data(): Download a File
friendly_download(): Download a File With Diagnostic Error Messages

SIU report mining

Deterministic, offline parse/resolve core for Ontario Special Investigations Unit director’s reports — schema, parser, subject-official resolver.

bricklayer_siu_schema(): The panel-reviewed SIU report field schema
bricklayer_parse_siu(): Parse an SIU director's report into the schema fields
bricklayer_fetch_parse_siu(): Fetch and parse one SIU director's report
bricklayer_siu_text(): Convert SIU report HTML to plain text
bricklayer_siu_iso_date(): Convert a human-readable SIU report date to ISO format
bricklayer_siu_resolve_so(): Resolve the subject-official count from SIU report text

Portal resolvers

Turn a dataset identifier on a government open-data portal into a direct resource URL — CKAN, Socrata, ArcGIS, and Wayback snapshots.

resolve_via_ckan(): Resolve a Download URL via CKAN package_show
resolve_via_ckan_search(): Resolve a Download URL via CKAN package_search
resolve_via_socrata(): Resolve a Download URL via the Socrata Metadata API
resolve_via_arcgis(): Resolve a Query URL via ArcGIS FeatureServer Metadata
wayback_snapshot_url(): Resolve a Wayback Machine snapshot URL
wayback_snapshot_url_native(): Resolve a Wayback Machine snapshot URL (C++/libcurl)

Provenance & reproducibility capsules

Capture the environment, record a citable data-provenance capsule, and verify it later so an analysis reproduces from the same inputs.

capture_environment(): Capture the Analysis Environment for a Manifest
cite_capsule(): Generate a Data Citation From Provenance
verify_capsule(): Re-Verify an Entire Reproducible Data Capsule
load_provenance(): Load a Pinned Data-Provenance Record
record(): Record a Cross-Check Result in a Manifest
make_manifest(): Construct a Reproducibility Manifest
manifest_canonical() manifest_digest(): The canonical serialisation of a manifest, and its digest
manifest_recompute(): Recompute a manifest's recorded statistics against the data
manifest_record_seed() manifest_restore_seed(): Record and restore the random number generator state
capture_dependencies(): Where each loaded package came from
write_manifest_json(): Write a Manifest to JSON
write_summary_txt(): Write a Plain-Language Run Summary
agent_bundle(): Agent-assisted reproducibility-bundle help

JSON (native codec)

jsonlite’s mapping in base R, byte-for-byte; no jsonlite at run time.

bricklayer_json_from_json(): Parse JSON into R objects (jsonlite's fromJSON, natively)
bricklayer_json_to_json(): Encode an R object as JSON (jsonlite's toJSON, natively)
bricklayer_json_serialize() bricklayer_json_unserialize(): Lossless JSON serialisation of an R object
json_gzip_encode() json_gzip_decode(): Compressed, base64-encoded JSON
bricklayer_json_base64_enc() bricklayer_json_base64_dec() bricklayer_json_base64url_enc() bricklayer_json_base64url_dec(): Base64 encoding

Integrity & checksums

SHA-256 hashing and verification for downloaded files.

sha256_file(): Compute a File's SHA256 Digest
verify_sha256(): Verify a File's SHA256 Against an Expected Digest
core_sha256(): SHA-256 hex digest (C backend)

Schema validation & synthetic data

Validate a data frame against a declared schema, and build typed synthetic columns/CSVs for tests and documentation.

apply_schema_validation(): Apply Schema Validation, Stopping on Fatal Issues
validate_schema(): Validate a Data Frame Against a Provenance Schema
make_synthetic_column(): Generate One Synthetic Column From a Spec
make_synthetic_csv(): Generate a Synthetic CSV From a Schema Recipe

Text & encoding utilities

ASCII transliteration and safe text fallbacks for non-UTF-8 sinks.

to_ascii(): Transliterate Text to Plain ASCII
ascii_fallback(): Use Text As-Is, Falling Back to ASCII When It Cannot Be Represented
write_text_fallback(): Write Text as UTF-8, Falling Back to ASCII on an Encoding Error

Core numeric primitives

Thin R wrappers over the shared compiled core that the whole ecosystem links against via LinkingTo: rmoriebricklayer, so every sibling package computes with one copy of the arithmetic.

core_mean() core_var() core_cor(): Fast summary statistics (C backend)
core_normal_pdf(): Normal density (C backend)
core_normal_logpdf(): Normal log-density (C backend)
core_sd() core_dist(): Standard deviation and Euclidean distance (C backend)
core_moments(): Mean, variance, skewness and kurtosis in one pass (C backend)
core_quantile() core_median() core_mad() core_iqr() core_tukey_fences(): Quantiles, median and robust spread (C backend)
core_trimmed_mean() core_winsorized_mean(): Trimmed and winsorized means (C backend)
core_weighted(): Weighted mean and variance (C backend)
core_cor_spearman() core_midranks(): Rank correlation and midranks (C backend)
core_cov(): Covariance matrix of a numeric matrix (C backend)
core_bootstrap_mean(): Bootstrap replicate means (C backend)
core_ipw_weights(): Trimmed inverse-probability weights (C backend)
core_gamma_cdf(): Regularized incomplete gamma function (C backend)
core_hawkes_nll(): Hawkes-process negative log-likelihood (C backend)

Distributional drift

Whether a freshly fetched column is still the column the capsule was pinned against. A byte digest cannot answer this: a re-release can be statistically identical yet differ byte-for-byte, and a column can keep its name and type while having been silently rescaled.

capsule_drift(): Compare a fetched data frame with the one a capsule was pinned against
drift_ks(): Two-sample Kolmogorov-Smirnov test (C backend)
drift_psi(): Population stability index and Jensen-Shannon divergence (C backend)
drift_chisq(): Chi-square test for a categorical column
drift_homogeneity(): Chi-square test of homogeneity for two categorical samples
benford_test(): Benford first-digit test

Signed provenance

Authenticated rather than merely checksummed. Keyed digests for a shared secret, post-quantum hash-based signatures for a public verifier, Merkle trees to pin a capsule chunk by chunk, and hash chains to make the run history itself tamper-evident.

capsule_attest() capsule_check_attestation(): Attest a capsule, binding a signature to what it covers
capsule_bundle() capsule_bundle_read() capsule_bundle_verify(): Bundle a capsule into one signed artifact
capsule_falsify(): Run falsification controls against a statistic
capsule_power(): Detect an injected effect of known size
prereg_declare() prereg_check(): Declare an analysis before running it
falsify_family(): Correct a family of falsification results for multiple testing
evalue_rr(): How strong would an unmeasured confounder have to be
timestamp_verify() timestamp_info(): Verify an RFC 3161 timestamp token
cert_parse(): Parse an X.509 certificate
cert_chain_verify(): Verify a certificate chain
revocation_fetch(): Fetch revocation data for a certificate path
capsule_sign(): Sign a capsule manifest
capsule_verify(): Verify a capsule manifest signature
pqc_keygen(): Generate a post-quantum signing key for capsule provenance
signing_public_key(): Public half of a signing key
fips_keygen() fips_public_key(): Generate a standardised post-quantum signing key
fips_key(): Assemble a standardised key from raw key material
fips_mu() fips_sign_mu() fips_verify_mu(): Sign an ML-DSA message digest computed elsewhere
fips_sizes(): Byte lengths of a standardised signature scheme
kem_keygen() kem_public_key(): Generate an ML-KEM key pair
kem_sizes(): Byte lengths of an ML-KEM parameter set
kem_encapsulate(): Encapsulate a shared secret under an ML-KEM key
kem_decapsulate(): Recover a shared secret from an ML-KEM ciphertext
oqs_keygen() oqs_public_key(): Generate a standardised post-quantum signing key (deprecated name)
pqc_backends(): Available post-quantum signature schemes
chain_new() chain_append() chain_head() chain_seal() chain_verify(): Tamper-evident chain of capsule manifests
merkle_root() merkle_leaves() merkle_proof() merkle_verify(): Merkle tree over capsule chunks (C backend)
chunk_file(): Split a file into fixed-size chunks

Digests and keys

The hashes and key-derivation the provenance layer is built on, each verified against its published test vectors.

core_sha512(): SHA-512 hex digest (C backend)
core_crc32(): CRC-32 checksum (C backend)
core_blake2b(): BLAKE2b digest, optionally keyed
core_hmac_sha256() core_digest_equal(): Keyed digest and constant-time comparison (C backend)
sha512_file() crc32_file(): SHA-512 and CRC-32 of a file
digest_object(): Digest an arbitrary R object
random_bytes(): Cryptographically strong random bytes
derive_key(): Derive a key from a passphrase

Schema and rules

Derive a pinnable schema from data you already trust, then declare the project-specific checks a generic schema cannot express.

infer_schema(): Infer a pinnable schema from a data frame
rule(): Declare a validation rule
validate_rules(): Apply declared rules to a data frame
rule_in_set() rule_between() rule_not_null() rule_unique() rule_regex() rule_increasing() rule_within_n_mads() rule_complete_rows() rule_distinct_rows() rule_col_count(): Ready-made validation rules

Period-over-period change

Year-over-year change matched on the period’s own value rather than on row order, so a missing year is a gap instead of a quietly multi-year comparison. A percent off a small base is withheld with its reason; a percentage is reported in percentage points; a ratio of counts carries the exact conditional-binomial interval. The same object renders to HTML, PDF, CSV, TSV, JSON or Markdown.

yoy() print(<rmbl_yoy>): Year-over-year (and period-over-period) change
yoy_summary(): Summarise a change table
yoy_label(): Label a period without changing it
fiscal_year_label(): Name a fiscal year by the years it spans
yoy_html() yoy_pdf(): Render a change table to HTML or PDF
yoy_write(): Write a change table to a file, in whatever format the name implies
yoy_csv() yoy_tsv() yoy_json() yoy_markdown(): Write a change table as delimited text, JSON or Markdown
yoy_palettes(): Colour palettes for change tables

Stock and flow over time

Person-days are one quantity with two denominators. Divided by TIME they give the average daily population, a stock: how many are held at once. Divided by PEOPLE they give the average length of stay. When stay length moves the two can carry opposite signs, so a report quoting one implies the wrong direction for the other. After Lakner (1976), A Manual of Statistical Sampling Methods for Corrections Planners.

adp(): Average daily population
alos(): Average length of stay
admissions(): Admissions implied by a population and a length of stay
adp_from_counts(): Person-days estimated from periodic headcounts
stock_flow(): Stock and flow side by side, with the decomposition
stay_summary(): Length of stay with its distribution and interval
period_days(): Period length in days, from dates

Rates, shares and rate change

A count is not comparable across places of different size or years of different population, and a share of a total is not a rate per population – labelling one as the other is the most common error in a published table, so they are separate functions with separate intervals. Rates carry the exact Poisson interval, shares carry Wilson-s, and the change in a rate conditions on the two counts and corrects for the exposure ratio rather than treating two rates as measured numbers.

rate(): Event rates per unit of population
share(): Share of a total, in percent
rate_change(): Change in a rate between periods

Banded categories

Published categories are intervals – “2 to 5”, “50+” – and anything computed from them rests on where inside each band the mass sits and on where the open top band ends. These parse to bounds, make the representative-value rule explicit, and measure how far a result moves as the open band’s assumed cap varies.

parse_bands(): Parse banded category labels into numeric bounds
band_values(): Representative values for banded categories
expand_bands(): Expand a banded frequency table into per-unit values
band_sensitivity() print(<rmbl_band_sensitivity>): How much a result depends on the open band's assumed cap

Concentration and association

Whether a few units account for most of a total, and how strongly two categorical variables move together. Gini is twice the area under its own Lorenz curve, and has a ceiling of 1 - 1/n rather than 1. The tail index maximises the exact discrete likelihood, the closed-form continuity correction being badly biased at the small thresholds administrative counts start from.

gini() lorenz() top_share(): Concentration of a total across units
hill_tail_index(): Tail index of a heavy-tailed count
hurwitz_zeta(): Hurwitz zeta function
cramers_v(): Association in a contingency table

Trend in a short series

An annual extract is five to ten points, which supports a rank test and a resistant slope but not a model with an autocorrelation structure. The step test takes its p-value from the permutation distribution of the maximum over splits, not from the best split’s own test.

trend_test(): Trend in a short series
step_change(): A single step change, with the scan's own null distribution
count_trend(): Trend in a count series, as a rate ratio per period

Region-coded counts

A region is an areal unit with a population, not a coordinate. Indirect standardisation removes the part of a difference explained by who the area holds; empirical Bayes stops a small area’s noise from ranking it to the top or bottom; a funnel plot shows the reader the same thing directly.

expected_counts(): Expected counts under indirect standardisation
sir(): Standardised incidence ratio, with an exact interval
eb_rates(): Empirical Bayes rates, shrunk toward the overall experience
funnel_limits(): Funnel-plot control limits
morans_i(): Global Moran's I over a neighbour list

Points, and the regions that contain them

A region map from point locations to statistical regions is built once and read many times, so an error in it reproduces downstream instead of showing up there. These check it three ways: that it is internally sound, that a recomputation agrees with it cell by cell, and that an assignment derived a DIFFERENT way agrees too, which is the only one of the three that can catch an error in the original method. The coverage summary reports what share of the population lives in a region holding a unit, and says each time that the share is context rather than a rate denominator: units serve catchments, which a point location does not state.

region_coverage(): Population of the regions that contain a unit, and of those that do not
region_map_integrity(): Internal soundness of a region map
region_map_compare(): Compare a recomputed region map against a published one
region_map_second_route(): Check a region map against an independently derived assignment
region_map_from_points(): Recompute a region map by point in polygon

Describe a capsule

The summaries worth reading before trusting a capsule: what every column looks like, where the gaps are, which values dominate, and which rows do not belong. capsule_report() runs the lot and orders the findings by severity.

capsule_report(): Assess a capsule in one call
report_markdown(): Write a capsule report as Markdown
profile_columns(): Column report for a data frame
inline_hist(): Inline histogram for a numeric vector
frequency_table(): Frequency table for one column
correlation_table(): Full pairwise correlation table
top_correlations(): Strongest pairwise correlations in a data frame
mahalanobis_outliers(): Multivariate outliers by Mahalanobis distance
duplicate_rows(): Duplicated rows, with their groups
drop_empty() drop_constant(): Drop empty or constant columns and rows
clean_column_names(): Tidy the column names of an ingested table
environment_diff(): Compare two captured environments

Missingness

Not just how much is missing, but the shape of it – which columns go together, whether the gaps are contiguous, and whether the missingness is related to the data (in which case dropping incomplete rows is biased).

missingness_summary(): Missingness in one line per question
missingness_pattern(): Which columns are missing together
missing_runs(): Runs of consecutive missing values
missingness_map(): Text map of where the missing values are
mcar_test(): Little's test for data missing completely at random

Large capsules

One-pass algorithms for members bigger than memory: exact block-wise moments, uniform sampling of a stream, and fixed-memory distinct counts.

online_summary() summary_update() summary_merge() summary_stats(): Exact summary statistics accumulated in blocks
reservoir_indices() reservoir_sample(): Uniform sample of a stream in one pass
distinct_sketch() distinct_count() sketch_merge(): Distinct-value count in fixed memory

Output

Printed reports for the objects the package returns.

print(<bricklayer_attestation>) print(<bricklayer_attestation_check>) print(<bricklayer_bundle>) print(<bricklayer_bundle_check>) print(<bricklayer_chain>) print(<bricklayer_falsification>) print(<bricklayer_kem_key>) print(<bricklayer_kem_public_key>) print(<bricklayer_kem_capsule>) print(<bricklayer_mcar>) print(<bricklayer_power>) print(<bricklayer_prereg>) print(<bricklayer_prereg_check>) print(<bricklayer_falsify_family>) print(<bricklayer_drift>) summary(<bricklayer_drift>) print(<bricklayer_benford>) print(<bricklayer_signing_key>) print(<bricklayer_public_key>) print(<bricklayer_signature>) summary(<bricklayer_benford>) print(<bricklayer_env_diff>) print(<bricklayer_report>) summary(<bricklayer_report>) print(<bricklayer_recompute>) print(<bricklayer_schema>) print(<bricklayer_oqs_key>) print(<bricklayer_oqs_public_key>) print(<bricklayer_timestamp>) print(<bricklayer_certificate>) print(<bricklayer_certpath_check>): Printed reports for bricklayer objects