Row-level sanity check on a parsed SIU table (regex-only, no LLM)
Source:R/siu.R
morie_siu_sanity_check.RdFor every row in a parser-emitted SIU table, flag cells that
don't match the expected format for their column – case_number
that doesn't look like an SIU case id, date_*_iso that isn't a
valid ISO 8601 date, number_of_* that isn't a positive integer,
charges_recommended that isn't "Yes" / "No", etc. Returns a
data frame ranked by issue count so the most-broken rows surface
at the top for manual inspection against the cached HTML.
Value
A data frame with one row per source row, columns:
case_number, drid, issues_count (integer
number of suspicious cells), issues (semicolon-separated
string of field:reason pairs). Ordered descending by
issues_count.
Details
Designed to be a fast first-pass quality filter – runs in
milliseconds, no network, no LLM, no API key. Doesn't try to
verify correctness against the underlying report (that's what
morie_siu_audit_columns() is for); just checks that each
value MATCHES THE EXPECTED FORMAT for its field. A clean sanity
check is necessary but not sufficient for correctness.
Examples
# \donttest{
# Corpus-first fetch is fast when rmoriedata is installed.
csv <- morie_fetch_siu(cache_dir = file.path(tempdir(), "siu_sanity"))
#> SIU: wrote the panel-reviewed corpus from rmoriedata (4613 rows) to /tmp/RtmpNbJQ2u/siu_sanity/SIU.csv. For a live re-fetch set options(morie.siu.allow_fetch = TRUE).
df <- utils::read.csv(csv, colClasses = "character")
sanity <- morie_siu_sanity_check(utils::head(df, 50))
head(sanity, 10) # worst 10 rows -- inspect against HTML
#> case_number drid issues_count issues
#> 1 17-OCI-340 100 0
#> 2 17-OCI-340 101 0
#> 3 17-TCD-350 102 0
#> 4 17-TCD-350 103 0
#> 5 17-OCD-317 104 0
#> 6 17-OCD-317 105 0
#> 7 18-OCD-033 106 0
#> 8 18-OCD-033 107 0
#> 9 18-OFD-030 108 0
#> 10 18-OFD-030 109 0
table(sanity$issues_count)
#>
#> 0
#> 50
# }