Skip to contents

Dataset catalogue

Source: vignettes/dataset-catalog.Rmd
dataset-catalog.Rmd

Overview

rmorie reaches datasets through three routes:

This vignette shows how to discover and load them from R.

Browsing the catalogue 

library(rmorie)
catalog <- morie_dataset_catalog()
dim(catalog)
#> [1] 71 20
head(catalog[, c("key", "name", "source", "survey", "year")])
#>       key                      name source survey      year
#> 1   ocp21      CPADS 2021-2022 PUMF     oc  cpads 2021-2022
#> 2   occ22        CCS 2018-2022 PUMF     oc    ccs 2018-2022
#> 3   occ23             CCS 2023 PUMF     oc    ccs      2023
#> 4   occ24             CCS 2024 PUMF     oc    ccs      2024
#> 5 ocs22mf      CSADS 2021-2022 PUMF     oc  csads 2021-2022
#> 6 ocs22bt CSADS 2021-2022 Bootstrap     oc  csads 2021-2022

Each row describes one dataset: its key (for example ocp21 for the CPADS 2021–2022 PUMF, or otisb01 for OTIS table b01), its name, source, survey, year, and where it is fetched from (ckan_resource_id, download_url, arcgis_url).

Per-dataset detail 

str(morie_dataset_info("ocp21"))
#> List of 20
#>  $ key             : chr "ocp21"
#>  $ name            : chr "CPADS 2021-2022 PUMF"
#>  $ source          : chr "oc"
#>  $ survey          : chr "cpads"
#>  $ year            : chr "2021-2022"
#>  $ format          : chr "csv"
#>  $ type            : chr "pumf"
#>  $ large_file      : logi FALSE
#>  $ local_path      : chr "data/datasets/oc/CPADS/2021-2022/cpads-2021-2022-pumf2.csv"
#>  $ table_name      : chr "ocp21"
#>  $ ckan_resource_id: chr "d2639429-c304-45a6-90b3-770562f4d46d"
#>  $ download_url    : chr ""
#>  $ zip_member      : chr ""
#>  $ hosted_key      : chr ""
#>  $ hosted_file     : chr ""
#>  $ ckan_portal     : chr ""
#>  $ rmoriedata      : chr ""
#>  $ arcgis_url      : chr ""
#>  $ fetcher         : chr ""
#>  $ fetcher_args    : chr ""

morie_dataset_info() returns the catalogue row for one key as a named list.

Loading data 

df <- morie_load_dataset("ocp21")
dim(df)
# 40931   394

This chunk is not run when the vignette is built: on the first call morie_load_dataset() downloads the PUMF from open.canada.ca. The copy is then kept in a per-user cache under morie_cache_dir() (set MORIE_CACHE_DIR to move it), and refresh = TRUE re-downloads it.

Bundled tables

The tables in rmoriedata need no network access:

bundled <- rmoriedata::morie_data_catalog()
nrow(bundled)
#> [1] 107
head(bundled[, c("slug", "kind", "n_rows", "n_cols")])
#>                                                          slug       kind n_rows
#> 1                                  arsau_2020_2022_dictionary dictionary     NA
#> 2 arsau_2020_2022_useofforce_agrregatesummarybyyear_2020_2022      table      5
#> 3        arsau_2020_2022_useofforce_detaileddataset_2020_2022      table      5
#> 4                                       arsau_2023_dictionary dictionary     NA
#> 5                           arsau_2023_uof_individual_records      table      5
#> 6                                 arsau_2023_uof_main_records      table      5
#>   n_cols
#> 1     NA
#> 2      6
#> 3    167
#> 4     NA
#> 5    112
#> 6     23

morie_sample() returns one of four MRM reference tables:

dim(morie_sample("otis_c11"))
#> [1] 33  4

Statistics Canada / Health Canada data acknowledgment

Several datasets in the catalogue are Statistics Canada and Health Canada PUMFs (CCS, CSADS, CPADS and others). The standard disclaimer applies: although the analyses use Statistics Canada / Health Canada data, the analyses, interpretations, and conclusions are those of the analyst and do not represent the views of either agency.

Where to go next