The wild cluster bootstrap for a difference in differences with few clusters
Source:R/panel_causal.R
wcb.RdA p-value and confidence interval for the treatment effect of
plndid that stay accurate when there are few clusters (a policy
assigned to a handful of districts or states), where the cluster-robust t
test over-rejects.
Arguments
- data
A data frame, one row per unit and period.
- unit, time, y
Column names: the unit, the period (numeric, for example a year or a month index) and the outcome.
- trtd
Column name of the treatment indicator (0 or 1).
- covs
Optional column names of time-varying controls.
- cl
Column name of the clusters (default the unit), or two column names for two-way clustering (Cameron, Gelbach and Miller 2011): \(V_a + V_b - V_{ab}\), each term with the small-sample factor of the dimension with fewer clusters, and t on its \(G - 1\) degrees of freedom. A composite cluster (district by month, say) is one column made with
interaction(). Fewer than ten clusters give a warning (CR1 errors are unreliable with so few;wcbis made for that case), and so does a cluster that changes within a unit.- reps
Bootstrap replications (default 9999).
- wtype
"rademacher"(default) or"webb".- seed
Optional seed for the bootstrap weights.
- ties
"count"(default) or"strict": whether a bootstrap \(|t^*|\) equal to \(|t|\) counts against the null.- conflvl
Confidence level of the intervals reported (default 0.95).
Value
A data frame of class rmbl_wild with the columns
estimate, t, p_value, lower, upper,
clusters, draws and enumerated.
Details
This is the restricted wild cluster bootstrap (WCR) of Cameron,
Gelbach and Miller (2008). The model is fitted with the null \(\beta =
\beta_0\) imposed; each bootstrap outcome is the restricted fit plus its
residuals multiplied, cluster by cluster, by a random sign (Rademacher) or by
one of Webb's six values; the cluster-robust t statistic is recomputed each
time. The p-value is the share of bootstrap \(|t^*|\) at least \(|t|\).
With ties = "count" (the default) a draw that equals \(|t|\) counts:
the sign patterns that reproduce the data are among the draws, as the
observed assignment is in a permutation test, and the test is exact at its
level. With "strict" only draws above \(|t|\) count, as
fwildclusterboot does; under full enumeration this gives a smaller
p-value (the two sign patterns that reproduce \(|t|\) are left out). With
Rademacher weights and \(2^G \le\) reps every sign pattern is
enumerated (as fwildclusterboot does); with very few clusters (under
about 12) Webb's weights, which have more distinct values, are better. The
interval collects the \(\beta_0\) not rejected at \(1 -\) conflvl,
found by bisection with the same draws; it is unbounded when no value can be
rejected (with Rademacher weights and \(G\) clusters the smallest
attainable p-value is \(2/2^G\), so four clusters can never reject at 5%:
that is the honest answer, not a failure of the method).
References
Cameron AC, Gelbach JB, Miller DL (2008). Bootstrap-based improvements for inference with clustered errors. Review of Economics and Statistics 90(3):414-427. doi:10.1162/rest.90.3.414
Webb MD (2023). Reworking wild bootstrap-based inference for clustered errors. Canadian Journal of Economics 56(3):839-858. doi:10.1111/caje.12661
Roodman D, Nielsen MO, MacKinnon JG, Webb MD (2019). Fast and wild: bootstrap inference in Stata using boottest. The Stata Journal 19(1):4-60. doi:10.1177/1536867X19830877
Examples
p <- polex()
# the reform was assigned to four regions: four clusters
wcb(p, "district", "month", "crmrate", "reformed",
cl = "region", wtype = "webb", reps = 99,
seed = 1)
#> ╭── Wild cluster bootstrap ───────────────────────────────────────────────╮
#> │ Method wild cluster bootstrap-t, null imposed │
#> │ Clusters 4 │
#> │ Draws 99 │
#> │ Weights random draws │
#> │ │
#> │ estimate interval t p_value │
#> │ -------- ---------------- ------ ------- │
#> │ -3.232 [-3.564, -2.910] -19.52 0.010 │
#> │ │
#> │ The treatment's effect is -3.232 (its interval [-3.564, -2.910]): the │
#> │ interval excludes 0, so the data distinguish it from no effect. │
#> ╰─────────────────────────────────────────────────────────────────────────╯
# your own panel: eight towns over ten years, four adopting a by-law (two
# in 2019, two in 2021), with each town's population, county and income
own <- data.frame(
town = rep(c("A", "B", "C", "D", "E", "F", "G", "H"), each = 10),
year = rep(2015:2024, times = 8)
)
i <- match(own$town, unique(own$town))
own$first <- c(2019, 2019, 2021, 2021, NA, NA, NA, NA)[i]
own$bylaw <- as.integer(!is.na(own$first) & own$year >= own$first)
own$pop <- c(12, 30, 8, 22, 15, 40, 9, 18)[i] * 1000
own$county <- rep(c("North", "South"), 4)[i]
own$income <- 50 + own$year - 2015 + cos(seq_len(80))
own$rate <- 30 + i + 0.5 * (own$year - 2015) - 4 * own$bylaw +
0.2 * own$income + sin(seq_len(80))
# eight towns: every one of the 2^8 sign patterns
wcb(own, "town", "year", "rate", "bylaw", covs = "income")
#> ╭── Wild cluster bootstrap ───────────────────────────────────────────────╮
#> │ Method wild cluster bootstrap-t, null imposed │
#> │ Clusters 8 │
#> │ Draws 256 │
#> │ Weights every sign pattern (exact) │
#> │ │
#> │ estimate interval t p_value │
#> │ -------- ---------------- ------ ------- │
#> │ -4.022 [-4.482, -3.521] -18.70 0.008 │
#> │ │
#> │ The treatment's effect is -4.022 (its interval [-4.482, -3.521]): the │
#> │ interval excludes 0, so the data distinguish it from no effect. │
#> ╰─────────────────────────────────────────────────────────────────────────╯