Differentially private logistic regression — morie_dp

Objective perturbation (a random linear term added to the penalised objective) or output perturbation (noise on the fitted coefficients).

Usage

morie_dp_logistic(
  X,
  y,
  epsilon = 1,
  method = c("objective", "output"),
  lam = 0.01,
  C = 1,
  n_iter = 100,
  lr = 0.1,
  seed = NULL
)

Arguments

X: design matrix.
y: 0/1 response.
epsilon: privacy budget.
method: "objective" or "output".
lam: L2 regularisation; must be positive for objective perturbation.
C: row-norm clipping bound.
n_iter, lr: gradient-descent controls.
seed: optional integer seed.

Value

list with beta, prob, accuracy, clipped_fraction.

Details

Objective perturbation is the better mechanism at equal epsilon: the noise enters before the optimisation, so the fit adapts to it instead of carrying it. Note what the regulariser buys – strong convexity is what BOUNDS the sensitivity in the first place, so lam is not a tuning knob here but part of the guarantee, and lam = 0 has no guarantee at all.

References

Chaudhuri, K., Monteleoni, C. and Sarwate, A. D. (2011). Differentially private empirical risk minimization. JMLR, 12, 1069-1109.

Examples

set.seed(1)
X <- matrix(rnorm(400), ncol = 2)
y <- as.numeric(runif(200) < 1 / (1 + exp(-(X %*% c(1, -1)))))
morie_dp_logistic(X, y, epsilon = 4)$accuracy > 0.4
#> [1] TRUE