Per-example gradients are clipped to L2 norm C, summed, Gaussian
noise of scale \(\sigma C\) added to the SUM, and the result divided
by the batch size.
Usage
morie_dp_sgd(grads, C = 1, sigma = 1, lr = 0.1, theta = NULL, seed = NULL)
Arguments
- grads
per-example gradient matrix, shape (B, p).
- C
L2 clipping norm.
- sigma
noise multiplier; the noise sd on the sum is
sigma * C.
- lr
learning rate.
- theta
optional current parameters; the updated vector is
returned when supplied.
- seed
optional integer seed.
Value
list with update, private_gradient,
clipped_fraction, noise_sd, and theta when
given.
Details
The per-example clipping is not a detail. Clipping an averaged
gradient provides NO privacy: the average has already mixed every
record together, and bounding its norm says nothing about how much
any one record moved it. This function requires a (B, p) matrix and
errors on a vector, because accepting one would mean returning an
epsilon the mechanism does not satisfy.
References
Abadi, M. et al. (2016). Deep learning with differential
privacy. CCS, 308-318.
