Mini-batch gradient descent (Geron Ch 4, morie.fn hmmbgd) — morie_geron_minibatch

b = m is batch GD, b = 1 is SGD. IMPORTANT: the Python module draws its shuffle from numpy.random.default_rng (PCG64), which R cannot reproduce, so this port takes the batch order explicitly. Pass order as a 0-based permutation of 0..m-1 (the parity tests read the permutation out of Python and pass it in); with order = NULL the identity order is used. This is the one honest divergence in this file.

Usage

morie_geron_minibatch_gd(X, y, theta, eta, b, order = NULL, n_steps = 1)

Arguments

X, y, theta: Design, targets, coefficients.
eta: Positive rate.
b: Batch size in 1..m.
order: Optional 0-based permutation; NULL means identity.
n_steps: Number of updates.

Value

List with theta, gradient, full_gradient, batch_indices (0-based), mse, batch_size, estimate, n.

Examples

set.seed(4)
X <- cbind(1, rnorm(30))
y <- X %*% c(1, 2) + rnorm(30, 0, 0.2)
r <- morie_geron_minibatch_gd(X, as.numeric(y), theta = c(0, 0),
                              eta = 0.05, b = 10, n_steps = 20)
str(r, max.level = 1)
#> List of 9
#>  $ theta        : num [1:2] 1.13 1.69
#>  $ gradient     : num [1:2] 0.246 -0.138
#>  $ full_gradient: num [1:2] -0.0215 -0.4199
#>  $ batch_indices: int [1:10] 10 11 12 13 14 15 16 17 18 19
#>  $ mse          : num 0.101
#>  $ batch_size   : int 10
#>  $ estimate     : num 0.101
#>  $ n            : int 30
#>  $ method       : chr "Mini-batch gradient descent step"