Batch gradient descent with full loss curve (Geron Ch 4 Eq 4-6, grbgd)

loss_history has n_iter + 1 entries: the initial MSE then one per step. eta_max_stable is 2 / lambda_max of (2/m) X^T X.

Usage

morie_geron_batch_gradient_descent(X, y, theta, eta, n_iter)

Arguments

X, y, theta

Design, targets, starting parameters.

eta

Positive learning rate.

n_iter

Steps, >= 1.

Value

List with theta, theta_path, loss_history, gradient, eta_max_stable, converged.

Examples

morie_geron_batch_gradient_descent(matrix(c(1, 2), ncol = 1),
    c(2, 4), 0, eta = 0.1, n_iter = 3)
#> $theta
#> [1] 1.75
#> 
#> $theta_path
#> $theta_path[[1]]
#> [1] 0
#> 
#> $theta_path[[2]]
#> [1] 1
#> 
#> $theta_path[[3]]
#> [1] 1.5
#> 
#> $theta_path[[4]]
#> [1] 1.75
#> 
#> 
#> $loss_history
#> [1] 10.00000  2.50000  0.62500  0.15625
#> 
#> $gradient
#> [1] -2.5
#> 
#> $eta
#> [1] 0.1
#> 
#> $eta_max_stable
#> [1] 0.4
#> 
#> $converged
#> [1] TRUE
#> 
#> $estimate
#> [1] 0.15625
#> 
#> $n
#> [1] 2
#> 
#> $method
#> [1] "Batch gradient descent (linear regression MSE)"
#>