Vanilla batch gradient descent for OLS (R parity)

theta := theta - lr * (2/n) X' (X theta - y), intercept included. Validates against stats::lm reference.

Usage

morie_gradient_descent_vanilla(x, y, lr = 0.01, n_iter = 1000, tol = 1e-08)

Arguments

x

Numeric matrix / vector of predictors.

y

Numeric response vector.

lr

Learning rate.

n_iter

Max iterations.

tol

L2 step-norm tolerance for early stopping.

Value

Named list with estimate, reference_ols, n_iter, loss, n, method.

Examples

set.seed(1)
morie_gradient_descent_vanilla(x = rnorm(50), y = rnorm(50))
#> $estimate
#> [1]  0.12190151 -0.04554779
#> 
#> $reference_ols
#> [1]  0.1219017 -0.0455485
#> 
#> $n_iter
#> [1] 873
#> 
#> $loss
#> [1] 0.9184496
#> 
#> $n
#> [1] 50
#> 
#> $method
#> [1] "Vanilla batch gradient descent (linear regression)"
#>