Early stopping on validation RMSE (Geron Ch 4, morie.fn greast)

Runs every one of n_iter batch-GD steps (gradient from the shared MSE core) and rolls back to the argmin validation RMSE, not the first uptick. best_iteration is 0-based.

Usage

morie_geron_early_stopping(
  X_train,
  y_train,
  X_val,
  y_val,
  n_iter,
  eta,
  theta0 = NULL
)

Arguments

X_train, y_train, X_val, y_val: Training and validation data.
n_iter: Number of gradient steps.
eta: Positive learning rate.
theta0: Optional starting parameters; default zeros.

Value

List with theta, best_iteration, best_val_rmse, final_val_rmse, val_rmse_history, train_rmse_history, overfitting_detected.

Examples

set.seed(1)
Xtr <- cbind(1, rnorm(40)); ytr <- Xtr %*% c(0.5, 1.2) + rnorm(40, 0, 0.2)
Xv <- cbind(1, rnorm(20));  yv <- Xv %*% c(0.5, 1.2) + rnorm(20, 0, 0.2)
morie_geron_early_stopping(Xtr, as.numeric(ytr), Xv, as.numeric(yv),
                           n_iter = 50, eta = 0.05)
#> $theta
#> [1] 0.5240425 1.2078343
#> 
#> $best_iteration
#> [1] 42
#> 
#> $best_val_rmse
#> [1] 0.1586443
#> 
#> $final_val_rmse
#> [1] 0.159651
#> 
#> $val_rmse_history
#>  [1] 1.2716016 1.1562765 1.0519922 0.9576836 0.8723947 0.7952674 0.7255317
#>  [8] 0.6624963 0.6055412 0.5541098 0.5077025 0.4658708 0.4282116 0.3943621
#> [15] 0.3639954 0.3368157 0.3125547 0.2909675 0.2718293 0.2549321 0.2400820
#> [22] 0.2270973 0.2158060 0.2060451 0.1976600 0.1905039 0.1844385 0.1793340
#> [29] 0.1750700 0.1715356 0.1686302 0.1662630 0.1643532 0.1628299 0.1616307
#> [36] 0.1607019 0.1599973 0.1594773 0.1591085 0.1588626 0.1587160 0.1586487
#> [43] 0.1586443 0.1586892 0.1587720 0.1588835 0.1590161 0.1591637 0.1593212
#> [50] 0.1594847 0.1596510
#> 
#> $train_rmse_history
#>  [1] 1.2759165 1.1638799 1.0626346 0.9711366 0.8884489 0.8137301 0.7462247
#>  [8] 0.6852540 0.6302082 0.5805389 0.5357526 0.4954050 0.4590950 0.4264602
#> [15] 0.3971723 0.3709329 0.3474701 0.3265351 0.3078994 0.2913523 0.2766990
#> [22] 0.2637591 0.2523647 0.2423602 0.2336011 0.2259538 0.2192951 0.2135118
#> [29] 0.2085007 0.2041681 0.2004295 0.1972091 0.1944394 0.1920605 0.1900197
#> [36] 0.1882705 0.1867727 0.1854908 0.1843944 0.1834571 0.1826560 0.1819716
#> [43] 0.1813869 0.1808875 0.1804609 0.1800966 0.1797854 0.1795196 0.1792926
#> [50] 0.1790987 0.1789331
#> 
#> $overfitting_detected
#> [1] TRUE
#> 
#> $estimate
#> [1] 0.5240425 1.2078343
#> 
#> $n
#> [1] 40
#>