Gradient boosting residual stage (Geron Ch 7, morie.fn grgbm)

r = y - F_prev; a best-SSE decision stump (midpoint cuts over the sorted unique values of each feature, first minimum wins) is fitted to r unless learner is supplied; F_new = F_prev + nu * h. The stump's feature index is 0-based.

Usage

morie_geron_gradient_boosting_residual(
  X,
  y,
  F_prev,
  learner = NULL,
  learning_rate = 1
)

Arguments

X

(m, n) design.

y

Targets.

F_prev

Current ensemble prediction (scalar or length m).

learner

Optional function(X, r) returning m predictions.

learning_rate

Shrinkage nu in (0, 1].

Value

List with residuals, h_prediction, F_new, mse_before, mse_after, stump.

Examples

X <- matrix(c(1, 2, 1, 3, 1, 5, 1, 7), 4, byrow = TRUE)
y <- c(2, 3, 6, 9)
morie_geron_minibatch_gradient_descent(X, y, c(0, 0), 0.02, 2,
    5, seed = 3)
#> $theta
#> [1] 0.1321851 1.2569443
#> 
#> $cost_history
#> [1] 32.5000000 10.9993000  4.2661237  0.7877623  0.9527690  0.3528633
#> 
#> $theta_history
#> $theta_history[[1]]
#> [1] 0 0
#> 
#> $theta_history[[2]]
#> [1] 0.30 1.86
#> 
#> $theta_history[[3]]
#> [1] 0.2020 1.6064
#> 
#> $theta_history[[4]]
#> [1] 0.108384 1.040448
#> 
#> $theta_history[[5]]
#> [1] 0.1000038 1.0190931
#> 
#> $theta_history[[6]]
#> [1] 0.1321851 1.2569443
#> 
#> 
#> $initial_cost
#> [1] 32.5
#> 
#> $final_cost
#> [1] 0.3528633
#> 
#> $eta
#> [1] 0.02
#> 
#> $batch_size
#> [1] 2
#> 
#> $estimate
#> [1] 0.1321851 1.2569443
#> 
#> $n
#> [1] 4
#> 
morie_geron_gradient_boosting_residual(X = X, y = y, F_prev = y)
#> $residuals
#> [1] 0 0 0 0
#> 
#> $h_prediction
#> [1] 0 0 0 0
#> 
#> $F_new
#> [1] 2 3 6 9
#> 
#> $mse_before
#> [1] 0
#> 
#> $mse_after
#> [1] 0
#> 
#> $stump
#> $stump$sse
#> [1] 0
#> 
#> $stump$feature
#> [1] 1
#> 
#> $stump$threshold
#> [1] 2.5
#> 
#> $stump$left_value
#> [1] 0
#> 
#> $stump$right_value
#> [1] 0
#> 
#> 
#> $learning_rate
#> [1] 1
#> 
#> $estimate
#> [1] 2 3 6 9
#> 
#> $n
#> [1] 4
#>