Stacking (blending): meta-learner combines outputs of base learners

Meta-learner trained on out-of-fold base predictions (K-fold cross-prediction) to avoid the leakage of in-sample base predictions.

Usage

morie_geron_stacking(X, y, base_models, meta_model = NULL, k_folds = 3)

Arguments

X: Design matrix (n, d).
y: Targets, length n.
base_models: List of function(Xtr, ytr, Xte) -> predictions.
meta_model: Optional blender function(Xtr, ytr, Xte); default OLS w/ intercept.
k_folds: Folds for out-of-fold predictions.

Value

list with predicted, meta_features, oof_mse, stacked_mse, best_base_mse, gain, n_base, estimate, n, method.

Examples

set.seed(6)
X <- matrix(rnorm(80), 40, 2)
y <- as.numeric(X[, 1] > 0)
base <- function(Xtr, ytr, Xte) {
  b <- qr.solve(cbind(1, as.matrix(Xtr)), as.numeric(ytr))
  as.numeric(cbind(1, as.matrix(Xte)) %*% b)
}
morie_geron_stacking(X, y, base_models = list(base, base), k_folds = 3)
#> $predicted
#>  [1]  0.51766185  0.23277892  0.75352636  1.04538985  0.51623933  0.58458330
#>  [7] -0.03633405  0.86783460  0.50982241  0.06234228  1.15086924  0.14080386
#> [13]  0.65782243  0.41748037  0.26609188  0.46621532  1.06923024  0.09528872
#> [19]  0.32860675  1.34203340  0.66595369 -0.05986254  1.20991016  0.06450378
#> [25]  0.49642980  0.05890386  0.47528816  1.25909012  0.98553840  0.33851757
#> [31]  0.18912185  0.66005211  0.12347586  0.04865400  0.91302862  0.42067658
#> [37]  0.65476110  1.16729019  0.46356115 -0.12318154
#> 
#> $meta_features
#>              [,1]        [,2]
#>  [1,]  0.53601193  0.53601193
#>  [2,]  0.22291829  0.22291829
#>  [3,]  0.79523307  0.79523307
#>  [4,]  1.11599852  1.11599852
#>  [5,]  0.53444854  0.53444854
#>  [6,]  0.60956032  0.60956032
#>  [7,] -0.07284376 -0.07284376
#>  [8,]  0.92086075  0.92086075
#>  [9,]  0.52739618  0.52739618
#> [10,]  0.03560404  0.03560404
#> [11,]  1.23192308  1.23192308
#> [12,]  0.12183533  0.12183533
#> [13,]  0.69005200  0.69005200
#> [14,]  0.42590991  0.42590991
#> [15,]  0.25953008  0.25953008
#> [16,]  0.47947087  0.47947087
#> [17,]  1.14219973  1.14219973
#> [18,]  0.07181303  0.07181303
#> [19,]  0.32823553  0.32823553
#> [20,]  1.44201739  1.44201739
#> [21,]  0.69898846  0.69898846
#> [22,] -0.09870218 -0.09870218
#> [23,]  1.29681057  1.29681057
#> [24,]  0.03797959  0.03797959
#> [25,]  0.51267736  0.51267736
#> [26,]  0.03182514  0.03182514
#> [27,]  0.48944216  0.48944216
#> [28,]  1.35086059  1.35086059
#> [29,]  1.05022025  1.05022025
#> [30,]  0.33912777  0.33912777
#> [31,]  0.17493804  0.17493804
#> [32,]  0.69250247  0.69250247
#> [33,]  0.10279142  0.10279142
#> [34,]  0.02056028  0.02056028
#> [35,]  0.97053014  0.97053014
#> [36,]  0.42942263  0.42942263
#> [37,]  0.68668751  0.68668751
#> [38,]  1.24997012  1.24997012
#> [39,]  0.47655386  0.47655386
#> [40,] -0.16829138 -0.16829138
#> 
#> $oof_mse
#> [1] 0.08961757 0.08961757
#> 
#> $stacked_mse
#> [1] 0.08766794
#> 
#> $best_base_mse
#> [1] 0.08961757
#> 
#> $gain
#> [1] 0.001949634
#> 
#> $n_base
#> [1] 2
#> 
#> $estimate
#> [1] 0.08766794
#> 
#> $n
#> [1] 40
#> 
#> $method
#> [1] "Stacking with K-fold out-of-fold meta-features and a least-squares blender by default"
#>