Linear regression by SGD, cross-checked against normal equations (Geron Ch 10, morie.fn hmlrpt)

Shuffle order via the shared LCG stream, not numpy PCG64. With batch_size == nrow(X) (the default) descent is full-batch and shuffle-independent, so the anchor is exact.

Usage

morie_geron_linreg_pytorch(
  X,
  y,
  epochs = 100,
  lr = 0.01,
  batch_size = NULL,
  seed = 0
)

Arguments

X, y: Data.
epochs: Passes.
lr: Learning rate.
batch_size: Mini-batch size.
seed: Shuffle seed.

Value

List with w, b, loss_history, final_loss, w_closed_form, b_closed_form, gap, lr_limit, estimate, n, method.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
morie_geron_linreg_pytorch(V, V)
#> $w
#> [1] 0.9794452
#> 
#> $b
#> [1] 0.1155652
#> 
#> $loss_history
#>   [1] 5.733750000 1.292854170 0.295086585 0.070882567 0.020474390 0.009112996
#>   [7] 0.006524483 0.005907249 0.005733183 0.005658965 0.005607461 0.005561336
#>  [13] 0.005516696 0.005472662 0.005429035 0.005385769 0.005342851 0.005300275
#>  [19] 0.005258038 0.005216138 0.005174572 0.005133337 0.005092431 0.005051851
#>  [25] 0.005011594 0.004971658 0.004932040 0.004892738 0.004853749 0.004815071
#>  [31] 0.004776701 0.004738637 0.004700876 0.004663416 0.004626254 0.004589389
#>  [37] 0.004552817 0.004516537 0.004480546 0.004444841 0.004409422 0.004374284
#>  [43] 0.004339427 0.004304847 0.004270543 0.004236512 0.004202752 0.004169262
#>  [49] 0.004136038 0.004103079 0.004070383 0.004037947 0.004005769 0.003973848
#>  [55] 0.003942182 0.003910768 0.003879604 0.003848688 0.003818019 0.003787594
#>  [61] 0.003757412 0.003727470 0.003697767 0.003668300 0.003639069 0.003610070
#>  [67] 0.003581302 0.003552764 0.003524453 0.003496367 0.003468506 0.003440866
#>  [73] 0.003413447 0.003386246 0.003359262 0.003332493 0.003305937 0.003279593
#>  [79] 0.003253459 0.003227533 0.003201813 0.003176299 0.003150988 0.003125878
#>  [85] 0.003100969 0.003076258 0.003051744 0.003027426 0.003003301 0.002979369
#>  [91] 0.002955627 0.002932074 0.002908709 0.002885531 0.002862537 0.002839726
#>  [97] 0.002817097 0.002794648 0.002772378 0.002750286
#> 
#> $final_loss
#> [1] 0.002750286
#> 
#> $w_closed_form
#> [1] 1
#> 
#> $b_closed_form
#> [1] 1.128963e-15
#> 
#> $gap
#> [1] 0.117379
#> 
#> $lr_limit
#> [1] 0.03802226
#> 
#> $estimate
#> [1] 0.002750286
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "Linear regression by SGD (PyTorch listing, computed in numpy)"
#>