Skip to contents

Supervised fine-tuning (SFT) on instruction-response pairs

Source: R/geron_w4d_native.R
morie_geron_sft.Rd

L = -sum_i log P(y_i | x_i). Softmax head trained by exact-gradient descent on bag-of-words (string prompts) or raw numeric features.

Usage

morie_geron_sft(model = NULL, instruction_data, epochs = 200, lr = 0.5, l2 = 0)

Arguments

model

Optional initial weight matrix (n_features, n_responses); default zeros.

instruction_data

List of list(instruction, response) pairs (strings or numeric/int).

epochs, lr, l2

As in the Python original.

Value

list with W, loss, sum_loss, loss_curve, accuracy, predicted, probabilities, vocab, labels, estimate, n, method.

Examples

morie_geron_sft(instruction_data = data.frame(x = c(1, 2, 3, 4), y = c(2, 4, 5, 9)))
#> $W
#>            [,1]      [,2]
#> [1,] -0.2098088 0.2098088
#> 
#> $loss
#> [1] 0.6419534
#> 
#> $sum_loss
#> [1] 1.283907
#> 
#> $loss_curve
#>   [1] 0.6931472 0.6500082 0.6434243 0.6422439 0.6420129 0.6419658 0.6419560
#>   [8] 0.6419540 0.6419535 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#>  [15] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#>  [22] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#>  [29] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#>  [36] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#>  [43] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#>  [50] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#>  [57] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#>  [64] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#>  [71] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#>  [78] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#>  [85] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#>  [92] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#>  [99] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> [106] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> [113] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> [120] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> [127] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> [134] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> [141] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> [148] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> [155] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> [162] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> [169] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> [176] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> [183] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> [190] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> [197] 0.6419534 0.6419534 0.6419534 0.6419534 0.6419534
#> 
#> $accuracy
#> [1] 0.5
#> 
#> $predicted
#> [1] 4 4
#> 
#> $probabilities
#>        [,1]      [,2]
#> x 0.3966083 0.6033917
#> y 0.3016959 0.6983041
#> 
#> $vocab
#> NULL
#> 
#> $labels
#> [1] 2 4
#> 
#> $estimate
#> [1] 0.6419534
#> 
#> $n
#> [1] 2
#> 
#> $method
#> [1] "Softmax maximum-likelihood fine-tuning on instruction-response demonstrations"
#>