Prompt tuning soft-prompt prepend (Kamath Ch 4)

X_aug = [P; X]; only the L_p x d prompt is trained.

Usage

morie_kamath_prompt_tuning(P, X)

Arguments

P

Soft prompt rows.

X

Input embedding rows.

Value

List with X_aug, prompt_len, n_trainable.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
morie_kamath_prompt_tuning(V, V)
#> $X_aug
#>       [,1]
#>  [1,]    1
#>  [2,]    2
#>  [3,]    3
#>  [4,]    4
#>  [5,]    5
#>  [6,]    6
#>  [7,]    7
#>  [8,]    8
#>  [9,]    1
#> [10,]    2
#> [11,]    3
#> [12,]    4
#> [13,]    5
#> [14,]    6
#> [15,]    7
#> [16,]    8
#> 
#> $prompt_len
#> [1] 8
#> 
#> $seq_len
#> [1] 16
#> 
#> $d_model
#> [1] 1
#> 
#> $n_trainable
#> [1] 8
#> 
#> $estimate
#> [1] 8
#> 
#> $n
#> [1] 16
#> 
#> $method
#> [1] "Prompt tuning soft-prompt prepend"
#>