P-tuning v2 deep prefix concatenation (Kamath Ch 4) — morie_kamath_p_tuning

[P_K_l; K_l] and [P_V_l; V_l] at every layer.

Usage

morie_kamath_p_tuning_v2(prefixes_by_layer, inputs_by_layer)

Arguments

prefixes_by_layer: List of list(P_K, P_V) per layer.
inputs_by_layer: List of list(K, V) per layer.

Value

List with K, V, prefix_len, n_trainable.

Examples

morie_kamath_p_tuning_v2(list(list(matrix(c(1, 1), 1), matrix(c(2,
    2), 1))), list(list(matrix(c(3, 3), 1), matrix(c(4, 4), 1))))
#> $K
#> $K[[1]]
#>      [,1] [,2]
#> [1,]    1    1
#> [2,]    3    3
#> 
#> 
#> $V
#> $V[[1]]
#>      [,1] [,2]
#> [1,]    2    2
#> [2,]    4    4
#> 
#> 
#> $prefix_len
#> [1] 1
#> 
#> $n_layers
#> [1] 1
#> 
#> $n_trainable
#> [1] 4
#> 
#> $estimate
#> [1] 4
#> 
#> $n
#> [1] 1
#> 
#> $method
#> [1] "P-tuning v2 deep prefix concatenation"
#>