QLoRA 4-bit forward (Kamath Ch 4)

The NF4 grid dequantises W0, then morie_kamath_lora_weight_update adds (alpha/r) B A x. Codes are 0-BASED indices into the grid.

Usage

morie_kamath_qlora_4bit(W0_nf4, A, B, alpha, r, x, n_bins = 16)

morie_kamath_dequantize_nf4(codes, absmax, n_bins = 16)

Arguments

W0_nf4

List with codes and absmax.

A, B

LoRA factors.

alpha, r

Scaling.

x

Input.

n_bins

Grid size.

codes

0-based NF4 codes.

absmax

Scalar or per-row scale.

Value

List with h, W0_dequantized, n_frozen_4bit.

Examples

morie_kamath_qlora_4bit(list(codes = matrix(c(0, 1, 3, 2), 2),
    absmax = 2), matrix(c(1, 0), 1), matrix(c(1, 0), 2), 2, 1,
    c(1, 1))
#> $h
#> [1] -0.8336377 -2.4995557
#> 
#> $base
#> [1] -2.833638 -2.499556
#> 
#> $delta
#> [1] 2 0
#> 
#> $W0_dequantized
#>           [,1]       [,2]
#> [1,] -2.000000 -0.8336377
#> [2,] -1.415138 -1.0844182
#> 
#> $scaling
#> [1] 2
#> 
#> $rank
#> [1] 1
#> 
#> $n_trainable
#> [1] 4
#> 
#> $n_frozen_4bit
#> [1] 4
#> 
#> $estimate
#> [1] -0.8336377
#> 
#> $n
#> [1] 2
#> 
#> $method
#> [1] "QLoRA: NF4 dequantised base + LoRA adapter"
#>