Double quantization of the scale constants (Kamath Ch 4)

int8 codes for the block scales plus one shared fp32 constant. Rounding is banker's rounding in both languages.

Usage

morie_kamath_double_quantization(scales_fp32, bits = 8)

Arguments

scales_fp32

Block scales.

bits

Code width in [2, 16].

Value

List with estimate (max error), scales_int8, shared_const.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
morie_kamath_double_quantization(V)
#> $estimate
#> [1] 0.03149606
#> 
#> $scales_int8
#> [1]  16  32  48  64  79  95 111 127
#> 
#> $shared_const
#> [1] 0.06299213
#> 
#> $dequantized
#> [1] 1.007874 2.015748 3.023622 4.031496 4.976378 5.984252 6.992126 8.000000
#> 
#> $max_abs_error
#> [1] 0.03149606
#> 
#> $bits_saved_per_block
#> [1] 24
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "double quantization of the scale constants (Kamath Ch 4)"
#>