Symmetric INT8 quantization (Geron Ch 19, morie.fn grq8)

s = max|x|/qmax, q = round(x/s) clipped; zero maps to zero exactly.

Usage

morie_geron_int8_quantization(x, bits = 8)

Arguments

x

Numeric object.

bits

Width in [2, 32].

Value

List with q, scale, dequantized, max_abs_error, snr_db.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
morie_geron_int8_quantization(V)
#> $q
#> [1]  16  32  48  64  79  95 111 127
#> 
#> $scale
#> [1] 0.06299213
#> 
#> $dequantized
#> [1] 1.007874 2.015748 3.023622 4.031496 4.976378 5.984252 6.992126 8.000000
#> 
#> $max_abs_error
#> [1] 0.03149606
#> 
#> $snr_db
#> [1] 48.73785
#> 
#> $bits
#> [1] 8
#> 
#> $estimate
#> [1]  16  32  48  64  79  95 111 127
#> 
#> $n
#> [1] 8
#>