Dynamic quantization: static per-tensor weights, dynamic per-batch activations (hmdqnt)

Usage

morie_geron_dynamic_quantization_alt(model, dtype = "int8", activations = NULL)

Arguments

model: Named list of weight tensors, or one array.
dtype: "int8"/"uint8"/"int16".
activations: Optional batch to quantize dynamically.

Value

List with quantized, scales, zero_points, dequantized, max_abs_error, compression.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
morie_geron_dynamic_quantization_alt(V)
#> $quantized
#> $quantized$weight
#> [1]  16  32  48  64  79  95 111 127
#> 
#> 
#> $scales
#> $scales$weight
#> [1] 0.06299213
#> 
#> 
#> $zero_points
#> $zero_points$weight
#> [1] 0
#> 
#> 
#> $dequantized
#> $dequantized$weight
#> [1] 1.007874 2.015748 3.023622 4.031496 4.976378 5.984252 6.992126 8.000000
#> 
#> 
#> $max_abs_error
#> $max_abs_error$weight
#> [1] 0.03149606
#> 
#> 
#> $compression
#> [1] 4
#> 
#> $bits
#> [1] 8
#> 
#> $dtype
#> [1] "int8"
#> 
#> $symmetric
#> [1] TRUE
#> 
#> $activation
#> NULL
#> 
#> $estimate
#> [1] 0.03149606
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "per-tensor affine weight quantization with per-batch dynamic activation scaling"
#>