Skip to contents

Static post-training quantization (Geron Ch 19, morie.fn grptq)

Source: R/geron_train_native.R
morie_geron_static_ptq.Rd

Runs a calibration batch through the layers and derives one activation scale per tensor from the requested percentile of |a|. numpy's default (linear) percentile interpolation is matched by stats::quantile(type = 7).

Usage

morie_geron_static_ptq(model, calibration_data, bits = 8, percentile = 100)

Arguments

model

List of functions applied in order.

calibration_data

Representative input batch.

bits

Quantization width.

percentile

Calibration percentile in (0, 100].

Value

List with scales, activation_ranges, quantized_output, dequantized_output, output, max_abs_error.

Examples

set.seed(1)
model <- list(function(a) pmax(a, 0), function(a) a * 0.5)
calib <- matrix(rnorm(20), 5, 4)
r <- morie_geron_static_ptq(model, calib, bits = 8)
str(r, max.level = 1)
#> List of 9
#>  $ scales            : num [1:3] 0.01744 0.01256 0.00628
#>  $ activation_ranges : num [1:3] 2.215 1.595 0.798
#>  $ quantized_output  : num [1:5, 1:4] 0 15 0 127 26 0 39 59 46 0 ...
#>  $ dequantized_output: num [1:5, 1:4] 0 0.0942 0 0.7976 0.1633 ...
#>  $ output            : num [1:5, 1:4] 0 0.0918 0 0.7976 0.1648 ...
#>  $ max_abs_error     : num 0.00279
#>  $ percentile        : num 100
#>  $ estimate          : num [1:3] 0.01744 0.01256 0.00628
#>  $ n                 : int 5