chronos_quantize

A step of the chronos_native implementation. Called by chronos_tokenize. See the file header for the source the module follows. the source it follows.

Usage

chronos_quantize(x, bins)

Arguments

x

Coerced to numeric by the body, with as.numeric.

bins

A list; the body reads $centers, $edges from it.

Value

A list with tokens, n_clipped, clipped_fraction, in_range, note.

Examples

bins <- chronos_uniform_bins(lo = -3, hi = 3, n_bins = 12L)
chronos_quantize(c(-2.5, 0, 0.4, 5), bins)$tokens
#> [1]  1  6  6 11