Speculative decoding accept/reject (Kamath Ch 8)

min(1, p_t/p_d) acceptance with residual resampling; proposed is a 0-BASED token id.

Usage

morie_kamath_speculative_decoding(
  draft_probs,
  target_probs,
  proposed = NULL,
  u = NULL
)

Arguments

draft_probs, target_probs

Distributions over one vocabulary.

proposed

Optional 0-based proposal.

u

Optional uniform draw.

Value

List with estimate, accept_prob, residual, rejection_rate.

Examples

morie_kamath_speculative_decoding(c(0.6, 0.3, 0.1), c(0.3, 0.5,
    0.2), proposed = 0)
#> $estimate
#> [1] 0.5
#> 
#> $accept_prob
#> [1] 0.5
#> 
#> $ratio
#> [1] 0.5
#> 
#> $proposed
#> [1] 0
#> 
#> $residual
#> [1] 0.0000000 0.6666667 0.3333333
#> 
#> $residual_mass
#> [1] 0.3
#> 
#> $rejection_rate
#> [1] 0.3
#> 
#> $n
#> [1] 3
#> 
#> $method
#> [1] "Speculative decoding accept/reject with residual resampling"
#>