MoE per-expert capacity (Kamath Ch 2)

C * tokens / experts, with the drop threshold reported.

Usage

morie_kamath_expert_capacity_factor(tokens_per_batch, num_experts, C)

Arguments

tokens_per_batch

Token count.

num_experts

Expert count.

C

Capacity factor.

Value

List with estimate, slots, total_slots, min_dropped.

Examples

morie_kamath_expert_capacity_factor(100, 8, 1.25)
#> $estimate
#> [1] 15.625
#> 
#> $capacity
#> [1] 15.625
#> 
#> $slots
#> [1] 16
#> 
#> $total_slots
#> [1] 128
#> 
#> $min_dropped
#> [1] 0
#> 
#> $headroom_per_expert
#> [1] 3.125
#> 
#> $num_experts
#> [1] 8
#> 
#> $n
#> [1] 100
#> 
#> $method
#> [1] "MoE per-expert capacity (Kamath Ch 2)"
#>