Optimal contiguous model-parallel layer partition (Geron Ch 17, morie.fn hmmpp)

Usage

morie_geron_model_parallelism(model, n_devices)

Arguments

model: Sequence/list of per-layer parameter counts, arrays, or mappings with a "params" entry.
n_devices: Devices.

Value

List with assignment, device_loads, max_load, imbalance, naive_utilisation, cut_points, estimate, n, method.

Examples

morie_geron_model_parallelism(model = c(1, 2, 3, 4, 5, 6, 7, 8), n_devices = 5L)
#> $assignment
#> [1] 0 0 0 1 1 2 3 4
#> 
#> $layer_sizes
#> [1] 1 2 3 4 5 6 7 8
#> 
#> $device_loads
#> [1] 6 9 6 7 8
#> 
#> $max_load
#> [1] 9
#> 
#> $imbalance
#> [1] 0.25
#> 
#> $naive_utilisation
#> [1] 0.2
#> 
#> $cut_points
#> [1] 2 4 5 6
#> 
#> $n_transfers
#> [1] 4
#> 
#> $estimate
#> [1] 0 0 0 1 1 2 3 4
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "Optimal contiguous layer partition minimising the maximum device load"
#>