FP16 mixed precision with loss scaling (Geron Ch 19, morie.fn grfp6)

Usage

morie_geron_fp16_mixed_precision(loss, S, gradients = NULL)

Arguments

loss: Non-negative finite loss.
S: Positive loss scale.
gradients: Optional gradients to scale and recover.

Value

List with loss_scaled, overflow, scaled_gradients, recovered_gradients, max_roundtrip_error, n_underflow_before, n_underflow_after, fp16_max, fp16_min_normal.

Examples

morie_geron_fp16_mixed_precision(loss = 5L, S = 5L)
#> $loss_scaled
#> [1] 25
#> 
#> $overflow
#> [1] FALSE
#> 
#> $scaled_gradients
#> NULL
#> 
#> $recovered_gradients
#> NULL
#> 
#> $max_roundtrip_error
#> [1] 0
#> 
#> $n_underflow_before
#> [1] 0
#> 
#> $n_underflow_after
#> [1] 0
#> 
#> $fp16_max
#> [1] 65504
#> 
#> $fp16_min_normal
#> [1] 6.103516e-05
#> 
#> $S
#> [1] 5
#> 
#> $estimate
#> [1] 25
#> 
#> $n
#> [1] 1
#>