Softmax cross-entropy gradient (Geron Ch 4, morie.fn hmceg)

(1/m) X^T (P - Y). Softmax is over-parameterised, so every ROW of the gradient sums to zero – column_sum_max_abs measures exactly that and the parity test asserts it is ~0.

Usage

morie_geron_cross_entropy_gradient(X, Y, theta)

Arguments

X, Y, theta

As for morie_geron_cross_entropy_cost.

Value

List with gradient, probabilities, cost, grad_norm, column_sum_max_abs, estimate, n.

Examples

morie_geron_cross_entropy_gradient(X = 5L, Y = 5L, theta = c(1, 2, 3, 4, 5, 6, 7, 8))
#> $gradient
#>              [,1]         [,2]         [,3]         [,4]         [,5]      [,6]
#> [1,] 3.131317e-15 4.647286e-13 6.897184e-11 1.023633e-08 1.519206e-06 -4.999775
#>            [,7]    [,8]
#> [1,] 0.03346274 4.96631
#> 
#> $probabilities
#>              [,1]         [,2]         [,3]         [,4]         [,5]
#> [1,] 6.262633e-16 9.294572e-14 1.379437e-11 2.047266e-09 3.038412e-07
#>              [,6]        [,7]      [,8]
#> [1,] 4.509403e-05 0.006692547 0.9932621
#> 
#> $cost
#> [1] 10.00676
#> 
#> $grad_norm
#> [1] 7.047205
#> 
#> $column_sum_max_abs
#> [1] 7.546047e-16
#> 
#> $estimate
#> [1] 7.047205
#> 
#> $n
#> [1] 1
#> 
#> $method
#> [1] "softmax cross-entropy gradient (1/m) X^T (P - Y); forward pass delegated to hmcec"
#>