Dueling DQN decomposition (Geron Ch 19, morie.fn grduel)

Q = V + (A - rowMeans(A)); best_action is 0-based.

Usage

morie_geron_dueling_dqn(V, A)

Arguments

V

State values (scalar or one per state).

A

(n_states, n_actions) advantage matrix, at least 2 actions.

Value

List with Q, centered_advantage, mean_advantage, best_action, advantage_sums_to_zero.

Examples

morie_geron_dueling_dqn(V = 5L, A = c(1, 2, 3, 4, 5, 6, 7, 8))
#> $Q
#>      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8]
#> [1,]  1.5  2.5  3.5  4.5  5.5  6.5  7.5  8.5
#> 
#> $centered_advantage
#>      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8]
#> [1,] -3.5 -2.5 -1.5 -0.5  0.5  1.5  2.5  3.5
#> 
#> $mean_advantage
#> [1] 4.5
#> 
#> $best_action
#> [1] 7
#> 
#> $advantage_sums_to_zero
#> [1] TRUE
#> 
#> $estimate
#>      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8]
#> [1,]  1.5  2.5  3.5  4.5  5.5  6.5  7.5  8.5
#> 
#> $n
#> [1] 1
#>