DQN bootstrap MSE loss (Geron Ch 19, morie.fn grdqnl)

Usage

morie_geron_dqn_loss(Q, Q_target, batch, gamma = 0.99)

Arguments

Q, Q_target: (n_states, n_actions) matrices.
batch: List of transitions c(s, a, r, s_next) or c(s, a, r, s_next, done); s / a / s_next are 0-based.
gamma: Discount in [0, 1].

Value

List with loss, targets, predictions, td_errors, max_abs_td_error, n_terminal.

Examples

Q <- matrix(0.5, 4, 2)
Qt <- matrix(0.5, 4, 2)
batch <- list(c(0L, 0L, 1, 1L, 0L), c(1L, 1L, 0, 2L, 1L))
morie_geron_dqn_loss(Q, Qt, batch, gamma = 0.9)
#> $loss
#> [1] 0.57625
#> 
#> $targets
#> [1] 1.45 0.00
#> 
#> $predictions
#> [1] 0.5 0.5
#> 
#> $td_errors
#> [1]  0.95 -0.50
#> 
#> $max_abs_td_error
#> [1] 0.95
#> 
#> $n_terminal
#> [1] 1
#> 
#> $gamma
#> [1] 0.9
#> 
#> $estimate
#> [1] 0.57625
#> 
#> $n
#> [1] 2
#>