Double DQN training loop; target computed by the online/target argmax split (hmddqn)

Target computation delegated to morie_geron_double_dqn_target.

Usage

morie_geron_double_dqn(
  env,
  Q,
  Q_target,
  buffer,
  epochs = 10,
  lr = 0.1,
  gamma = 0.95,
  target_sync = 5,
  batch_size = NULL
)

Arguments

env

Unused (provenance only).

Q, Q_target

(S,A) tables.

buffer

Transitions.

epochs, lr, gamma, target_sync, batch_size

Config.

Value

List with Q, Q_target, loss_history, overestimation_gap.

Examples

morie_geron_double_dqn(NULL, matrix(c(0, 1), 1, 2), matrix(c(10,
    -10), 1, 2), list(list(0, 0, 0, 0, FALSE)), epochs = 1, lr = 1,
    gamma = 1)
#> $Q
#>      [,1] [,2]
#> [1,]  -10    1
#> 
#> $Q_target
#>      [,1] [,2]
#> [1,]   10  -10
#> 
#> $loss_history
#> [1] 100
#> 
#> $targets
#> [1] -10
#> 
#> $vanilla_targets
#> [1] 10
#> 
#> $overestimation_gap
#> [1] 20
#> 
#> $greedy_policy
#> [1] 1
#> 
#> $sync_epochs
#> integer(0)
#> 
#> $gamma
#> [1] 1
#> 
#> $lr
#> [1] 1
#> 
#> $estimate
#> [1] 100
#> 
#> $n
#> [1] 1
#> 
#> $method
#> [1] "double DQN training loop; target computation delegated to grddqn"
#>