Tabular DQN: replay mini-batches + periodically synced target net (hmdqn)

Usage

morie_geron_dqn(
  env,
  Q,
  Q_target,
  buffer,
  epochs = 10,
  lr = 0.1,
  gamma = 0.95,
  target_sync = 5,
  batch_size = NULL
)

Arguments

env: Kept for provenance only (unused).
Q, Q_target: (S,A) tables.
buffer: Transitions.
epochs, lr, gamma, target_sync, batch_size: Config.

Value

List with Q, Q_target, loss_history, greedy_policy, sync_epochs.

Examples

morie_geron_dqn(NULL, matrix(c(0, 0), 1, 2), matrix(c(0, 0),
    1, 2), list(list(0, 0, 1, 0, TRUE)), epochs = 1, lr = 0.5)
#> $Q
#>      [,1] [,2]
#> [1,]  0.5    0
#> 
#> $Q_target
#>      [,1] [,2]
#> [1,]    0    0
#> 
#> $loss_history
#> [1] 1
#> 
#> $td_errors
#> [1] 1
#> 
#> $greedy_policy
#> [1] 0
#> 
#> $state_values
#> [1] 0.5
#> 
#> $sync_epochs
#> integer(0)
#> 
#> $n_updates
#> [1] 1
#> 
#> $gamma
#> [1] 0.95
#> 
#> $lr
#> [1] 0.5
#> 
#> $estimate
#> [1] 1
#> 
#> $n
#> [1] 1
#> 
#> $method
#> [1] "tabular DQN with replay mini-batches and a periodically synced target network"
#>