Dueling DQN: exact gradients through Q = V + A - mean(A) (hmdldqn)

Dueling DQN: exact gradients through Q = V + A - mean(A) (hmdldqn)

Usage

morie_geron_dueling_dqn_alt(
  env,
  V,
  A,
  buffer,
  epochs = 10,
  lr = 0.1,
  gamma = 0.95,
  target_sync = 5
)

Arguments

env

Unused (provenance only).

V, A

Initial streams.

buffer

Transitions.

epochs, lr, gamma, target_sync

Config.

Value

List with Q, V, A, loss_history, advantage_mean, value_share.

Examples

morie_geron_dueling_dqn_alt(NULL, 0, matrix(c(0, 0), 1, 2), list(list(0,
    0, 1, 0, TRUE)), epochs = 1, lr = 1)
#> $Q
#>      [,1] [,2]
#> [1,]  1.5  0.5
#> 
#> $V
#> [1] 1
#> 
#> $A
#>      [,1] [,2]
#> [1,]  0.5 -0.5
#> 
#> $loss_history
#> [1] 1
#> 
#> $advantage_mean
#> [1] 0
#> 
#> $value_share
#> [1] 1
#> 
#> $greedy_policy
#> [1] 0
#> 
#> $sync_epochs
#> integer(0)
#> 
#> $gamma
#> [1] 0.95
#> 
#> $lr
#> [1] 1
#> 
#> $estimate
#> [1] 1
#> 
#> $n
#> [1] 1
#> 
#> $method
#> [1] "dueling DQN with exact gradients through Q = V + A - mean(A)"
#>