Linear DDPG: TD critic, deterministic-policy-gradient actor, Polyak targets (hmddpg)

Usage

morie_geron_ddpg(
  env,
  actor,
  critic,
  epochs = 20,
  lr = 0.01,
  gamma = 0.95,
  tau = 0.01,
  ou_theta = 0.15,
  ou_sigma = 0.2,
  seed = 0,
  s0 = NULL,
  actor_target = NULL,
  critic_target = NULL
)

Arguments

env: function(s, a) -> list(s2, r, done).
actor, critic: Initial weights.
epochs, lr, gamma, tau, ou_theta, ou_sigma, seed, s0, actor_target, critic_target: Config.

Value

List with actor, critic, actor_target, critic_target, critic_losses, rewards, actions, ou_noise, q_values.

Examples

a <- morie_geron_distilbert(matrix(c(10, 0), 1, 2), matrix(c(0,
    0), 1, 2), c(1), alpha_mlm = 0, alpha_ce = 1, temperature = 1)
env <- function(s, a) list(s, -((a - 1)^2), FALSE)
morie_geron_ddpg(env, 0.5, c(0, 0), epochs = 1, lr = 0, ou_sigma = 0,
    s0 = 2)
#> $actor
#> [1] 0.5
#> 
#> $critic
#> [1] 0 0
#> 
#> $actor_target
#> [1] 0.5
#> 
#> $critic_target
#> [1] 0 0
#> 
#> $critic_losses
#> [1] 0
#> 
#> $rewards
#> [1] 0
#> 
#> $actions
#> [1] 1
#> 
#> $ou_noise
#> [1] 0
#> 
#> $q_values
#> [1] 0
#> 
#> $gamma
#> [1] 0.95
#> 
#> $tau
#> [1] 0.01
#> 
#> $estimate
#> [1] 0
#> 
#> $n
#> [1] 1
#> 
#> $method
#> [1] "linear DDPG: TD critic, deterministic policy gradient actor, Polyak targets, OU exploration"
#>