MDP validation plus value iteration (Geron Ch 19, morie.fn hmmdp)

Usage

morie_geron_mdp(
  states,
  actions,
  P,
  R,
  gamma = 0.95,
  max_iter = 1000,
  tol = 1e-10
)

Arguments

states, actions: Label vectors.
P: Array (n_s,n_a,n_s).
R: Array (n_s,n_a,n_s) or (n_s,n_a).
gamma: Discount in [0,1).
max_iter: Sweep cap.
tol: Convergence threshold.

Value

List with V, Q, policy, policy_labels, n_iter, effective_horizon, estimate, n, method.

Examples

morie_geron_mdp(states = data.frame(x = c(1, 2, 3, 4), y = c(2, 4, 5, 9)),
  actions = c(1, 2, 3, 4, 5, 6, 7, 8), P = 0.5, R = c(1, 2, 3, 4, 5, 6, 7, 8))
#> $V
#> [1] 149.5 150.5
#> 
#> $Q
#>       [,1]  [,2]  [,3]  [,4]  [,5]  [,6]  [,7]  [,8]
#> [1,] 143.5 145.5 147.5 149.5 143.5 145.5 147.5 149.5
#> [2,] 144.5 146.5 148.5 150.5 144.5 146.5 148.5 150.5
#> 
#> $policy
#> [1] 3 3
#> 
#> $policy_labels
#> [1] 4 4
#> 
#> $expected_reward
#>      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8]
#> [1,]    1    3    5    7    1    3    5    7
#> [2,]    2    4    6    8    2    4    6    8
#> 
#> $n_iter
#> [1] 490
#> 
#> $gamma
#> [1] 0.95
#> 
#> $effective_horizon
#> [1] 20
#> 
#> $estimate
#> [1] 150.5
#> 
#> $n
#> [1] 2
#> 
#> $method
#> [1] "MDP validation and value iteration"
#>