Epsilon-greedy action selection (Geron Ch 18, morie.fn grepl)

Greedy w.p. 1-eps, otherwise uniform over ALL actions, so P(greedy) = 1 - eps + eps/|A|. Actions returned 0-based.

Usage

morie_geron_epsilon_greedy(Q_s, eps, seed = 0)

Arguments

Q_s

Action values for one state.

eps

Exploration rate in [0, 1].

seed

LCG seed; two uniforms are drawn (explore test, then action) exactly as in Python.

Value

List with action, greedy_action, explored, action_probabilities, greedy_probability.

Examples

morie_geron_epsilon_greedy(Q_s = c(1, 2, 3, 4, 5, 6, 7, 8), eps = 0.5)
#> $action
#> [1] 2
#> 
#> $greedy_action
#> [1] 7
#> 
#> $explored
#> [1] TRUE
#> 
#> $action_probabilities
#> [1] 0.0625 0.0625 0.0625 0.0625 0.0625 0.0625 0.0625 0.5625
#> 
#> $greedy_probability
#> [1] 0.5625
#> 
#> $eps
#> [1] 0.5
#> 
#> $seed
#> [1] 0
#> 
#> $estimate
#> [1] 2
#> 
#> $n
#> [1] 8
#>