The greedy allocation loses to the patient one when the discount is high

Greedy: reward 1 now then 1/2 forever; patient: 0 now then 1 forever. Values \(1 + \gamma/(2(1-\gamma))\) and \(\gamma/(1-\gamma)\); the patient action wins iff \(\gamma > 2/3\) (Research.P4.Bellman.myopic_suboptimal).

Usage

morie_myopic_example(gamma)

Arguments

gamma

Discount values in [0, 1).

Value

A data frame: gamma, greedy, patient, patient_better.

References

Puterman ML (1994). Markov Decision Processes, ch. 6. Wiley.

Examples

morie_myopic_example(gamma = c(0.5, 2 / 3, 0.9))
#>       gamma greedy patient patient_better
#> 1 0.5000000    1.5       1          FALSE
#> 2 0.6666667    2.0       2          FALSE
#> 3 0.9000000    5.5       9           TRUE