MuZero recurrent inference step — Mzrecur • rmorie

r^k, s^k = g(s^{k-1}, a^k) followed by p^k, v^k = f(s^k).

Usage

Mzrecur(state, action, dynamics, prediction = NULL)

Arguments

state: Hidden state s^{k-1}.
action: Action a^k.
dynamics: g, returning list(reward, state).
prediction: f, returning list(policy, value); NULL skips it.

Value

List with state, reward, policy, value.

References

Schrittwieser et al. (2020), arXiv:1911.08265, Methods (Search) and Equation (1). Read from the ar5iv rendering.

Examples

dynamics <- function(s, a) list(0.5, s + a)
Mzrecur(state = 0, action = 1, dynamics = dynamics)
#> $state
#> [1] 1
#> 
#> $reward
#> [1] 0.5
#> 
#> $policy
#> NULL
#> 
#> $value
#> NULL
#> 
#> $method
#> [1] "MuZero recurrent inference (Schrittwieser et al. 2020)"
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"