MuZero world model: representation plus dynamics rollout

s^0 = h(o_1..o_t); r^k, s^k = g(s^{k-1}, a^k) for k = 1..K.

Usage

Mzworld(observation, actions, representation, dynamics)

Arguments

observation

Observation history passed to representation.

actions

Actions a^1..a^K.

representation

h, returning the root hidden state.

dynamics

g, returning list(reward, state).

Value

List with states, rewards, root, K.

References

Schrittwieser et al. (2020), arXiv:1911.08265, Equation (1). Read from the ar5iv rendering of the arXiv source.

Examples

representation <- function(o) o
dynamics <- function(s, a) list(0.1, s + a)
Mzworld(observation = 0, actions = c(1, 2), representation = representation,
        dynamics = dynamics)
#> $states
#> $states[[1]]
#> [1] 0
#> 
#> $states[[2]]
#> [1] 1
#> 
#> $states[[3]]
#> [1] 3
#> 
#> 
#> $rewards
#> [1] 0.1 0.1
#> 
#> $root
#> [1] 0
#> 
#> $K
#> [1] 2
#> 
#> $method
#> [1] "MuZero world-model rollout (Schrittwieser et al. 2020 eq. 1)"
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"