Prioritized replay importance weights (Geron Ch 19, morie.fn grpex)

P ~ (|delta| + eps)^alpha; w = (N P)^-beta normalised by its maximum, so max(w) = 1. max_weight_index is 0-based.

Usage

morie_geron_prioritized_experience_weight(
  priorities,
  N = NULL,
  alpha = 0.6,
  beta = 0.4,
  eps = 1e-06,
  are_td_errors = TRUE
)

Arguments

priorities

TD errors (default) or explicit positive priorities.

N

Replay buffer size; defaults to the number of priorities.

alpha, beta

Prioritisation and correction exponents in [0, 1].

eps

Non-negative floor added to |delta|.

are_td_errors

Treat priorities as TD errors.

Value

List with weights, probabilities, priorities, max_weight_index.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
morie_geron_prioritized_experience_weight(V)
#> $weights
#> [1] 1.0000000 0.8467454 0.7682295 0.7169778 0.6795905 0.6504947 0.6268687
#> [8] 0.6070976
#> 
#> $probabilities
#> [1] 0.0526336 0.0797776 0.1017503 0.1209202 0.1382436 0.1542246 0.1691694
#> [8] 0.1832808
#> 
#> $priorities
#> [1] 1.000001 2.000001 3.000001 4.000001 5.000001 6.000001 7.000001 8.000001
#> 
#> $max_weight_index
#> [1] 0
#> 
#> $estimate
#> [1] 1.0000000 0.8467454 0.7682295 0.7169778 0.6795905 0.6504947 0.6268687
#> [8] 0.6070976
#> 
#> $n
#> [1] 8
#>