Linear Thompson sampling for contextual bandits

Formula: B = I_d + sum b b'; muhat = B^-1 f; mutilde ~ N(muhat, v^2 B^-1); a = argmax_i b_i' mutilde

Usage

Lints(
  contexts,
  played,
  rewards,
  R = 0.5,
  delta = 0.1,
  horizon = NULL,
  z = NULL
)

Arguments

contexts

See Usage.

played

See Usage.

rewards

See Usage.

R

See Usage.

delta

See Usage.

horizon

See Usage.

z

See Usage.

Value

List with arm, scores, mu_hat, mu_tilde, v, d.

References

Agrawal and Goyal (2013), Thompson Sampling for Contextual Bandits with Linear Payoffs, ICML/arXiv:1209.3352. Algorithm 1 and the definition v = R sqrt(9 d ln(T/delta)). Verified against the paper.

Examples

Lints(contexts = c(1, 2, 3, 4, 5, 6, 7, 8), played = c(1, 2, 3, 4, 5, 6, 7, 8),
  rewards = c(1, 2, 3, 4, 5, 6, 7, 8))
#> $arm
#> [1] 7
#> 
#> $scores
#> [1] 0.1002603 0.2005206 0.3007809 0.4010412 0.5013015 0.6015619 0.7018222
#> [8] 0.8020825
#> 
#> $mu_hat
#> [1] 0.995122
#> 
#> $mu_tilde
#> [1] 0.1002603
#> 
#> $v
#> [1] 3.139994
#> 
#> $d
#> [1] 1
#> 
#> $method
#> [1] "Linear Thompson sampling (Agrawal-Goyal)"
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"