Bayesian optimisation, chosen by acquisition function

Mirrors Python morie.fn.bayoptr.bayoptr: it resolves an acquisition spelling, calls the underlying optimiser, and appends the "ucb"/"lcb" note to the result. As n_iter grows the posterior on f sharpens around the observed points and the acquisition selects the next point; the trade-off between exploration and exploitation is governed by the acquisition choice and its kappa / xi.

Usage

morie_bayoptr(
  f,
  bounds,
  acquisition = "ei",
  n_iter = 20L,
  n_init = 5L,
  kernel = "matern52",
  amplitude = 1,
  length_scale = 1,
  noise = 1e-08,
  kappa = 2,
  xi = 0,
  n_candidates = 200L,
  seed = 0L,
  X0 = NULL,
  y0 = NULL
)

bayesian_optimization_ei_ucb(
  f,
  bounds,
  acquisition = "ei",
  n_iter = 20L,
  n_init = 5L,
  kernel = "matern52",
  amplitude = 1,
  length_scale = 1,
  noise = 1e-08,
  kappa = 2,
  xi = 0,
  n_candidates = 200L,
  seed = 0L,
  X0 = NULL,
  y0 = NULL
)

Arguments

f

Objective function(x) to minimise; x is a numeric vector of length length(bounds).

bounds

List of c(low, high) pairs, one per parameter.

acquisition

One of "ei", "expected_improvement", "pi", "probability_of_improvement", "ucb", "lcb" or "confidence_bound". "ucb" and "lcb" name the same rule – the paper writes the lower bound because it minimises.

n_iter

Number of optimisation iterations after the initial design.

n_init

Number of initial uniform random points.

kernel

Kernel name; only "matern52" is implemented.

amplitude, length_scale, noise

GP hyperparameters.

kappa

Exploration weight for the LCB rule.

xi

Exploration slack for the EI / PI rules.

n_candidates

Number of L-BFGS-B starts per iteration.

seed

Seed for the shared generator.

X0, y0

Optional initial design and values; if given, n_init is ignored.

Value

Named list whose element names match the Python payload keys, including best_x, best_y, X, y, x_trace, y_trace, the hyperparameters, acq, acquisition (the spelling the caller used), and note (the underlying optimiser's note with the "ucb"/"lcb" suffix appended).

References

Mockus (1975); Snoek, Larochelle & Adams (2012).

Examples

# \donttest{
f <- function(x) -sum((x - c(0.3, 0.7))^2)
morie_bayoptr(f, bounds = list(c(0, 1), c(0, 1)), n_iter = 8L, n_init = 5L,
              seed = 1L)
#> $best_x
#>      [,1] [,2]
#> [1,]    1    0
#> 
#> $best_y
#> [1] -0.98
#> 
#> $X
#>            [,1]       [,2]
#>  [1,] 0.5665616 0.76289439
#>  [2,] 0.7457818 0.87734869
#>  [3,] 0.9710028 0.52306718
#>  [4,] 0.4443592 0.28550868
#>  [5,] 0.4442647 0.79399661
#>  [6,] 1.0000000 0.00000000
#>  [7,] 0.0000000 0.00000000
#>  [8,] 0.8446213 0.00000000
#>  [9,] 1.0000000 0.08368545
#> [10,] 0.0000000 1.00000000
#> [11,] 0.0000000 0.42483416
#> [12,] 1.0000000 1.00000000
#> [13,] 0.3424532 0.00000000
#> 
#> $y
#>  [1] -0.07501078 -0.23017393 -0.48154992 -0.19264263 -0.02964767 -0.98000000
#>  [7] -0.58000000 -0.78661240 -0.86984362 -0.18000000 -0.16571624 -0.58000000
#> [13] -0.49180227
#> 
#> $x_trace
#> $x_trace[[1]]
#>      [,1] [,2]
#> [1,]    1    0
#> 
#> $x_trace[[2]]
#>      [,1] [,2]
#> [1,]    0    0
#> 
#> $x_trace[[3]]
#>           [,1] [,2]
#> [1,] 0.8446213    0
#> 
#> $x_trace[[4]]
#>      [,1]       [,2]
#> [1,]    1 0.08368545
#> 
#> $x_trace[[5]]
#>      [,1] [,2]
#> [1,]    0    1
#> 
#> $x_trace[[6]]
#>      [,1]      [,2]
#> [1,]    0 0.4248342
#> 
#> $x_trace[[7]]
#>      [,1] [,2]
#> [1,]    1    1
#> 
#> $x_trace[[8]]
#>           [,1] [,2]
#> [1,] 0.3424532    0
#> 
#> 
#> $y_trace
#> [1] -0.9800000 -0.5800000 -0.7866124 -0.8698436 -0.1800000 -0.1657162 -0.5800000
#> [8] -0.4918023
#> 
#> $n_init
#> [1] 5
#> 
#> $n_iter
#> [1] 8
#> 
#> $acq
#> [1] "ei"
#> 
#> $kernel
#> [1] "matern52"
#> 
#> $amplitude
#> [1] 1
#> 
#> $length_scale
#> [1] 1
#> 
#> $noise
#> [1] 1e-08
#> 
#> $kappa
#> [1] 2
#> 
#> $xi
#> [1] 0
#> 
#> $n_candidates
#> [1] 200
#> 
#> $seed
#> [1] 1
#> 
#> $note
#> [1] "minimised; 'ucb' and 'lcb' name the same rule -- the paper writes the lower bound because it minimises, and says 'upper, when considering maximization'"
#> 
#> $acquisition
#> [1] "ei"
#> 
# }