TSMixer time-mixing and feature-mixing, all-MLP

Quoted from the paper: (4) "TP_{L->T}(X){*,i} = W_1 X{*,i} + b_1"; (5) "TM(X){*,i} = Norm(X{*,i} + Drop(sigma(TP_{L->L}(X)_{*,i})))" – Chen, S.-A., Li, C.-L., Yoder, N. C., Arik, S. O. and Pfister, T., TSMixer: An All-MLP Architecture for Time Series Forecasting, TMLR 2023 (arXiv:2303.06053), Appendix B.3.1. Feature mixing and the 2D normalization are described in prose and implemented from it. Dropout is omitted: it is a training-time stochastic operation and this routine is evaluation-time and deterministic.

Usage

morie_tsmixer(x, wtime, btime, wfeat, bfeat, wproj, bproj, horizon)

Arguments

x

list of C channels, each of length L

wtime, btime

the time-mixing MLP

wfeat, bfeat

the feature-mixing MLP

wproj, bproj

the temporal projection of eq. (4)

horizon

forecast length

Value

list(forecast, mixed, nchannels, L, horizon, mean, first, last, sumsq)

Examples

set.seed(5)
x <- list(rnorm(8), rnorm(8))
r <- morie_tsmixer(x,
                   wtime = matrix(rnorm(64, 0, 0.3), 8, 8), btime = rep(0, 8),
                   wfeat = matrix(rnorm(4, 0, 0.3), 2, 2), bfeat = rep(0, 2),
                   wproj = matrix(rnorm(24, 0, 0.3), 3, 8), bproj = rep(0, 3),
                   horizon = 3)
str(r, max.level = 1)
#> List of 9
#>  $ forecast :List of 2
#>  $ mixed    :List of 2
#>  $ nchannels: int 2
#>  $ L        : int 8
#>  $ horizon  : int 3
#>  $ mean     : num -5.55e-17
#>  $ first    : num -0.504
#>  $ last     : num 0.748
#>  $ sumsq    : num 5.4