Skip to contents

Mamba selective SSM scan (Kamath Ch 2)

Source: R/kamath_llm2_native.R
morie_kamath_mamba_ssm.Rd

h_t = exp(dt A) h_{t-1} + dt B x_t; y_t = C . h_t, with a DIAGONAL A.

Usage

morie_kamath_mamba_ssm(x, A, B, C, delta)

Arguments

x

Input sequence.

A

State diagonal.

B, C

Per-step projections.

delta

Step sizes.

Value

List with y, states, A_bar, estimate.

Examples

morie_kamath_mamba_ssm(x = c(1, 2, 3, 4, 5, 6, 7, 8), A = c(1, 2, 3, 4, 5, 6, 7, 8),
  B = c(1, 2, 3, 4, 5, 6, 7, 8), C = c(1, 2, 3, 4, 5, 6, 7, 8), delta = 0.5)
#> $y
#> [1] 1.020000e+02 3.361831e+03 1.385400e+05 6.543732e+06 3.299990e+08
#> [6] 1.721683e+10 9.152906e+11 4.918783e+13
#> 
#> $states
#>           [,1]        [,2]         [,3]         [,4]         [,5]         [,6]
#> [1,]  0.500000    1.000000     1.500000 2.000000e+00 2.500000e+00 3.000000e+00
#> [2,]  1.824361    4.718282     9.722534 1.877811e+01 3.545623e+01 6.625661e+01
#> [3,]  4.507862   15.825620    48.073373 1.447525e+02 4.394454e+02 1.339800e+03
#> [4,]  9.432208   47.018495   221.449909 1.077585e+03 5.363541e+03 2.692259e+04
#> [5,] 18.051082  132.809519   999.969635 7.972332e+03 6.535380e+04 5.407698e+05
#> [6,] 32.761204  367.013703  4490.552983 5.892001e+04 7.961873e+05 1.086167e+07
#> [7,] 57.514093 1004.646681 20135.762222 4.353773e+05 9.699564e+06 2.181625e+08
#> [8,] 98.824709 2738.912817 90254.225472 3.217043e+06 1.181649e+08 4.381910e+09
#>              [,7]         [,8]
#> [1,] 3.500000e+00 4.000000e+00
#> [2,] 1.229041e+02 2.263926e+02
#> [3,] 4.080524e+03 1.237262e+04
#> [4,] 1.351424e+05 6.755380e+05
#> [5,] 4.475319e+06 3.688315e+07
#> [6,] 1.482022e+08 2.013752e+09
#> [7,] 4.907784e+09 1.099471e+11
#> [8,] 1.625235e+11 6.002909e+12
#> 
#> $A_bar
#>          [,1]     [,2]     [,3]     [,4]     [,5]     [,6]     [,7]     [,8]
#> [1,] 1.648721 2.718282 4.481689 7.389056 12.18249 20.08554 33.11545 54.59815
#> [2,] 1.648721 2.718282 4.481689 7.389056 12.18249 20.08554 33.11545 54.59815
#> [3,] 1.648721 2.718282 4.481689 7.389056 12.18249 20.08554 33.11545 54.59815
#> [4,] 1.648721 2.718282 4.481689 7.389056 12.18249 20.08554 33.11545 54.59815
#> [5,] 1.648721 2.718282 4.481689 7.389056 12.18249 20.08554 33.11545 54.59815
#> [6,] 1.648721 2.718282 4.481689 7.389056 12.18249 20.08554 33.11545 54.59815
#> [7,] 1.648721 2.718282 4.481689 7.389056 12.18249 20.08554 33.11545 54.59815
#> [8,] 1.648721 2.718282 4.481689 7.389056 12.18249 20.08554 33.11545 54.59815
#> 
#> $estimate
#> [1] 4.918783e+13
#> 
#> $state_dim
#> [1] 8
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "Mamba selective SSM scan (ZOH, diagonal A)"
#>