Linearly embeds pixels to d_model, partitions the token grid into
non-overlapping window_size x window_size windows, runs attention
inside each window via morie_geron_scaled_dot_product. Odd
layers (0-based) cyclically shift the grid by window_size %/% 2
before partitioning and undo it after.
Value
list with Y, pooled, n_windows, window_tokens, shifted_layers, attention_pairs, full_attention_pairs, estimate, n, method.
Examples
img <- matrix(0:15, 4, 4, byrow = TRUE)
morie_geron_swin(img, window_size = 2, n_layers = 1, d_model = 4)
#> $Y
#> , , 1
#>
#> [,1] [,2] [,3] [,4]
#> [1,] 0.03348922 -0.01920552 -0.04510888 -0.09780361
#> [2,] -0.17728973 -0.22998447 -0.25588783 -0.30858257
#> [3,] -0.28090318 -0.33359792 -0.35950128 -0.41219602
#> [4,] -0.49168214 -0.54437688 -0.57028024 -0.62297498
#>
#> , , 2
#>
#> [,1] [,2] [,3] [,4]
#> [1,] 0.01841916 -0.00771892 -0.01912167 -0.04525975
#> [2,] -0.08613316 -0.11227124 -0.12367399 -0.14981207
#> [3,] -0.13174417 -0.15788225 -0.16928500 -0.19542308
#> [4,] -0.23629649 -0.26243457 -0.27383732 -0.29997540
#>
#> , , 3
#>
#> [,1] [,2] [,3] [,4]
#> [1,] 0.007333533 0.008185039 0.01490337 0.01575488
#> [2,] 0.010739558 0.011591064 0.01830940 0.01916090
#> [3,] 0.037612887 0.038464393 0.04518273 0.04603423
#> [4,] 0.041018911 0.041870418 0.04858875 0.04944026
#>
#> , , 4
#>
#> [,1] [,2] [,3] [,4]
#> [1,] 0.03682714 0.07781936 0.1482733 0.1892655
#> [2,] 0.20079602 0.24178824 0.3122422 0.3532344
#> [3,] 0.48261175 0.52360397 0.5940579 0.6350501
#> [4,] 0.64658062 0.68757284 0.7580268 0.7990190
#>
#>
#> $pooled
#> [1] -0.29474288 -0.14077812 0.02838689 0.41792307
#>
#> $n_windows
#> [1] 4
#>
#> $window_tokens
#> [1] 4
#>
#> $shifted_layers
#> [1] 0
#>
#> $attention_pairs
#> [1] 64
#>
#> $full_attention_pairs
#> [1] 256
#>
#> $estimate
#> [1] 64
#>
#> $n
#> [1] 16
#>
#> $method
#> [1] "Swin: window-local scaled dot-product attention with alternating cyclic shift"
#>