A step of the mistr_native implementation. No other function in the package calls it. See the file header for the source the module follows.
Examples
cat(mistr_cheatsheet())
#> mistr: SWA -- token i attends to (i-W, i]; span grows to k*W over k layers because attention composes. RoPE rotates pairs (2i, 2i+1) by pos*theta_i, and <R_m q, R_n k> = <R_{m-n} q, k> EXACTLY. GQA shares one kv head across n_heads/n_kv query heads. SwiGLU gates; RMSNorm is scale-invariant but NOT shift-invariant.