Neighbourhood-restricted attention

Standard scaled dot-product softmax attention, but only over the neighbours: dense attention would throw the graph away.

Usage

morie_gtrf_attention(H, adj, WQ, WK, WV, edge_bias = NULL)

Arguments

H

Node feature matrix (n x d).

adj

Adjacency list keyed by node id.

WQ, WK, WV

Projection matrices.

edge_bias

Optional per-pair scalar biases.

Value

A list with output and note.

References

Dwivedi, V. P. and Bresson, X. (2020).

Examples

set.seed(9)
H <- matrix(rnorm(8), 4, 2)
adj <- list(`0` = 1L, `1` = c(0L, 2L), `2` = c(1L, 3L), `3` = 2L)  # 0-based node ids
I2 <- diag(2)
morie_gtrf_attention(H, adj, I2, I2, I2)$output
#>            [,1]       [,2]
#> [1,] -0.8164583 -1.1868725
#> [2,] -0.5980441  0.6402577
#> [3,] -0.4299387 -0.3485765
#> [4,] -0.1415352  1.1919869