Multi-head graph attention layer

The single-head layer is Gat; this function adds the multi-head interface on top of it and does not repeat the attention arithmetic. With no weights supplied the deterministic choice W = I and a = 1 is used, every head then sees the same input, and averaging identical heads is exact – which is what makes the heads argument checkable at all.

Usage

Gatemd(G, X, heads = 1)

Arguments

G: Square adjacency matrix.
X: Node feature matrix, one row per node.
heads: Number of attention heads.

Value

List with estimate, H, heads, n, method.

Details

Formula: average over heads of Gat(A, X, I, 1).

References

Velickovic, Cucurull, Casanova, Romero, Lio and Bengio (2018), Graph attention networks, ICLR 2018, eqs. (1)-(4) and (6). arXiv:1710.10903

Examples

set.seed(1)
G <- matrix(c(0, 1, 0, 1, 0, 1, 0, 1, 0), 3, 3, byrow = TRUE)
X <- matrix(rnorm(6), 3, 2)
Gatemd(G, X)
#> $estimate
#> [1] 0.1667297
#> 
#> $H
#>            [,1]       [,2]
#> [1,] -0.3121263  1.1041455
#> [2,] -0.3642703  0.9124424
#> [3,] -0.2171420 -0.1226714
#> 
#> $heads
#> [1] 1
#> 
#> $n
#> [1] 3
#> 
#> $method
#> [1] "average of gat() heads, Velickovic et al. (2018) eqs. (1)-(4) and (6)"
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"