Let each node decide how much to weight each neighbour

Fixed aggregation treats every neighbour alike, which is wrong when some edges matter more. Attention makes the weight a learned function of the two endpoints, computed per edge from node features alone, so the layer never needs the whole graph and transfers to unseen graphs.

Usage

Gat(A, X, W, a, alpha_leaky = 0.2)

Arguments

A

Adjacency; self-loops are added.

X

Node features.

W

Shared linear map.

a

Attention vector of length 2 f_out.

alpha_leaky

LeakyReLU negative slope.

Value

List with H, alpha, estimate, n, f_out.

Details

Formula: \(e_ij = LeakyReLU(a'[W h_i || W h_j])\), alpha_ij = softmax_j(e_ij), \(h_i' = sum_j alpha_ij W h_j\).

References

Velickovic et al. (2018). Graph attention networks. ICLR 2018, equations (1)-(4).

Examples

Gat(A = c(1, 2, 3, 4, 5, 6, 7, 8), X = c(1, 2, 3, 4, 5, 6, 7, 8), W = 5L,
  a = c(1, 2, 3, 4, 5, 6, 7, 8))
#> $H
#>          [,1]
#> [1,] 39.99977
#> [2,] 39.99977
#> [3,] 39.99977
#> [4,] 39.99977
#> [5,] 39.99977
#> [6,] 39.99977
#> [7,] 39.99977
#> [8,] 39.99977
#> 
#> $alpha
#>              [,1]         [,2]         [,3]         [,4]         [,5]
#> [1,] 3.975269e-31 8.756113e-27 1.928662e-22 4.248161e-18 9.357198e-14
#> [2,] 3.975269e-31 8.756113e-27 1.928662e-22 4.248161e-18 9.357198e-14
#> [3,] 3.975269e-31 8.756113e-27 1.928662e-22 4.248161e-18 9.357198e-14
#> [4,] 3.975269e-31 8.756113e-27 1.928662e-22 4.248161e-18 9.357198e-14
#> [5,] 3.975269e-31 8.756113e-27 1.928662e-22 4.248161e-18 9.357198e-14
#> [6,] 3.975269e-31 8.756113e-27 1.928662e-22 4.248161e-18 9.357198e-14
#> [7,] 3.975269e-31 8.756113e-27 1.928662e-22 4.248161e-18 9.357198e-14
#> [8,] 3.975269e-31 8.756113e-27 1.928662e-22 4.248161e-18 9.357198e-14
#>             [,6]         [,7]      [,8]
#> [1,] 2.06106e-09 4.539787e-05 0.9999546
#> [2,] 2.06106e-09 4.539787e-05 0.9999546
#> [3,] 2.06106e-09 4.539787e-05 0.9999546
#> [4,] 2.06106e-09 4.539787e-05 0.9999546
#> [5,] 2.06106e-09 4.539787e-05 0.9999546
#> [6,] 2.06106e-09 4.539787e-05 0.9999546
#> [7,] 2.06106e-09 4.539787e-05 0.9999546
#> [8,] 2.06106e-09 4.539787e-05 0.9999546
#> 
#> $estimate
#> [1] 39.99977
#> 
#> $n
#> [1] 8
#> 
#> $f_out
#> [1] 1
#> 
#> $method
#> [1] "Graph attention layer"
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"