Contrastive learning with in-batch negatives (hmcst)

Delegates the InfoNCE loss to morie_geron_contrastive_infonce; this wrapper builds the negative set from positives (every other row of the batch except the anchor and its positive).

Usage

morie_geron_contrastive_learning(
  embeddings,
  positives,
  tau = 0.1,
  normalize = TRUE
)

Arguments

embeddings

Matrix (B, d), B >= 3.

positives

0-based index of each anchor's positive partner (length B).

tau

Temperature.

normalize

Cosine-normalise rows.

Value

List with loss, per_anchor_loss, pos_sim, neg_sim, hardest_negative, accuracy, chance_loss, n_negatives.

Examples

morie_geron_contrastive_learning(matrix(c(1, 0, 1, 0, 0, 1),
    3, 2, byrow = TRUE), c(1, 0, 0), tau = 1)
#> $loss
#> [1] 0.4398902
#> 
#> $per_anchor_loss
#> [1] 0.3132617 0.3132617 0.6931472
#> 
#> $pos_sim
#> [1] 1 1 0
#> 
#> $neg_sim
#>      [,1]
#> [1,]    0
#> [2,]    0
#> [3,]    0
#> 
#> $hardest_negative
#> [1] 0 0 0
#> 
#> $accuracy
#> [1] 1
#> 
#> $chance_loss
#> [1] 0.6931472
#> 
#> $n_negatives
#> [1] 1
#> 
#> $negative_indices
#> $negative_indices[[1]]
#> [1] 2
#> 
#> $negative_indices[[2]]
#> [1] 2
#> 
#> $negative_indices[[3]]
#> [1] 1
#> 
#> 
#> $tau
#> [1] 1
#> 
#> $estimate
#> [1] 0.4398902
#> 
#> $n
#> [1] 3
#> 
#> $method
#> [1] "InfoNCE with in-batch negatives; loss delegated to grctr"
#>