InfoNCE contrastive loss (Geron Ch 16, morie.fn grctr)

Cross-entropy over 1 positive and N negatives; chance loss log(1+N).

Usage

morie_geron_contrastive_infonce(
  anchors,
  positives,
  negatives,
  tau = 0.1,
  normalize = TRUE
)

Arguments

anchors, positives

Matrices (B, d).

negatives

Matrix (N, d) shared, or 3-D array (B, N, d).

tau

Positive temperature.

normalize

Cosine similarity.

Value

List with loss, per_anchor_loss, pos_sim, neg_sim, hardest_negative, accuracy, chance_loss.

Examples

morie_geron_contrastive_infonce(anchors = matrix(c(1, 2, 3, 4, 5, 6), nrow = 2),
  positives = matrix(c(1, 2, 3, 4, 5, 6), nrow = 2),
  negatives = matrix(c(1, 2, 3, 4, 5, 6), nrow = 2))
#> $loss
#> [1] 1.078557
#> 
#> $per_anchor_loss
#> [1] 1.078557 1.078557
#> 
#> $pos_sim
#> [1] 1 1
#> 
#> $neg_sim
#>           [,1]      [,2]
#> [1,] 1.0000000 0.9938587
#> [2,] 0.9938587 1.0000000
#> 
#> $hardest_negative
#> [1] 1 1
#> 
#> $accuracy
#> [1] 1
#> 
#> $chance_loss
#> [1] 1.098612
#> 
#> $tau
#> [1] 0.1
#> 
#> $estimate
#> [1] 1.078557
#> 
#> $n
#> [1] 2
#> 
#> $method
#> [1] "InfoNCE contrastive loss"
#>