BERT next-sentence-prediction loss (morie.fn grnsp)

A 1-D logits vector is promoted to (n, 2) with a zero first column, so a single score is read as the IsNext logit.

Usage

morie_geron_bert_nsp_loss(logits, labels)

Arguments

logits

(n, 2) matrix or length-n vector.

labels

0/1 labels.

Value

List with loss, per_pair, probabilities, accuracy, baseline_loss.

Examples

morie_geron_bert_nsp_loss(matrix(c(1, 2, 0.5, -1, 0, 0), 3, byrow = TRUE),
    c(1, 0, 1))
#> $loss
#> [1] 0.4026074
#> 
#> $per_pair
#> [1] 0.3132617 0.2014133 0.6931472
#> 
#> $probabilities
#>           [,1]      [,2]
#> [1,] 0.2689414 0.7310586
#> [2,] 0.8175745 0.1824255
#> [3,] 0.5000000 0.5000000
#> 
#> $accuracy
#> [1] 0.6666667
#> 
#> $baseline_loss
#> [1] 0.6931472
#> 
#> $estimate
#> [1] 0.4026074
#> 
#> $n
#> [1] 3
#>