Formula: DINO + iBOT mask + KoLeo
Usage
Dinov2(
x,
student,
teacher,
tau = 0.1,
tau_t = 0.04,
mask = NULL,
w_ibot = 1,
w_koleo = 0.1
)Arguments
- x
An n x d matrix of batch embeddings, for KoLeo.
- student
A (1 + P) x k matrix: class token then patch tokens.
- teacher
A (1 + P) x k teacher matrix, same layout.
- tau
Student temperature.
- tau_t
Teacher temperature.
- mask
0/1 flag per patch, or NULL for every second patch.
- w_ibot, w_koleo
Weights of the iBOT and KoLeo terms.
Details
Three terms. The image-level DINO cross-entropy between the sharpened teacher and the student; the patch-level iBOT cross-entropy over the MASKED patches only; and the KoLeo regulariser -mean log d_i, where d_i is the distance from embedding i to its nearest neighbour, which spreads the batch out and stops features from piling up.
References
Oquab et al. (2024), DINOv2, TMLR 2024; Zhou et al. (2022), iBOT, ICLR 2022; Sablayrolles et al. (2019), ICLR 2019.
Examples
Dinov2(x = c(1, 2, 3, 4, 5, 6, 7, 8), student = c(1, 2, 3, 4, 5, 6, 7, 8),
teacher = c(1, 2, 3, 4, 5, 6, 7, 8))
#> $estimate
#> [1] 2.763102
#>
#> $loss
#> [1] 2.763102
#>
#> $dino
#> [1] 0
#>
#> $ibot
#> [1] 0
#>
#> $koleo
#> [1] 27.63102
#>
#> $n_masked
#> [1] 3
#>
#> $n
#> [1] 8
#>
#> $d
#> [1] 1
#>
#> $method
#> [1] "DINOv2 objective: DINO + iBOT + KoLeo"
#>
#> attr(,"class")
#> [1] "morie_rich_result" "list"