DINO cross-view self-distillation with centering, sharpening, momentum teacher (hmdino)

Usage

morie_geron_dino(
  images,
  student,
  teacher,
  center = NULL,
  tau_s = 0.1,
  tau_t = 0.04,
  momentum = 0.996,
  center_momentum = 0.9
)

Arguments

images: Passed to callables, otherwise ignored.
student, teacher: (V,K) logits or function.
center: Running teacher center (default zeros).
tau_s, tau_t: Temperatures (tau_t < tau_s).
momentum, center_momentum: EMA coefficients in [0, 1).

Value

List with loss, teacher_probs, student_probs, teacher_entropy, teacher_next.

Examples

morie_geron_dino(images = c(1, 2, 3, 4, 5, 6, 7, 8), student = c(1, 2, 3, 4, 5, 6, 7, 8),
  teacher = c(1, 2, 3, 4, 5, 6, 7, 8))
#> $loss
#> [1] 0
#> 
#> $per_pair_loss
#>  [1] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
#> [39] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
#> 
#> $teacher_probs
#>      [,1]
#> [1,]    1
#> [2,]    1
#> [3,]    1
#> [4,]    1
#> [5,]    1
#> [6,]    1
#> [7,]    1
#> [8,]    1
#> 
#> $student_probs
#>      [,1]
#> [1,]    1
#> [2,]    1
#> [3,]    1
#> [4,]    1
#> [5,]    1
#> [6,]    1
#> [7,]    1
#> [8,]    1
#> 
#> $teacher_entropy
#> [1] 0
#> 
#> $max_entropy
#> [1] 0
#> 
#> $kl_to_uniform
#> [1] 0
#> 
#> $center_next
#> [1] 0.45
#> 
#> $teacher_next
#>      [,1]
#> [1,]    1
#> [2,]    2
#> [3,]    3
#> [4,]    4
#> [5,]    5
#> [6,]    6
#> [7,]    7
#> [8,]    8
#> 
#> $n_views
#> [1] 8
#> 
#> $n_pairs
#> [1] 56
#> 
#> $tau_s
#> [1] 0.1
#> 
#> $tau_t
#> [1] 0.04
#> 
#> $momentum
#> [1] 0.996
#> 
#> $estimate
#> [1] 0
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "DINO cross-view self-distillation with centering, sharpening and a momentum teacher"
#>