Knowledge distillation loss (Geron Ch 17, morie.fn hmkd)
Usage
morie_geron_knowledge_distillation(
teacher,
student,
X = NULL,
y = NULL,
T = 2,
alpha = 0.5
)
Arguments
- teacher, student
Logit matrices (m,C).
- X
Unused placeholder (callables not supported here).
- y
Optional integer hard labels (0-based).
- T
Temperature.
- alpha
Hard-label weight.
Value
List with loss, ce_loss, kl_loss, teacher_probs, student_probs, agreement,
estimate, n, method.
Examples
set.seed(1)
teacher <- matrix(rnorm(5 * 3), 5, 3)
student <- matrix(rnorm(5 * 3), 5, 3)
y <- sample(0:2, 5, replace = TRUE)
kd <- morie_geron_knowledge_distillation(teacher, student, y = y,
T = 2, alpha = 0.5)
is.list(kd)
#> [1] TRUE
