Multimodal MAE squared reconstruction loss (Kamath Ch 9)

sum_m ||x_masked - Decoder_m(visible)||^2 across the modalities.

Usage

morie_kamath_multimodal_mae(x_visible, x_masked_true, masks, decoders = NULL)

Arguments

x_visible, x_masked_true, masks, decoders

Named lists keyed by modality (a bare object is treated as the single modality "default").

Value

List with estimate, per_modality, n_masked.

Examples

morie_kamath_multimodal_mae(list(v = 0), list(v = c(1, 2)), list(v = c(TRUE,
    FALSE, TRUE)), decoders = list(v = function(x, m) c(0.5,
    1)))
#> $estimate
#> [1] 1.25
#> 
#> $loss
#> [1] 1.25
#> 
#> $per_modality
#> $per_modality$v
#> [1] 1.25
#> 
#> 
#> $modalities
#> [1] "v"
#> 
#> $n_masked
#> [1] 2
#> 
#> $n
#> [1] 1
#> 
#> $method
#> [1] "Multimodal MAE squared reconstruction loss"
#>