Denoising score matching loss (Geron Ch 18, morie.fn grscm)

L = mean w ||eps/sigma - s_theta(x0 + sigma eps)||^2.

Usage

morie_geron_score_matching_loss(x0, sigma, eps, score_pred, weight = NULL)

Arguments

x0

Clean data (m, d).

sigma

Scalar or length-m noise levels.

eps

Noise of the same shape as x0.

score_pred

Matrix or function(x_noisy, sigma).

weight

NULL, "sigma2", or a length-m vector.

Value

List with loss, per_sample, target, x_noisy, residual.

Examples

X <- matrix(c(1, 2, 1, 3, 1, 5, 1, 7), 4, byrow = TRUE)
y <- c(2, 3, 6, 9)
morie_geron_minibatch_gradient_descent(X, y, c(0, 0), 0.02, 2,
    5, seed = 3)
#> $theta
#> [1] 0.1321851 1.2569443
#> 
#> $cost_history
#> [1] 32.5000000 10.9993000  4.2661237  0.7877623  0.9527690  0.3528633
#> 
#> $theta_history
#> $theta_history[[1]]
#> [1] 0 0
#> 
#> $theta_history[[2]]
#> [1] 0.30 1.86
#> 
#> $theta_history[[3]]
#> [1] 0.2020 1.6064
#> 
#> $theta_history[[4]]
#> [1] 0.108384 1.040448
#> 
#> $theta_history[[5]]
#> [1] 0.1000038 1.0190931
#> 
#> $theta_history[[6]]
#> [1] 0.1321851 1.2569443
#> 
#> 
#> $initial_cost
#> [1] 32.5
#> 
#> $final_cost
#> [1] 0.3528633
#> 
#> $eta
#> [1] 0.02
#> 
#> $batch_size
#> [1] 2
#> 
#> $estimate
#> [1] 0.1321851 1.2569443
#> 
#> $n
#> [1] 4
#> 
morie_geron_score_matching_loss(x0 = X, sigma = y, eps = X, score_pred = X)
#> $loss
#> [1] 15.81404
#> 
#> $per_sample
#> [1]  1.250000  4.444444 18.055556 39.506173
#> 
#> $target
#>           [,1]      [,2]
#> [1,] 0.5000000 1.0000000
#> [2,] 0.3333333 1.0000000
#> [3,] 0.1666667 0.8333333
#> [4,] 0.1111111 0.7777778
#> 
#> $x_noisy
#>      [,1] [,2]
#> [1,]    3    6
#> [2,]    4   12
#> [3,]    7   35
#> [4,]   10   70
#> 
#> $residual
#>            [,1]      [,2]
#> [1,] -0.5000000 -1.000000
#> [2,] -0.6666667 -2.000000
#> [3,] -0.8333333 -4.166667
#> [4,] -0.8888889 -6.222222
#> 
#> $estimate
#> [1] 15.81404
#> 
#> $n
#> [1] 4
#>