Outcome-weighted learning for an optimal treatment regime

Zhao et al.'s reformulation is that maximizing the value \((d) = E[Y 1{A = d(X)} / pi(A | X)]\) is the same problem as MINIMIZING the weighted misclassification risk \([(Y / pi(A | X)) 1{A != d(X)}]\), a classification problem in which the outcome is the weight and the observed treatment is the label. The surrogate is theirs – the weighted HINGE loss with a ridge penalty, a linear-kernel weighted SVM, \(in_b (1/n) sum w_i max(0, 1 - A_i x_i primeb) + lam ||b_slope||^2\), minimized by full-batch subgradient descent on the Pegasos step schedule eta_t = 1 / (lam t): a fixed iteration count, no sampling and no tolerance test. The intercept is not penalized.

Usage

Owltrn(y, D, W, pi = NULL, lam = 0.01, n_iter = 2000L)

Arguments

y

Observed outcome, larger is better.

D

Observed binary treatment, 0/1.

W

Covariates, no intercept column; one is added.

pi

Propensity P(A = D_i | W_i) per unit, or NULL for the marginal randomization probability.

lam

Ridge penalty on the slope coefficients, positive.

n_iter

Subgradient iterations.

Value

List with beta, estimate, value, value_all_treated, value_all_control, rule, n_treated_by_rule, hinge, shift, n, p.

Details

The cheaper surrogate – weighted least squares of the label on the covariates – has the right population minimiser but is not robust to leverage. On a design where the high-weight units sit near the decision boundary and the low-weight ones sit far from it, the squared loss lets the far units outvote the informative ones and returns the exactly INVERTED rule. That happened on this module primes own anchor fixture, which is why the hinge is used.

The rule is d(x) = 1{x primeb > 0}. NEGATIVE outcomes break the equivalence, because a negative weight rewards misclassification; the standard fix is to shift Y by its minimum. That shift is NOT innocuous – it changes the relative weights and so can change the learned rule, which is exactly the objection that motivated residual weighted learning – so the amount shifted is RETURNED in shift rather than applied silently.

References

Zhao, Y., Zeng, D., Rush, A. J. & Kosorok, M. R. (2012). Estimating individualized treatment rules using outcome weighted learning. Journal of the American Statistical Association, 107(499), 1106-1118. doi:10.1080/01621459.2012.695674 Shalev-Shwartz, S., Singer, Y., Srebro, N. & Cotter, A. (2011). Pegasos: primal estimated sub-gradient solver for SVM. Mathematical Programming, 127(1), 3-30.

Examples

y <- c(2.9, 5.1, 6.8, 9.4, 11.2, 13.1, 15.0, 17.6)
g <- c(0L, 1L, 0L, 1L, 1L, 0L, 1L, 0L)
x <- c(1.2, 2.4, 3.1, 4.8, 5.3, 6.7, 7.1, 8.9)
res <- Owltrn(y = y, D = g, W = x)
res
#> $beta
#> [1]  3.0794800 -0.4277682
#> 
#> $estimate
#> [1] 11.66
#> 
#> $value
#> [1] 11.66
#> 
#> $value_all_treated
#> [1] 10.175
#> 
#> $value_all_control
#> [1] 10.1
#> 
#> $rule
#> [1] 1 1 1 1 1 1 1 0
#> 
#> $n_treated_by_rule
#> [1] 7
#> 
#> $hinge
#> [1] 0.8183704
#> 
#> $shift
#> [1] 0
#> 
#> $n
#> [1] 8
#> 
#> $p
#> [1] 2
#> 
#> $method
#> [1] "Outcome-weighted learning, weighted hinge (Zhao et al. 2012)"
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"