Detect separation in a logistic regression and show why the fit cannot converge

With labels coded as signs \(s_i = 2y_i - 1\), a direction \(d\) with \(s_i x_i \cdot d > 0\) for every observation (complete separation), or \(\ge 0\) for all and \(> 0\) for one (quasi-complete), makes the log-likelihood strictly increase along \(d\) from every starting point (Research.P5.loglik_lt_shift, loglik_lt_shift_quasi), so no maximum-likelihood estimate exists (no_mle, no_mle_quasi) and the likelihood climbs towards its supremum 0 without reaching it (loglik_neg, loglik_tendsto_zero). A fitting routine that stops with huge coefficients and a warning is reporting this geometry, not a misspecified model. The exact check is a linear programme (Konis 2007): maximise \(\sum_i s_i x_i \cdot d\) subject to \(0 \le s_i x_i \cdot d \le 1\); a positive optimum is a separating direction. The programme is solved by rmorie's own dense tableau simplex (Dantzig pricing with Bland's rule after a run of degenerate pivots, so it cannot cycle); it is the same programme the earlier lpSolve path solved and reaches the same optimum (cross-validated in the test suite). method = "glm" instead inspects a fit with many iterations for fitted probabilities at 0 or 1, which is reported as the heuristic it is.

Usage

morie_logit_separation(y, x, intercept = TRUE, method = c("auto", "lp", "glm"))

Arguments

y

Binary outcome (0/1).

x

Design matrix (one row per observation); an intercept column is added unless intercept = FALSE.

intercept

Add a column of ones.

method

"auto" or "lp" (the exact linear programme, native) or "glm" (the fit heuristic).

Value

A list with separation ("none", "quasi-complete" or "complete"), direction (a separating \(d\), or NULL), margins (\(s_i x_i \cdot d\)), n_zero_margin, loglik_along (the log-likelihood at \(t d\) for \(t = 0, 1, 10, 100\): increasing towards 0 when separated), mle_exists (TRUE when the linear programme finds no separating direction and the design has full column rank: by Albert-Anderson's converse, Research.P5.mle_exists, the log-likelihood then attains its maximum at a finite coefficient vector; FALSE under separation or a rank-deficient design; NA for the "glm" heuristic), method and theorems.

References

Konis, K. (2007). Linear programming algorithms for detecting separated data in binary logistic regression models. DPhil thesis, University of Oxford.

Examples

# one perfectly predictive dummy: every y = 1 has z = 1
x <- cbind(z = c(1, 1, 1, 0, 0, 0, 0), w = c(0.2, -1, 0.5, 0.1, -0.4, 1.2, 0.3))
y <- c(1, 1, 1, 0, 0, 0, 0)
morie_logit_separation(y, x)[c("separation", "loglik_along")]
#> $separation
#> [1] "complete"
#> 
#> $loglik_along
#>           t=0           t=1          t=10         t=100 
#> -4.852030e+00 -2.192832e+00 -3.177923e-04 -2.604053e-43 
#>