Zero-shot learning: LLM generalizes to unseen tasks from prompt only

Scores labels under model(prompt, labels) or model(prompt) with no weight updates; softmax-normalises. If null_prompt given, subtracts the null-prompt scores before normalising (contextual calibration).

Usage

morie_geron_zero_shot(model, prompt, labels = NULL, null_prompt = NULL)

Arguments

model: Function(prompt) or function(prompt, labels) -> named list/numeric scores.
prompt: Passed to model unchanged.
labels: Optional candidate labels (>=2).
null_prompt: Optional content-free prompt for calibration.

Value

list with probabilities, scores, raw_scores, labels, predicted (0-based), predicted_label, margin, entropy, calibrated, estimate, n, method.

Examples

scores <- c(negative = 0, positive = 1)
morie_geron_zero_shot(function(p) scores, "Review: it was great. Sentiment:")
#> $probabilities
#> [1] 0.2689414 0.7310586
#> 
#> $scores
#> [1] 0 1
#> 
#> $raw_scores
#> [1] 0 1
#> 
#> $labels
#> [1] "negative" "positive"
#> 
#> $predicted
#> [1] 1
#> 
#> $predicted_label
#> [1] "positive"
#> 
#> $margin
#> [1] 0.4621172
#> 
#> $entropy
#> [1] 0.5822031
#> 
#> $calibrated
#> [1] FALSE
#> 
#> $estimate
#> [1] 0.7310586
#> 
#> $n
#> [1] 2
#> 
#> $method
#> [1] "Zero-shot label scoring with softmax normalisation"
#>