Emergent-ability step metric (Kamath Ch 1)

Heaviside-gated scores plus the above/below jump. A threshold that puts every model on one side is refused.

Usage

morie_kamath_emergent_abilities(scales, scores, threshold)

Arguments

scales

Model scales.

scores

Benchmark scores.

threshold

Scale threshold.

Value

List with estimate (the jump), emergent_score, n_above.

Examples

morie_kamath_emergent_abilities(c(1, 10, 100), c(0.1, 0.2, 0.9),
    50)
#> $estimate
#> [1] 0.75
#> 
#> $jump
#> [1] 0.75
#> 
#> $emergent_score
#> [1] 0.0 0.0 0.9
#> 
#> $mean_above
#> [1] 0.9
#> 
#> $mean_below
#> [1] 0.15
#> 
#> $n_above
#> [1] 1
#> 
#> $threshold
#> [1] 50
#> 
#> $n
#> [1] 3
#> 
#> $method
#> [1] "emergent-ability step metric (Kamath Ch 1)"
#>