RAGAS faithfulness (Kamath Ch 7)

faithfulness = supported claims / total claims. answer is a character vector of claims or one string split on sentence punctuation; entails an optional judge function(claim, context) -> logical, defaulting to the same strict lexical containment the Python uses (every content token of the claim present in the context) – under-reports rather than hallucinating support, and the judge used is named in the payload. Mirrors morie.fn.kmfait.

Usage

morie_kamath_ragas_faithfulness(answer, context, entails = NULL)

Arguments

answer

Claims.

context

Retrieved passage(s).

entails

Optional judge.

Value

A list with estimate, supported, n_claims, claims, claim_supported, judge, n, method.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
morie_kamath_ragas_faithfulness(V, V)
#> $estimate
#> [1] 1
#> 
#> $supported
#> [1] 8
#> 
#> $n_claims
#> [1] 8
#> 
#> $claims
#> [1] "1" "2" "3" "4" "5" "6" "7" "8"
#> 
#> $claim_supported
#> [1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
#> 
#> $judge
#> [1] "lexical containment"
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "RAGAS faithfulness = supported/total claims (Kamath Ch 7)"
#>