Rank-based score that only cares where the first hit lands

The reciprocal makes the metric top-heavy: moving the first relevant item from rank two to rank one is worth as much as moving it from ten to two. That suits question answering, where there is one right answer, and suits recommendation badly, where there are many.

Usage

Mrr(pred_rank, relevant)

Arguments

pred_rank

Ranked item ids per query, best first.

relevant

Relevant item id(s) per query.

Value

List with estimate, rr, n_hit, Q.

Details

Formula: MRR = (1/|Q|) sum_q 1/rank_q.

References

Voorhees, E. M. (1999). TREC-8 question answering track report, 77-82.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
Mrr(V, V)
#> $estimate
#> [1] 1
#> 
#> $rr
#> [1] 1 1 1 1 1 1 1 1
#> 
#> $n_hit
#> [1] 8
#> 
#> $Q
#> [1] 8
#> 
#> $method
#> [1] "Mean reciprocal rank"
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"