METEOR machine-translation metric (exact-match stage)

Candidate unigrams align to reference unigrams by exact match (left to right, earliest unmatched reference occurrence). With m matches, precision P = m/len(candidate), recall R = m/len(reference), and \(Fmean = 10 P R / (R + 9 P)\). Matched unigrams group into the fewest chunks of contiguous candidate positions mapped to contiguous reference positions, giving \(Penalty = 0.5 (chunks/m)^3\) and \(Score = Fmean (1 - Penalty)\). No matches gives score 0.

Usage

Meteor(candidate, reference, lowercase = TRUE)

Arguments

candidate: System translation (string split on whitespace, or character vector of tokens).
reference: Reference translation.
lowercase: Case-fold before matching (default TRUE).

Value

List with score, fmean, penalty, precision, recall, matches, chunks, len_candidate, len_reference, method.

References

Banerjee, S. and Lavie, A. (2005), METEOR: An automatic metric for MT evaluation with improved correlation with human judgments, Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Ann Arbor, 65-72. Fmean, penalty and score formulas with the worked example, Section 2.1, p. 68.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
Meteor(V, V)
#> $score
#> [1] 0.9990234
#> 
#> $fmean
#> [1] 1
#> 
#> $penalty
#> [1] 0.0009765625
#> 
#> $precision
#> [1] 1
#> 
#> $recall
#> [1] 1
#> 
#> $matches
#> [1] 8
#> 
#> $chunks
#> [1] 1
#> 
#> $len_candidate
#> [1] 8
#> 
#> $len_reference
#> [1] 8
#> 
#> $method
#> [1] "METEOR exact-match stage (Banerjee-Lavie 2005)"
#>