The BART text-infilling corruption: contiguous spans with Poisson
(lambda = 3) lengths are each replaced by a SINGLE mask token, so
the decoder must recover both content and span length; the score is
reconstruction cross-entropy of the target. There is exactly one
implementation: this function delegates to morie_geron_bart.
Usage
Barte(
src,
tgt,
mask_ratio = 0.3,
mean_span = 3,
permute = FALSE,
model = NULL,
seed = 0
)
Arguments
- src
Source token sequence.
- tgt
Target token sequence.
- mask_ratio
Fraction of tokens corrupted. Default 0.3.
- mean_span
Poisson mean span length. Default 3.
- permute
Also permute sentences. Default FALSE.
- model
Optional scoring model.
- seed
RNG seed for the corruption draw. Default 0.
Value
List as returned by morie_geron_bart.
References
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O.,
Stoyanov, V. and Zettlemoyer, L. (2020), BART: Denoising Sequence-to-Sequence
Pre-training for Natural Language Generation, Translation, and Comprehension, ACL
2020, arXiv:1910.13461, Section 2.2.
Examples
Barte(c("a", "b", "c", "d"), c("a", "b", "c", "d"))$estimate
#> [1] 0.9162907
