Skip to contents

T5: text-to-text transfer transformer (encoder-decoder)

Source: R/geron_w4d_native.R
morie_geron_t5.Rd

Span-corruption pretraining (drop contiguous spans, replace with numbered sentinels; target is the dropped spans) plus text-to-text framing of a supervised pair as (prefix + source) -> target.

Usage

morie_geron_t5(
  src,
  tgt = NULL,
  noise_density = 0.15,
  mean_span = 3,
  seed = 0,
  prefix = "translate:"
)

Arguments

src

Source sequence (string, whitespace-split, or char vector).

tgt

Optional target sequence for text-to-text framing.

noise_density, mean_span, seed, prefix

As in the Python original.

Value

list with encoder_input, decoder_target, spans, restored, lossless, n_masked, sentinels, text_to_text, estimate, n, method.

Examples

r <- morie_geron_t5("the quick brown fox jumps over the lazy dog")
r$encoder_input
#> [1] "the"          "quick"        "<extra_id_0>" "fox"          "jumps"       
#> [6] "over"         "the"          "lazy"         "dog"         
r$lossless
#> [1] TRUE