Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
Bug fixes:
This minor release includes fixes for torch.distributed.launch, --user-dir
and a few smaller bugs. We also include prebuilt wheels for common platforms.
It's been a long time since our last release (0.9.0) nearly a year ago! There have been numerous changes and new features added since then, which we've tried to summarize below. While this release carries the same major version as our previous release (0.x.x), if you have code that relies on 0.9.0, it is likely you'll need to adapt it before updating to 0.10.0.
Looking forward, this will also be the last significant release with the 0.x.x numbering. The next release will be 1.0.0 and will include a major migration to the Hydra configuration system, with an eye towards modularizing fairseq to be more usable as a library.
--tpu
and --bf16
options (775122950d145382146e9120308432a9faf9a9b8)--patience
for early stopping--shorten-method=[none|truncate|random_crop]
to language modeling (and other) tasks--eval-bleu
for computing BLEU scores during training (60fbf64f302a825eee77637a0b7de54fde38fb2c)hf_gpt2
) (2728f9b06d9a3808cc7ebc2afa1401eddef35e35)--optimizer=lamb
) (f75411af2690a54a5155871f3cf7ca1f6fa15391)lstm_lm
) (9f4256edf60554afbcaadfa114525978c141f2bd)--max-sentences
renamed to --batch-size
fairseq_cli
H-
now corresponds to tokenized system outputs and newly added D-
lines correspond to detokenized outputs (f353913420b6ef8a31ecc55d2ec0c988178698e0)train_inner
) instead of a rolling average over each epoch.--print-alignment
FairseqIncrementalState
(4e48c4ae5da48a5f70c969c16793e55e12db3c81; 88185fcc3f32bd24f65875bd841166daa66ed301)LegacyFairseqCriterion
for BC) (46b773a393c423f653887c382e4d55e69627454d)FairseqCriterion::aggregate_logging_outputs
interface, use FairseqCriterion::reduce_metrics
instead (86793391e38bf88c119699bfb1993cb0a7a33968)fairseq.meters
to fairseq.logging.meters
and added new metrics aggregation module (fairseq.logging.metrics
) (1e324a5bbe4b1f68f9dadf3592dab58a54a800a8; f8b795f427a39c19a6b7245be240680617156948)#fairseq:overwrite
option (dd1298e15fdbfc0c3639906eee9934968d63fc29; 937535dba036dc3759a5334ab5b8110febbe8e6e)FairseqTask::begin_epoch
hook (122fc1db49534a5ca295fcae1b362bbd6308c32f)FairseqTask::build_generator
interface changed (cd2555a429b5f17bc47260ac1aa61068d9a43db8)RobertaModel
base class to FairseqEncoder
(307df5604131dc2b93cc0a08f7c98adbfae9d268)FairseqOptimizer.param_groups
property (8340b2d78f2b40bc365862b24477a0190ad2e2c2)--fast-stat-sync
and replace with FairseqCriterion::logging_outputs_can_be_summed
interface (fe6c2edad0c1f9130847b9a19fbbef169529b500)--raw-text
and --lazy-load
are fully deprecated; use --dataset-impl
insteadexamples/
(8845dcf5ff43ca4d3e733ade62ceca52f1f1d634)--data-buffer-size
) (411531734df8c7294e82c68e9d42177382f362ef)Possibly breaking changes:
in_proj_weight
into separate k, v, q projections in MultiheadAttention (fdf4c3e)New features:
--fast-stat-sync
option (e1ba32a)--empty-cache-freq
option (315c463)New papers:
Speed improvements:
Changelog:
generate
renamed to translate
)--tokenizer
and --bpe
47fd985
: Deprecate old Masked LM components5f78106
: Set mmap as default dataset format and infer format automaticallyNo major API changes since the last release. Cutting a new release since we'll be merging significant (possibly breaking) changes to logging, data loading and the masked LM implementation soon.
Changelog: