fairseq stabilizes modular transformer support
In one sentence Facebook AI Research consolidates fairseq as the reference sequence-to-sequence framework: it adds modular support for BART, RoBERTa, mBART, wav2vec and becomes the primary codebase for FAIR's 2020 models.
Fairseq is a Facebook open-source Python library for training sequence models — the ones that read text or audio and produce more text or audio. It's less famous than transformers or PyTorch Lightning, but it's the one many important models ship on.
Through 2020 fairseq becomes the "birthplace" of a very influential collection: BART (summarization), RoBERTa (improved encoder), mBART (multilingual translation), wav2vec 2.0 (speech). All are later redistributed via HuggingFace, but the official code lives here.
For NLP researchers it's the "carpenter's kit" behind Facebook's big 2020 announcements.
Companies
Meta, Facebook AI Research
Tools
fairseq
Tags
Sources