MDN_Web_Docs

2023-09-25

NLLB

2023-09-07

Liv4ever and ELITR-ECA

2021-12-08

CCMatrix

2021-06-28

Updated: ParaCrawl and MultiParaCrawl

2021-06-11

New: MT560 dataset

2021-04-02

GoURMET and MIZAN

2020-11-27

EuroPat and tico-19

2020-10-31

OPUS-100 corpus

2020-06-30

ELRC public

2020-05-22

MultiParaCrawl

2019-10-16

Infopankki v1

2019-10-14

New corpus: memat (Xhosa/English)

2018-10-06

New corpora: ParaCrawl, XhosaNavy

2018-02-15

New version: OpenSubtitles2018

2017-11-06

An overview of the OPUS collection

1,212 corpora

58,851,021,412 total sentence pairs

747 languages available

This table displays 100 corpora , which make up a total 94.85% of the entire OPUS collection

CorpusSentences% of OPUS
OpenSubtitles20B34.34
NLLB13B22.10
CCMatrix11B18.46
MultiCCAligned2.2B3.80864
ParaCrawl1.5B2.54692
DGT1.1B1.85690
MultiHPLT897M1.52502
XLEnt883M1.50013
MultiParaCrawl789M1.34012
LinguaTools-WikiTitles487M0.82820
CCAligned439M0.74513
HPLT381M0.64691
UNPC323M0.54947
EUbookshop279M0.47410
EMEA243M0.41283
GNOME225M0.38186
KDE4201M0.34133
Europarl186M0.31626
MultiUN159M0.27100
JRC-Acquis147M0.25024
TED2020143M0.24308
TildeMODEL128M0.21747
WikiMatrix127M0.21611
QED122M0.20764
ParaCrawl-Bonus102M0.17344
EuroPat89M0.15140
bible-uedin85M0.14501
NeuLab-TedTalks74M0.12592
Samanantar50M0.08458
Tanzil42M0.07179
MultiMaCoCu28M0.04675
JParaCrawl26M0.04388
MaCoCu26M0.04339
wikimedia23M0.03894
giga-fren23M0.03827
ELITR-ECA20M0.03427
Anuvaad18M0.03080
StanfordNLP-NMT16M0.02706
ECB15M0.02607
Wikipedia13M0.02201
SETIMES8.8M0.01496
Tatoeba8.7M0.01479
DOGC8.5M0.01440
GlobalVoices7.3M0.01233
News-Commentary6.4M0.01092
PHP6.1M0.01043
MBS5M0.008544859
SciELO3.8M0.006413549
Finlex3.1M0.005291565
infopankki2.9M0.004986148
JESC2.8M0.004753340
fiskmo2.1M0.003568332
ParIce2.1M0.003563272
GoURMET2.1M0.003561185
EUconst2.1M0.003509015
OpenOffice2M0.003478278
EOPC2M0.003422357
TED20131.9M0.003236420
EhuHac1.8M0.003054292
pmindia1.7M0.002881996
SUMMA1.6M0.002675588
IITB1.6M0.002643512
Books1.3M0.002125081
ChuBiCo1.2M0.002042471
CAPES1.2M0.001967018
Joshua-IPC1.1M0.001876583
MIZAN1M0.001735904
SCB_MT_EN_TH988K0.001679257
MDN_Web_Docs874K0.001485945
ECDC683K0.001161390
Elhuyar642K0.001091480
EiTB-ParCC637K0.001082703
TEP612K0.001040060
KDEdoc610K0.001037236
WMT-News447K0.000760204
KFTT440K0.000748137
tico-19319K0.000542699
tldr-pages258K0.000438030
memat155K0.000262974
hrenWaC99K0.000168223
TedTalks86K0.000146723
FFR82K0.000139564
SPC68K0.000115138
MontenegrinSubs65K0.000110520
OfisPublik63K0.000107767
XhosaNavy50K0.000084928
Bianet48K0.000082406
WikiSource33K0.000056555
ALT18K0.000030737
Salome9.4K0.000016017
sardware6.2K0.000010450
ada834.1K0.000007004
RF1.2K0.000001964
komiNot specifiedNot specified
liv4everNot specifiedNot specified
Mozilla-I10nNot specifiedNot specified
MPC1Not specifiedNot specified
Nunavut_HansardNot specifiedNot specified
UbuntuNot specifiedNot specified
WikiTitlesNot specifiedNot specified

OUR CONTRIBUTORS

NLPLuniversity of helsinkicschpltlets mt