OPUS-MT Dashboard

Language pair: eng-deu
Models: [all models] [OPUS-MT] [external] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: bleu [spbleu][chrf][chrf++][comet]
Chart Type: [barchart][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (bleu)	Output	OPUS-MT	bleu	external	bleu	Diff
0	flores101-devtest	compare	eng-deu/opus...2021-12-08	40	facebook/wmt19-en-de	40.9	-0.9
1	flores200-devtest	compare	eng-deu/opus...2021-12-08	40	facebook/wmt19-en-de	40.9	-0.9
2	generaltest2022	compare	eng-deu/opus...2021-12-08	46.4	facebook/nll...illed-1.3B	43.7	2.7
3	multi30k_task2_test_2016	compare	gmw-gmw/opus...2022-08-11	2.7	facebook/wmt19-en-de	3.2	-0.5
4	multi30k_test_2016_flickr	compare	en-de/opus-2020-02-26	36.3	facebook/wmt19-en-de	39.7	-3.4
5	multi30k_test_2017_flickr	compare	eng-deu/opus-2021-02-22	35.8	facebook/wmt19-en-de	39.5	-3.7
6	multi30k_test_2017_mscoco	compare	gmw-deu+eng+...2024-05-30	30.7	facebook/wmt19-en-de	34.2	-3.5
7	multi30k_test_2018_flickr	compare	eng-deu/opus-2021-02-19	31.6	facebook/wmt19-en-de	36.7	-5.1
8	newssyscomb2009	compare	eng-deu/opus...2021-12-08	24.2	facebook/wmt19-en-de	25.5	-1.3
9	newstest2008	compare	eng-deu/opus...2021-12-08	24.7	facebook/nllb-200-3.3B	23.1	1.6
10	newstest2009	compare	eng-deu/opus...2021-12-08	23.6	facebook/wmt19-en-de	24.6	-1.0
11	newstest2010	compare	eng-deu/opus...2021-12-08	26.8	facebook/wmt19-en-de	28	-1.2
12	newstest2011	compare	eng-deu/opus...2021-12-08	24	facebook/wmt19-en-de	25.3	-1.3
13	newstest2012	compare	eng-deu/opus...2021-12-08	24.7	facebook/wmt19-en-de	29.4	-4.7
14	newstest2013	compare	eng-deu/opus...2021-12-08	28.3	facebook/wmt19-en-de	32.8	-4.5
15	newstest2014	compare	eng-deu/opus...2021-12-08	30.4	facebook/wmt19-en-de	35	-4.6
16	newstest2015	compare	eng-deu/opus...2021-12-08	33.3	facebook/wmt19-en-de	40.7	-7.4
17	newstest2016	compare	eng-deu/opus...2021-12-08	39.8	facebook/wmt19-en-de	40.6	-0.8
18	newstest2017	compare	eng-deu/opus...2021-12-08	31.9	facebook/wmt19-en-de	38.7	-6.8
19	newstest2018	compare	eng-deu/opus...2021-12-08	48.8	gsarti/opus-...-big-en-de	48.6	0.2
20	newstest2019	compare	eng-deu/opus...2021-12-08	44.6	gsarti/opus-...-big-en-de	44.3	0.3
21	newstest2020	compare	eng-deu/opus...2021-12-08	34.6	facebook/wmt19-en-de	36.6	-2.0
22	newstest2021	compare	eng-deu/opus...2021-12-08	52.8	facebook/nll...illed-1.3B	49.8	3.0
23	newstestALL2020	compare	eng-deu/opus...2021-12-08	47.9	facebook/nllb-200-1.3B	42	5.9
24	newstestB2020	compare	eng-deu/opus...2021-12-08	34.4	facebook/wmt19-en-de	36.8	-2.4
25	ntrex128	compare	eng-deu/opus...2021-12-08	31.7	facebook/nll...illed-1.3B	30.4	1.3
26	tatoeba-test-v2020-07-28	compare	en-de/opus-2020-02-26	46.7	facebook/nll...illed-1.3B	46.5	0.2
27	tatoeba-test-v2021-03-30	compare	en-de/opus-2020-02-26	45.5	facebook/nll...illed-1.3B	45.5	0.0
28	tatoeba-test-v2021-08-07	compare	en-de/opus-2020-02-26	44.7	facebook/nll...illed-1.3B	44.8	-0.1
		average		34.7		36.1	-1.4