OPUS-MT Dashboard

Language pair: deu-eng
Models: [all models] [OPUS-MT] [external] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: bleu [spbleu][chrf][chrf++][comet]
Chart Type: [barchart][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (bleu)	Output	OPUS-MT	bleu	external	bleu	Diff
0	flores101-devtest	compare	gmw-gmw/opus...2022-08-11	41.8	facebook/wmt19-de-en	45.1	-3.3
1	flores200-devtest	compare	gmw-eng/opus...2024-08-17	43.7	facebook/wmt19-de-en	45.1	-1.4
2	generaltest2022	compare	gmw-eng/opus...2024-08-17	48.4	facebook/nllb-200-1.3B	40.7	7.7
3	multi30k_task2_test_2016	compare	ine-eng/opus...2024-08-17	4.3	facebook/nll...illed-1.3B	4.8	-0.5
4	multi30k_test_2016_flickr	compare	gmw-eng/opus...2024-08-17	42.7	facebook/nllb-200-1.3B	43.9	-1.2
5	multi30k_test_2017_flickr	compare	gmw-eng/opus...2024-08-17	42.7	facebook/wmt19-de-en	45.4	-2.7
6	multi30k_test_2017_mscoco	compare	gmw-eng/opus...2024-08-17	34.6	facebook/nllb-200-3.3B	37	-2.4
7	multi30k_test_2018_flickr	compare	gmw-eng/opus...2024-08-17	38.4	facebook/wmt19-de-en	40.4	-2.0
8	newssyscomb2009	compare	gmw-eng/opus...2024-08-17	29.4	allenai/wmt1...en-6-6-big	29.9	-0.5
9	newstest2008	compare	deu-eng/opus-2021-02-22	28	facebook/nllb-200-1.3B	27.8	0.2
10	newstest2009	compare	deu-eng/opus-2021-02-19	27.1	facebook/wmt19-de-en	28.3	-1.2
11	newstest2010	compare	gmw-eng/opus...2024-08-17	31.5	facebook/wmt19-de-en	34.3	-2.8
12	newstest2011	compare	deu-eng/opus-2021-02-19	27.6	facebook/wmt19-de-en	29.5	-1.9
13	newstest2012	compare	deu-eng/opus-2021-02-12	29.4	facebook/wmt19-de-en	33.1	-3.7
14	newstest2013	compare	deu-eng/opus-2021-02-12	32.4	facebook/wmt19-de-en	36.5	-4.1
15	newstest2014	compare	gmw-eng/opus...2024-08-17	35.7	facebook/wmt19-de-en	39.3	-3.6
16	newstest2015	compare	gmw-eng/opus...2024-08-17	35.2	facebook/wmt19-de-en	40.6	-5.4
17	newstest2016	compare	gmw-eng/opus...2024-08-17	42.5	facebook/wmt19-de-en	47.4	-4.9
18	newstest2017	compare	gmw-eng/opus...2024-08-17	37.5	facebook/wmt19-de-en	42.9	-5.4
19	newstest2018	compare	gmw-eng/opus...2024-08-17	45.4	facebook/wmt19-de-en	49.1	-3.7
20	newstest2019	compare	gmw-eng/opus...2024-08-17	41.3	facebook/wmt19-de-en	41.2	0.1
21	newstest2020	compare	deu-eng/opus-2021-02-22	40.1	facebook/wmt19-de-en	39.2	0.9
22	newstest2021	compare	gmw-eng/opus...2024-08-17	49.3	facebook/nllb-200-1.3B	44.2	5.1
23	newstestALL2020	compare	deu-eng/opus-2021-02-22	54	facebook/nllb-200-1.3B	43.9	10.1
24	newstestB2020	compare	deu-eng/opus-2021-02-22	39.1	facebook/wmt19-de-en	38.5	0.6
25	ntrex128	compare	gmw-eng/opus...2024-08-17	37.2	facebook/nllb-200-3.3B	40.2	-3.0
26	tatoeba-test-v2020-07-28	compare	de-en/opus-2020-02-26	55.7	facebook/nllb-200-3.3B	57.6	-1.9
27	tatoeba-test-v2021-03-30	compare	de-en/opus-2020-02-26	54.2	facebook/nllb-200-3.3B	56.5	-2.3
28	tatoeba-test-v2021-08-07	compare	de-en/opus-2020-02-26	52.7	facebook/nllb-200-3.3B	54.9	-2.2
		average		38.7		39.9	-1.2