OPUS-MT Dashboard

Language pair: deu-eng
Models: [all models] [OPUS-MT] [external] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: [bleu] spbleu [chrf][chrf++][comet]
Chart Type: [barchart][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (spbleu)	Output	OPUS-MT	spbleu	external	spbleu	Diff
0	flores101-devtest	compare	gmw-gmw/opus...2022-08-11	45.8	facebook/wmt19-de-en	49.3	-3.500
1	flores200-devtest	compare	gmw-eng/opus...2024-08-17	48.1	facebook/wmt19-de-en	49.3	-1.200
2	generaltest2022	compare	gmw-eng/opus...2024-08-17	52.3	facebook/nllb-200-1.3B	44.2	8.100
3	multi30k_task2_test_2016	compare	gmw-eng/opus...2024-08-17	5.2	facebook/nll...illed-1.3B	5.7	-0.500
4	multi30k_test_2016_flickr	compare	gmw-eng/opus...2024-08-17	45	facebook/nll...illed-1.3B	45.8	-0.800
5	multi30k_test_2017_flickr	compare	gmw-eng/opus...2024-08-17	45.5	facebook/wmt19-de-en	48.1	-2.600
6	multi30k_test_2017_mscoco	compare	gmw-eng/opus...2024-08-17	37.7	facebook/nllb-200-3.3B	40.3	-2.600
7	multi30k_test_2018_flickr	compare	gmw-eng/opus...2024-08-17	41.3	facebook/wmt19-de-en	43.2	-1.900
8	newssyscomb2009	compare	gmw-eng/opus...2024-08-17	32.7	facebook/wmt19-de-en	33.1	-0.400
9	newstest2008	compare	gmw-eng/opus...2024-08-17	31.1	facebook/nllb-200-1.3B	30.9	0.200
10	newstest2009	compare	deu-eng/opus-2021-02-12	30.6	facebook/wmt19-de-en	32	-1.400
11	newstest2010	compare	gmw-eng/opus...2024-08-17	35.4	facebook/wmt19-de-en	38.3	-2.900
12	newstest2011	compare	deu-eng/opus-2021-02-22	31.7	facebook/wmt19-de-en	33.6	-1.900
13	newstest2012	compare	deu-eng/opus-2021-02-12	32.8	facebook/wmt19-de-en	36.4	-3.600
14	newstest2013	compare	deu-eng/opus-2021-02-12	35.9	facebook/wmt19-de-en	39.7	-3.800
15	newstest2014	compare	gmw-eng/opus...2024-08-17	39.7	facebook/wmt19-de-en	43.4	-3.700
16	newstest2015	compare	gmw-eng/opus...2024-08-17	39.3	facebook/wmt19-de-en	44.7	-5.400
17	newstest2016	compare	gmw-eng/opus...2024-08-17	46.4	facebook/wmt19-de-en	51	-4.600
18	newstest2017	compare	gmw-eng/opus...2024-08-17	41.1	facebook/wmt19-de-en	46.5	-5.400
19	newstest2018	compare	gmw-eng/opus...2024-08-17	49	facebook/wmt19-de-en	52.6	-3.600
20	newstest2019	compare	gmw-eng/opus...2024-08-17	45.5	facebook/wmt19-de-en	45.6	-0.100
21	newstest2020	compare	deu-eng/opus-2021-02-12	43.8	facebook/wmt19-de-en	43.1	0.700
22	newstest2021	compare	gmw-eng/opus...2024-08-17	54.5	facebook/nllb-200-1.3B	48.5	6.000
23	newstestALL2020	compare	deu-eng/opus-2021-02-22	57.8	facebook/nllb-200-1.3B	47.1	10.700
24	newstestB2020	compare	deu-eng/opus-2021-02-22	43	allenai/wmt1...en-6-6-big	42.1	0.900
25	ntrex128	compare	gmw-eng/opus...2024-08-17	41.1	facebook/nllb-200-3.3B	44	-2.900
26	tatoeba-test-v2020-07-28	compare	deu-eng/opus-2021-02-22	56.2	facebook/nllb-200-3.3B	59.4	-3.200
27	tatoeba-test-v2021-03-30	compare	deu-eng/opus-2021-02-12	54.8	facebook/nllb-200-3.3B	58.3	-3.500
28	tatoeba-test-v2021-08-07	compare	deu-eng/opus-2021-02-12	53.5	facebook/nll...illed-1.3B	56.7	-3.200
		average		41.959		43.203	-1.245