OPUS-MT Dashboard

Language pair: deu-fra
Models: [all models] [OPUS-MT] [external] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: bleu [spbleu][chrf][chrf++][comet]
Chart Type: [barchart][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (bleu)	Output	OPUS-MT	bleu	external	bleu	Diff
0	flores101-devtest	compare	deu-fra/opus...2022-07-22	39.3	facebook/nll...illed-1.3B	36.2	3.1
1	flores200-devtest	compare	deu-fra/opus...2022-07-22	39.3	facebook/nll...illed-1.3B	36.2	3.1
2	generaltest2022	compare	deu-fra/opus...2022-07-22	40.4	facebook/nll...illed-1.3B	35.4	5.0
3	multi30k_test_2016_flickr	compare	deu-fra/opus...2022-07-22	41	facebook/nll...illed-1.3B	39.3	1.7
4	multi30k_test_2017_flickr	compare	deu-fra/opus...2022-07-22	40	facebook/nll...illed-1.3B	36.4	3.6
5	multi30k_test_2017_mscoco	compare	deu-fra/opus...2022-07-22	36.2	facebook/nll...illed-1.3B	33.8	2.4
6	multi30k_test_2018_flickr	compare	deu-fra/opus...2022-07-22	32.4	facebook/nll...illed-1.3B	30.4	2.0
7	newssyscomb2009	compare	deu-fra/opus...2022-07-22	28.3	facebook/nllb-200-1.3B	28.2	0.1
8	newstest2008	compare	deu-fra/opus...2022-07-22	27.5	facebook/nllb-200-1.3B	26.6	0.9
9	newstest2009	compare	deu-fra/opus...2022-07-22	26.4	facebook/nllb-200-1.3B	25.4	1.0
10	newstest2010	compare	deu-fra/opus...2022-07-22	30.6	facebook/nll...illed-1.3B	29.8	0.8
11	newstest2011	compare	deu-fra/opus...2022-07-22	28.5	facebook/nllb-200-1.3B	28.1	0.4
12	newstest2012	compare	deu-fra/opus...2022-07-22	28.8	facebook/nllb-200-1.3B	28.5	0.3
13	newstest2013	compare	deu-fra/opus...2022-07-22	31.3	facebook/nllb-200-3.3B	30.8	0.5
14	newstest2019	compare	deu-fra/opus...2022-07-22	37.2	facebook/nllb-200-1.3B	32.1	5.1
15	newstest2020	compare	deu-fra/opus...2022-07-22	33.9	facebook/nllb-200-1.3B	29.5	4.4
16	newstest2021	compare	gmw-deu+eng+...2024-05-30	32.2	facebook/nll...illed-1.3B	27.6	4.6
17	ntrex128	compare	deu-fra/opus...2022-07-22	29	facebook/nllb-200-3.3B	26.9	2.1
18	tatoeba-test-v2020-07-28	compare	deu-fra/opus...2022-07-22	50.9	facebook/nll...illed-1.3B	49.2	1.7
19	tatoeba-test-v2021-03-30	compare	deu-fra/opus...2022-07-22	51.8	facebook/nll...illed-1.3B	50.2	1.6
20	tatoeba-test-v2021-08-07	compare	deu-fra/opus...2022-07-22	52.1	facebook/nll...illed-1.3B	50.5	1.6
		average		36.1		33.9	2.2