OPUS-MT Dashboard

Language pair: eng-fra
Models: [all models] [OPUS-MT] [external] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: bleu [spbleu][chrf][chrf++][comet]
Chart Type: [barchart][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (bleu)	Output	OPUS-MT	bleu	external	bleu	Diff
0	flores101-devtest	compare	eng-fra/opus...2022-03-09	52.2	facebook/nllb-200-3.3B	48	4.2
1	flores200-devtest	compare	eng-fra/opus...2022-03-09	52.2	facebook/nllb-200-3.3B	47.8	4.4
2	multi30k_test_2016_flickr	compare	eng-fra/opus...2022-03-09	52.4	facebook/nllb-200-3.3B	51.3	1.1
3	multi30k_test_2017_flickr	compare	eng-fra/opus...2022-03-09	52.8	facebook/nll...illed-1.3B	51.1	1.7
4	multi30k_test_2017_mscoco	compare	gmw-deu+eng+...2024-05-30	55.7	facebook/nll...illed-1.3B	52.3	3.4
5	multi30k_test_2018_flickr	compare	eng-fra/opus...2022-03-09	43.7	facebook/nll...illed-1.3B	44.2	-0.5
6	newsdiscusstest2015	compare	eng-fra/opus...2022-03-09	40.3	facebook/nll...illed-1.3B	38.8	1.5
7	newssyscomb2009	compare	eng-fra/opus...2022-03-09	30.7	facebook/nllb-200-3.3B	32.6	-1.9
8	newstest2008	compare	eng-fra/opus...2022-03-09	27.6	facebook/nllb-200-1.3B	28.3	-0.7
9	newstest2009	compare	eng-fra/opus...2022-03-09	30	facebook/nllb-200-3.3B	30.9	-0.9
10	newstest2010	compare	eng-fra/opus...2022-03-09	33.5	facebook/nll...illed-1.3B	33.9	-0.4
11	newstest2011	compare	eng-fra/opus...2022-03-09	35	facebook/nllb-200-3.3B	35.8	-0.8
12	newstest2012	compare	eng-fra/opus...2022-03-09	32.8	facebook/nllb-200-3.3B	33.4	-0.6
13	newstest2013	compare	eng-fra/opus...2022-03-09	34.6	facebook/nll...illed-1.3B	35.2	-0.6
14	newstest2014	compare	eng-fra/opus...2022-03-09	41.9	facebook/nll...illed-1.3B	41.6	0.3
15	ntrex128	compare	eng-fra/opus...2022-03-09	37.3	facebook/nll...illed-1.3B	34.6	2.7
16	tatoeba-test-v2020-07-28	compare	eng-fra/opus...2022-03-09	51.7	facebook/nll...illed-1.3B	50	1.7
17	tatoeba-test-v2021-03-30	compare	eng-fra/opus...2022-03-09	52.5	facebook/nll...illed-1.3B	50.7	1.8
18	tatoeba-test-v2021-08-07	compare	eng-fra/opus...2022-03-09	53.2	facebook/nllb-200-3.3B	51.3	1.9
19	tico19-test	compare	eng-fra/opus...2022-03-09	40.6	facebook/nll...illed-1.3B	40.5	0.1
		average		42.5		41.6	0.9