OPUS-MT Dashboard

Language pair: eng-fra
Models: [all models] [OPUS-MT] [external] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: [bleu] spbleu [chrf][chrf++][comet]
Chart Type: [barchart][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (spbleu)	Output	OPUS-MT	spbleu	external	spbleu	Diff
0	flores101-devtest	compare	eng-fra/opus...2022-03-09	56.7	facebook/nllb-200-3.3B	52.7	4.000
1	flores200-devtest	compare	eng-fra/opus...2022-03-09	56.7	facebook/nllb-200-3.3B	52.4	4.300
2	multi30k_test_2016_flickr	compare	eng-fra/opus...2022-03-09	56.8	facebook/nllb-200-3.3B	55	1.800
3	multi30k_test_2017_flickr	compare	eng-fra/opus...2022-03-09	58.3	facebook/nll...illed-1.3B	56.2	2.100
4	multi30k_test_2017_mscoco	compare	gmw-deu+eng+...2024-05-30	59.8	facebook/nll...illed-1.3B	56.3	3.500
5	multi30k_test_2018_flickr	compare	eng-fra/opus...2022-03-09	49.7	facebook/nll...illed-1.3B	49.2	0.500
6	newsdiscusstest2015	compare	eng-fra/opus...2022-03-09	45.2	facebook/nll...illed-1.3B	43.5	1.700
7	newssyscomb2009	compare	eng-fra/opus-2021-02-19	37.1	facebook/nllb-200-3.3B	38.9	-1.800
8	newstest2008	compare	eng-fra/opus-2021-02-22	33.8	facebook/nll...illed-1.3B	33.8	0.000
9	newstest2009	compare	eng-fra/opus-2021-02-22	36	facebook/nllb-200-3.3B	36.5	-0.500
10	newstest2010	compare	eng-fra/opus...2022-03-09	39.1	facebook/nll...illed-1.3B	39.2	-0.100
11	newstest2011	compare	eng-fra/opus-2021-02-22	40.8	facebook/nllb-200-3.3B	41.4	-0.600
12	newstest2012	compare	eng-fra/opus...2022-03-09	38.1	facebook/nllb-200-3.3B	38.7	-0.600
13	newstest2013	compare	eng-fra/opus...2022-03-09	39.9	facebook/nll...illed-1.3B	40.5	-0.600
14	newstest2014	compare	eng-fra/opus...2022-03-09	47.5	facebook/nllb-200-3.3B	47	0.500
15	ntrex128	compare	eng-fra/opus...2022-03-09	42.2	facebook/nll...illed-1.3B	39	3.200
16	tatoeba-test-v2020-07-28	compare	eng-fra/opus...2022-03-09	54.1	facebook/nll...illed-1.3B	51.9	2.200
17	tatoeba-test-v2021-03-30	compare	eng-fra/opus...2022-03-09	55	facebook/nll...illed-1.3B	52.7	2.300
18	tatoeba-test-v2021-08-07	compare	eng-fra/opus...2022-03-09	55.7	facebook/nll...illed-1.3B	53.2	2.500
19	tico19-test	compare	eng-fra/opus...2022-03-09	44.5	facebook/nllb-200-3.3B	44.6	-0.100
		average		47.350		46.135	1.215