OPUS-MT Dashboard

Language pair: eng-fra
Models: [all models] [OPUS-MT] [external] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: [bleu][spbleu] chrf [chrf++][comet]
Chart Type: [barchart][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]
include scores of user-contributed translations

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (chrf)	Output	OPUS-MT	chrf	external	chrf	Diff
0	flores101-devtest	compare	eng-fra/opus...2022-03-09	0.72494	facebook/nllb-200-3.3B	0.69188	0.033
1	flores200-devtest	compare	eng-fra/opus...2022-03-09	0.72494	facebook/nllb-200-3.3B	0.69043	0.035
2	multi30k_test_2016_flickr	compare	eng-fra/opus...2022-03-09	0.72361	facebook/nllb-200-3.3B	0.70675	0.017
3	multi30k_test_2017_flickr	compare	eng-fra/opus...2022-03-09	0.72826	facebook/nll...illed-1.3B	0.71385	0.014
4	multi30k_test_2017_mscoco	compare	gem-fra+ita+...2024-08-17	0.74406	facebook/nll...illed-1.3B	0.71615	0.028
5	multi30k_test_2018_flickr	compare	eng-fra/opus...2022-03-09	0.66723	facebook/nllb-200-3.3B	0.66042	0.007
6	newsdiscusstest2015	compare	eng-fra/opus...2022-03-09	0.64915	facebook/nll...illed-1.3B	0.63423	0.015
7	newssyscomb2009	compare	eng-fra/opus...2022-03-09	0.58903	facebook/nllb-200-3.3B	0.59189	-0.003
8	newstest2008	compare	gem-fra+ita+...2024-08-17	0.55648	facebook/nllb-200-1.3B	0.55442	0.002
9	newstest2009	compare	gmw-deu+eng+...2024-05-30	0.5803	facebook/nllb-200-3.3B	0.5793	0.001
10	newstest2010	compare	eng-fra/opus...2022-03-09	0.60156	facebook/nll...illed-1.3B	0.59915	0.002
11	newstest2011	compare	eng-fra/opus...2022-03-09	0.61632	facebook/nllb-200-3.3B	0.61468	0.002
12	newstest2012	compare	eng-fra/opus...2022-03-09	0.59736	facebook/nllb-200-3.3B	0.59492	0.002
13	newstest2013	compare	eng-fra/opus...2022-03-09	0.597	facebook/nll...illed-1.3B	0.59539	0.002
14	newstest2014	compare	eng-fra/opus...2022-03-09	0.66686	facebook/nllb-200-3.3B	0.65887	0.008
15	ntrex128	compare	eng-fra/opus...2022-03-09	0.62861	facebook/nll...illed-1.3B	0.61188	0.017
16	tatoeba-test-v2020-07-28	compare	eng-fra/opus...2022-03-09	0.6809	facebook/nll...illed-1.3B	0.66411	0.017
17	tatoeba-test-v2021-03-30	compare	eng-fra/opus...2022-03-09	0.68816	facebook/nll...illed-1.3B	0.67018	0.018
18	tatoeba-test-v2021-08-07	compare	eng-fra/opus...2022-03-09	0.69621	facebook/nll...illed-1.3B	0.67405	0.022
19	tico19-test	compare	gmw-fra+ita+...2024-08-17	0.63024	facebook/nllb-200-3.3B	0.64383	-0.014
		average		0.655		0.643	0.011