OPUS-MT Dashboard

Language pair: eng-fra
Models: [all models] [OPUS-MT] [external] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: [bleu][spbleu][chrf][chrf++] comet
Chart Type: [barchart][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (comet)	Output	OPUS-MT	comet	external	comet	Diff
0	flores101-devtest	compare	eng-fra/opus...2022-03-09	0.8177	facebook/nllb-200-3.3B	0.7651	0.053
1	flores200-devtest	compare	deu+eng+fra+...2024-05-30	0.8736	facebook/nllb-200-1.3B	0.7581	0.116
2	multi30k_test_2016_flickr	compare	deu+eng+fra+...2024-05-30	0.8577	facebook/nll...illed-1.3B	0.8188	0.039
3	multi30k_test_2017_flickr	compare	deu+eng+fra+...2024-05-30	0.8622	facebook/nll...illed-1.3B	0.807	0.055
4	multi30k_test_2017_mscoco	compare	gem-fra+ita+...2024-08-17	0.8494	facebook/nll...illed-1.3B	0.7924	0.057
5	multi30k_test_2018_flickr	compare	gem-fra+ita+...2024-08-17	0.8298	facebook/nllb-200-3.3B	0.7112	0.119
6	newsdiscusstest2015	compare	deu+eng+fra+...2024-05-30	0.8258	facebook/nll...illed-1.3B	0.6011	0.225
7	newssyscomb2009	compare	deu+eng+fra+...2024-05-30	0.8368	facebook/nllb-200-3.3B	0.5985	0.238
8	newstest2008	show	eng-fra/opus...2022-03-09	0.8231			0.823
9	newstest2009	compare	deu+eng+fra+...2024-05-30	0.8244	facebook/nllb-200-1.3B	0.5761	0.248
10	newstest2010	compare	deu+eng+fra+...2024-05-30	0.8332	facebook/nll...illed-1.3B	0.6276	0.206
11	newstest2011	compare	deu+eng+fra+...2024-05-30	0.8305	facebook/nllb-200-3.3B	0.6213	0.209
12	newstest2012	compare	deu+eng+fra+...2024-05-30	0.8291	facebook/nll...illed-1.3B	0.5849	0.244
13	newstest2013	compare	deu+eng+fra+...2024-05-30	0.8444	facebook/nll...illed-1.3B	0.6334	0.211
14	newstest2014	compare	deu+eng+fra+...2024-05-30	0.863	facebook/nll...illed-1.3B	0.7322	0.131
15	ntrex128	show	eng-fra/opus...2022-03-09	0.8345			0.835
16	tatoeba-test-v2020-07-28	compare	eng-fra/opus...2022-03-09	0.9035	facebook/nll...illed-1.3B	0.8813	0.022
17	tatoeba-test-v2021-03-30	compare	eng-fra/opus...2022-03-09	0.9107	facebook/nll...illed-1.3B	0.8868	0.024
18	tatoeba-test-v2021-08-07	compare	eng-fra/opus...2022-03-09	0.9059	facebook/nll...illed-1.3B	0.8791	0.027
19	tico19-test	compare	deu+eng+fra+...2024-05-30	0.8049	facebook/nll...illed-1.3B	0.4846	0.320
		average		0.848		0.709	0.139