OPUS-MT Dashboard

Language pair: eng-fra
Models: [all models] [OPUS-MT] [external] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: [bleu][spbleu][chrf][chrf++] comet
Chart Type: [standard][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (comet)	Output	OPUS-MT	comet	external	comet	Diff
0	flores101-devtest	compare	eng-fra/opus..2022-03-09	0.8177	facebook/nllb-200-3.3B	0.7651	0.053
1	flores200-devtest	compare	eng-fra/opus..2022-03-09	0.8177	facebook/nllb-200-1.3B	0.7581	0.060
2	multi30k_test_2016_flickr	compare	eng-fra/opus..2022-03-09	0.8048	facebook/nll..illed-1.3B	0.8188	-0.014
3	multi30k_test_2017_flickr	compare	eng-fra/opus..2022-03-09	0.7843	facebook/nll..illed-1.3B	0.807	-0.023
4	multi30k_test_2017_mscoco	compare	eng-fra/opus..2022-03-09	0.7629	facebook/nll..illed-1.3B	0.7924	-0.029
5	multi30k_test_2018_flickr	compare	eng-fra/opus..2022-03-09	0.6839	facebook/nllb-200-3.3B	0.7112	-0.027
6	newsdiscusstest2015	compare	eng-fra/opus..2022-03-09	0.6208	facebook/nll..illed-1.3B	0.6011	0.020
7	newssyscomb2009	compare	eng-fra/opus..2022-03-09	0.5966	facebook/nllb-200-3.3B	0.5985	-0.002
8	newstest2009	compare	eng-fra/opus..2022-03-09	0.5799	facebook/nllb-200-1.3B	0.5761	0.004
9	newstest2010	compare	eng-fra/opus..2022-03-09	0.6138	facebook/nll..illed-1.3B	0.6276	-0.014
10	newstest2011	compare	eng-fra/opus..2022-03-09	0.619	facebook/nllb-200-3.3B	0.6213	-0.002
11	newstest2012	compare	eng-fra/opus..2022-03-09	0.5939	facebook/nll..illed-1.3B	0.5849	0.009
12	newstest2013	compare	eng-fra/opus..2022-03-09	0.6369	facebook/nll..illed-1.3B	0.6334	0.004
13	newstest2014	compare	eng-fra/opus..2022-03-09	0.7423	facebook/nll..illed-1.3B	0.7322	0.010
14	tatoeba-test-v2020-07-28	compare	eng-fra/opus..2022-03-09	0.9035	facebook/nll..illed-1.3B	0.8813	0.022
15	tatoeba-test-v2021-03-30	compare	eng-fra/opus..2022-03-09	0.9107	facebook/nll..illed-1.3B	0.8868	0.024
16	tatoeba-test-v2021-08-07	compare	eng-fra/opus..2022-03-09	0.9059	facebook/nll..illed-1.3B	0.8791	0.027
17	tico19-test	compare	eng-fra/opus..2022-03-09	0.4541	facebook/nll..illed-1.3B	0.4846	-0.030
		average		0.714		0.709	0.005