OPUS-MT Dashboard

Language pair: fra-eng
Models: [all models] [OPUS-MT] [external] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: [bleu][spbleu][chrf][chrf++] comet
Chart Type: [barchart][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (comet)	Output	OPUS-MT	comet	external	comet	Diff
0	flores101-devtest	compare	fra-eng/opus..2022-03-09	0.811	facebook/nllb-200-3.3B	0.7973	0.014
1	flores200-devtest	compare	roa-deu+eng+..2024-05-30	0.8902	facebook/nllb-200-3.3B	0.7973	0.093
2	multi30k_test_2016_flickr	compare	roa-deu+eng+..2024-05-30	0.8839	facebook/nll..illed-1.3B	0.8145	0.069
3	multi30k_test_2017_flickr	compare	roa-deu+eng+..2024-05-30	0.8903	facebook/nllb-200-1.3B	0.7918	0.099
4	multi30k_test_2017_mscoco	compare	roa-deu+eng+..2024-05-30	0.8764	facebook/nll..illed-1.3B	0.8057	0.071
5	multi30k_test_2018_flickr	compare	roa-deu+eng+..2024-05-30	0.8666	facebook/nll..illed-1.3B	0.7185	0.148
6	newsdiscusstest2015	compare	roa-deu+eng+..2024-05-30	0.8395	facebook/nll..illed-1.3B	0.5728	0.267
7	newssyscomb2009	compare	itc-deu+eng+..2024-05-30	0.8349	facebook/nllb-200-3.3B	0.5576	0.277
8	newstest2008	show	itc-gmw/opus..2022-08-23	0.8225			0.823
9	newstest2009	compare	roa-deu+eng+..2024-05-30	0.8235	facebook/nll..illed-1.3B	0.556	0.267
10	newstest2010	compare	deu+eng+fra+..2024-05-30	0.8321	facebook/nllb-200-1.3B	0.594	0.238
11	newstest2011	compare	roa-deu+eng+..2024-05-30	0.8349	facebook/nllb-200-3.3B	0.5833	0.252
12	newstest2012	compare	roa-deu+eng+..2024-05-30	0.8337	facebook/nllb-200-1.3B	0.5649	0.269
13	newstest2013	compare	itc-deu+eng+..2024-05-30	0.8497	facebook/nllb-200-3.3B	0.6159	0.234
14	newstest2014	compare	roa-deu+eng+..2024-05-30	0.8649	facebook/nll..illed-1.3B	0.6878	0.177
15	ntrex128	show	fra-eng/opus..2022-03-09	0.8686			0.869
16	tatoeba-test-v2020-07-28	compare	roa-deu+eng+..2024-05-30	0.9221	facebook/nllb-200-3.3B	0.911	0.011
17	tatoeba-test-v2021-03-30	compare	fra-eng/opus..2022-03-09	0.923	facebook/nllb-200-3.3B	0.9137	0.009
18	tatoeba-test-v2021-08-07	compare	roa-deu+eng+..2024-05-30	0.9209	facebook/nllb-200-3.3B	0.8975	0.023
19	tico19-test	compare	roa-deu+eng+..2024-05-30	0.8271	facebook/nll..illed-1.3B	0.506	0.321
		average		0.861		0.705	0.156