OPUS-MT Dashboard

Language pair: fra-eng
Models: [all models] [OPUS-MT] [external] [contributed] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: [bleu][spbleu] chrf [chrf++][comet]
Chart Type: [barchart][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]
exclude scores of user-contributed translations

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (chrf)	Output	OPUS-MT	chrf	external	chrf	Diff	contributed	chrf	Diff
0	flores101-devtest	compare	fra-eng/opus...2022-03-09	0.6935	facebook/nllb-200-3.3B	0.68758	0.006
1	flores200-devtest	compare	roa-eng/opus...2024-08-17	0.69372	facebook/nllb-200-3.3B	0.6897	0.004	nllb-200-54.5B	0.70051	-0.007
2	multi30k_test_2016_flickr	compare	roa-eng/opus...2024-08-17	0.68181	facebook/nll...illed-1.3B	0.68136	0.000
3	multi30k_test_2017_flickr	compare	roa-eng/opus...2024-08-17	0.7071	facebook/nllb-200-1.3B	0.69104	0.016
4	multi30k_test_2017_mscoco	compare	roa-eng/opus...2024-08-17	0.70206	facebook/nllb-200-3.3B	0.69966	0.002
5	multi30k_test_2018_flickr	compare	roa-eng/opus...2024-08-17	0.66363	facebook/nllb-200-3.3B	0.65264	0.011
6	newsdiscusstest2015	compare	roa-eng/opus...2024-08-17	0.6276	facebook/nllb-200-3.3B	0.59909	0.029
7	newssyscomb2009	compare	roa-eng/opus...2024-08-17	0.57723	facebook/nllb-200-1.3B	0.56847	0.009
8	newstest2008	compare	roa-eng/opus...2024-08-17	0.54629	facebook/nllb-200-3.3B	0.54426	0.002
9	newstest2009	compare	fra-eng/opus-2021-02-19	0.57119	facebook/nllb-200-3.3B	0.56412	0.007
10	newstest2010	compare	roa-eng/opus...2024-08-17	0.59825	facebook/nllb-200-1.3B	0.59709	0.001
11	newstest2011	compare	roa-eng/opus...2024-08-17	0.60537	facebook/nllb-200-3.3B	0.59732	0.008
12	newstest2012	compare	roa-eng/opus...2024-08-17	0.59834	facebook/nllb-200-1.3B	0.59323	0.005
13	newstest2013	compare	roa-eng/opus...2024-08-17	0.597	facebook/nllb-200-1.3B	0.59108	0.006
14	newstest2014	compare	roa-eng/opus...2024-08-17	0.64645	facebook/nllb-200-3.3B	0.64511	0.001
15	ntrex128	compare	roa-eng/opus...2024-08-17	0.63065	facebook/nllb-200-3.3B	0.63871	-0.008
16	tatoeba-test-v2020-07-28	compare	roa-eng/opus...2024-08-17	0.73438	facebook/nllb-200-3.3B	0.73776	-0.003
17	tatoeba-test-v2021-03-30	compare	roa-eng/opus...2024-08-17	0.73753	facebook/nllb-200-3.3B	0.74179	-0.004
18	tatoeba-test-v2021-08-07	compare	roa-eng/opus...2024-08-17	0.74026	facebook/nllb-200-3.3B	0.73885	0.001
19	tico19-test	compare	roa-eng/opus...2024-08-17	0.63394	facebook/nllb-200-3.3B	0.65213	-0.018
		average		0.649		0.646	0.004		0.701	-0.051