OPUS-MT Dashboard

Language pair: fra-eng
Models: [all models] [OPUS-MT] [external] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: [bleu] spbleu [chrf][chrf++][comet]
Chart Type: [barchart][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (spbleu)	Output	OPUS-MT	spbleu	external	spbleu	Diff
0	flores101-devtest	compare	fra-eng/opus...2022-03-09	49.9	facebook/nllb-200-3.3B	49	0.900
1	flores200-devtest	compare	fra-eng/opus...2022-03-09	49.9	facebook/nllb-200-3.3B	49.4	0.500
2	multi30k_test_2016_flickr	compare	fra-eng/opus...2022-03-09	51.5	facebook/nll...illed-1.3B	51.1	0.400
3	multi30k_test_2017_flickr	compare	roa-eng/opus...2024-08-17	54.1	facebook/nllb-200-1.3B	52.4	1.700
4	multi30k_test_2017_mscoco	compare	roa-eng/opus...2024-08-17	54.3	facebook/nll...illed-1.3B	53.6	0.700
5	multi30k_test_2018_flickr	compare	roa-eng/opus...2024-08-17	47.6	facebook/nll...illed-1.3B	46.8	0.800
6	newsdiscusstest2015	compare	fra-eng/opus...2022-03-09	42.5	facebook/nllb-200-3.3B	39.4	3.100
7	newssyscomb2009	compare	roa-eng/opus...2024-08-17	35	facebook/nllb-200-1.3B	34.2	0.800
8	newstest2008	compare	roa-deu+eng+...2024-05-30	29.9	facebook/nllb-200-3.3B	30	-0.100
9	newstest2009	compare	roa-deu+eng+...2024-05-30	33.9	facebook/nllb-200-3.3B	33.6	0.300
10	newstest2010	compare	fra-eng/opus...2022-03-09	36.9	facebook/nllb-200-1.3B	37.6	-0.700
11	newstest2011	compare	roa-deu+eng+...2024-05-30	37.6	facebook/nllb-200-3.3B	37.4	0.200
12	newstest2012	compare	roa-deu+eng+...2024-05-30	36.7	facebook/nllb-200-3.3B	36.5	0.200
13	newstest2013	compare	fra-eng/opus...2022-03-09	37.9	facebook/nllb-200-3.3B	37.7	0.200
14	newstest2014	compare	roa-eng/opus...2024-08-17	43.2	facebook/nllb-200-3.3B	43.5	-0.300
15	ntrex128	compare	roa-eng/opus...2024-08-17	41.4	facebook/nllb-200-3.3B	42.4	-1.000
16	tatoeba-test-v2020-07-28	compare	roa-eng/opus...2024-08-17	61.2	facebook/nllb-200-3.3B	61.5	-0.300
17	tatoeba-test-v2021-03-30	compare	roa-eng/opus...2024-08-17	61.4	facebook/nllb-200-3.3B	61.8	-0.400
18	tatoeba-test-v2021-08-07	compare	roa-eng/opus...2024-08-17	61.6	facebook/nllb-200-3.3B	61.2	0.400
19	tico19-test	compare	roa-eng/opus...2024-08-17	44.2	facebook/nllb-200-3.3B	45.8	-1.600
		average		45.535		45.245	0.290