OPUS-MT Dashboard

Language pair: fra-eng
Models: [all models] [OPUS-MT] [external] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: bleu [spbleu][chrf][chrf++][comet]
Chart Type: [barchart][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (bleu)	Output	OPUS-MT	bleu	external	bleu	Diff
0	flores101-devtest	compare	fra-eng/opus...2022-03-09	46	facebook/nllb-200-3.3B	45.4	0.6
1	flores200-devtest	compare	fra-eng/opus...2022-03-09	46	facebook/nllb-200-3.3B	45.7	0.3
2	multi30k_test_2016_flickr	compare	roa-eng/opus...2024-08-17	49.8	facebook/nll...illed-1.3B	49.5	0.3
3	multi30k_test_2017_flickr	compare	roa-eng/opus...2024-08-17	52.2	facebook/nllb-200-1.3B	50.7	1.5
4	multi30k_test_2017_mscoco	compare	roa-eng/opus...2024-08-17	51.9	facebook/nll...illed-1.3B	50.9	1.0
5	multi30k_test_2018_flickr	compare	roa-eng/opus...2024-08-17	45.4	facebook/nllb-200-1.3B	44.9	0.5
6	newsdiscusstest2015	compare	roa-eng/opus...2024-08-17	40.7	facebook/nllb-200-3.3B	36.6	4.1
7	newssyscomb2009	compare	roa-eng/opus...2024-08-17	31.7	facebook/nllb-200-1.3B	30.7	1.0
8	newstest2008	compare	roa-deu+eng+...2024-05-30	26.7	facebook/nllb-200-3.3B	26.9	-0.2
9	newstest2009	compare	roa-deu+eng+...2024-05-30	30.6	facebook/nllb-200-1.3B	30.1	0.5
10	newstest2010	compare	fra-eng/opus...2022-03-09	33.4	facebook/nllb-200-1.3B	34.1	-0.7
11	newstest2011	compare	roa-deu+eng+...2024-05-30	33.9	facebook/nllb-200-3.3B	33.8	0.1
12	newstest2012	compare	roa-deu+eng+...2024-05-30	33.7	facebook/nllb-200-3.3B	33.5	0.2
13	newstest2013	compare	fra-eng/opus...2022-03-09	34.8	facebook/nllb-200-1.3B	34.7	0.1
14	newstest2014	compare	roa-eng/opus...2024-08-17	39.7	facebook/nllb-200-3.3B	40.3	-0.6
15	ntrex128	compare	roa-eng/opus...2024-08-17	37.5	facebook/nllb-200-3.3B	39	-1.5
16	tatoeba-test-v2020-07-28	compare	roa-eng/opus...2024-08-17	59.6	facebook/nllb-200-3.3B	60.2	-0.6
17	tatoeba-test-v2021-03-30	compare	roa-eng/opus...2024-08-17	59.8	facebook/nllb-200-3.3B	60.6	-0.8
18	tatoeba-test-v2021-08-07	compare	roa-eng/opus...2024-08-17	60	facebook/nllb-200-3.3B	60	0.0
19	tico19-test	compare	roa-eng/opus...2024-08-17	41.6	facebook/nllb-200-3.3B	43.4	-1.8
		average		42.8		42.6	0.2