OPUS-MT Dashboard

Language pair: deu-eng
Models: [all models] [OPUS-MT] [external] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: [bleu][spbleu][chrf][chrf++] comet
Chart Type: [barchart][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (comet)	Output	OPUS-MT	comet	external	comet	Diff
0	flores101-devtest	compare	gmw-gmw/opus...2022-08-11	0.7328	facebook/wmt19-de-en	0.7626	-0.030
1	flores200-devtest	compare	gmw-eng/opus...2024-08-17	0.8874	facebook/wmt19-de-en	0.7626	0.125
2	multi30k_task2_test_2016	compare	gmw-eng/opus...2024-08-17	0.6136	facebook/nllb-200-3.3B	-0.7207	1.334
3	multi30k_test_2016_flickr	compare	gmw-eng/opus...2024-08-17	0.8799	facebook/wmt19-de-en	0.7291	0.151
4	multi30k_test_2017_flickr	compare	gmw-eng/opus...2024-08-17	0.874	facebook/wmt19-de-en	0.7037	0.170
5	multi30k_test_2017_mscoco	compare	gmw-eng/opus...2024-08-17	0.8492	facebook/wmt19-de-en	0.5979	0.251
6	multi30k_test_2018_flickr	compare	gmw-eng/opus...2024-08-17	0.8637	facebook/wmt19-de-en	0.6489	0.215
7	newssyscomb2009	compare	deu+eng+fra+...2024-05-30	0.8306	facebook/wmt19-de-en	0.5329	0.298
8	newstest2009	compare	gmw-deu+eng+...2024-05-30	0.8231	facebook/wmt19-de-en	0.5222	0.301
9	newstest2010	compare	gmw-deu+eng+...2024-05-30	0.8387	facebook/wmt19-de-en	0.5839	0.255
10	newstest2011	compare	gmw-deu+eng+...2024-05-30	0.8212	facebook/wmt19-de-en	0.5166	0.305
11	newstest2012	compare	gmw-deu+eng+...2024-05-30	0.8274	facebook/wmt19-de-en	0.5488	0.279
12	newstest2013	compare	gmw-deu+eng+...2024-05-30	0.8508	facebook/wmt19-de-en	0.6181	0.233
13	newstest2014	compare	gmw-eng/opus...2024-08-17	0.8521	facebook/wmt19-de-en	0.6245	0.228
14	newstest2015	compare	gmw-eng/opus...2024-08-17	0.8543	facebook/wmt19-de-en	0.6522	0.202
15	newstest2016	compare	gmw-eng/opus...2024-08-17	0.8643	facebook/wmt19-de-en	0.6872	0.177
16	newstest2017	compare	gmw-eng/opus...2024-08-17	0.8556	facebook/wmt19-de-en	0.6634	0.192
17	newstest2018	compare	gmw-eng/opus...2024-08-17	0.8696	facebook/wmt19-de-en	0.7025	0.167
18	newstest2019	compare	gmw-eng/opus...2024-08-17	0.829	facebook/wmt19-de-en	0.5614	0.268
19	newstest2020	compare	gmw-eng/opus...2024-08-17	0.837	facebook/wmt19-de-en	0.6608	0.176
20	newstestB2020	compare	gmw-eng/opus...2024-08-17	0.8368	facebook/wmt19-de-en	0.6502	0.187
21	tatoeba-test-v2020-07-28	compare	gmw-eng/opus...2024-08-17	0.9222	facebook/nll...illed-1.3B	0.8654	0.057
22	tatoeba-test-v2021-03-30	compare	gmw-eng/opus...2024-08-17	0.9196	facebook/nllb-200-3.3B	0.8598	0.060
23	tatoeba-test-v2021-08-07	compare	gmw-eng/opus...2024-08-17	0.9166	facebook/nll...illed-1.3B	0.8438	0.073
		average		0.844		0.607	0.236