OPUS-MT Dashboard

Language pair: deu-eng
Models: [all models] [OPUS-MT] [external] [contributed] [compare]
Benchmark: all benchmarks [average score]
Evaluation metric: [bleu][spbleu] chrf [chrf++][comet]
Chart Type: [barchart][diff]

blue = OPUS-MT / Tatoeba-MT models, grey = external models, purple = user-contributed
render chart with [gd] [plotly]

Model Scores (comparing between OPUS-MT and external models)

ID	Benchmark (chrf)	Output	OPUS-MT	chrf	external	chrf	Diff
0	flores101-devtest	compare	deu-eng/opus...2021-04-30	0.67082	facebook/wmt19-de-en	0.694	-0.023
1	flores200-devtest	compare	gmw-eng/opus...2024-08-17	0.68836	facebook/wmt19-de-en	0.694	-0.006
2	generaltest2022	compare	gmw-eng/opus...2024-08-17	0.64778	facebook/nllb-200-1.3B	0.59227	0.056
3	multi30k_task2_test_2016	compare	gmw-eng/opus...2024-08-17	0.21895	facebook/nll...illed-1.3B	0.22733	-0.008
4	multi30k_test_2016_flickr	compare	gmw-eng/opus...2024-08-17	0.62865	facebook/nll...illed-1.3B	0.64094	-0.012
5	multi30k_test_2017_flickr	compare	gmw-eng/opus...2024-08-17	0.6491	facebook/wmt19-de-en	0.66164	-0.013
6	multi30k_test_2017_mscoco	compare	gmw-eng/opus...2024-08-17	0.57967	facebook/nllb-200-3.3B	0.60076	-0.021
7	multi30k_test_2018_flickr	compare	gmw-eng/opus...2024-08-17	0.61025	facebook/wmt19-de-en	0.62143	-0.011
8	newssyscomb2009	compare	gmw-eng/opus...2024-08-17	0.55816	allenai/wmt1...en-6-6-big	0.56201	-0.004
9	newstest2008	compare	gmw-eng/opus...2024-08-17	0.55428	facebook/nllb-200-1.3B	0.54374	0.011
10	newstest2009	compare	deu-eng/opus-2021-02-12	0.54562	facebook/wmt19-de-en	0.55242	-0.007
11	newstest2010	compare	gmw-eng/opus...2024-08-17	0.59467	facebook/wmt19-de-en	0.60475	-0.010
12	newstest2011	compare	gmw-eng/opus...2024-08-17	0.55727	facebook/wmt19-de-en	0.56576	-0.008
13	newstest2012	compare	gmw-eng/opus...2024-08-17	0.57003	facebook/wmt19-de-en	0.59023	-0.020
14	newstest2013	compare	gmw-eng/opus...2024-08-17	0.58846	facebook/wmt19-de-en	0.60836	-0.020
15	newstest2014	compare	gmw-eng/opus...2024-08-17	0.61474	facebook/wmt19-de-en	0.63382	-0.019
16	newstest2015	compare	gmw-eng/opus...2024-08-17	0.60921	facebook/wmt19-de-en	0.64003	-0.031
17	newstest2016	compare	gmw-eng/opus...2024-08-17	0.66667	facebook/wmt19-de-en	0.69123	-0.025
18	newstest2017	compare	gmw-eng/opus...2024-08-17	0.6254	facebook/wmt19-de-en	0.65779	-0.032
19	newstest2018	compare	gmw-eng/opus...2024-08-17	0.68462	facebook/wmt19-de-en	0.70248	-0.018
20	newstest2019	compare	gmw-eng/opus...2024-08-17	0.65516	facebook/wmt19-de-en	0.65172	0.003
21	newstest2020	compare	deu-eng/opus-2021-02-12	0.664	facebook/wmt19-de-en	0.65707	0.007
22	newstest2021	compare	gmw-eng/opus...2024-08-17	0.6758	facebook/nllb-200-1.3B	0.63832	0.037
23	newstestALL2020	compare	deu-eng/opus-2021-02-22	0.69728	facebook/nllb-200-1.3B	0.61471	0.083
24	newstestB2020	compare	deu-eng/opus-2021-02-22	0.66345	allenai/wmt1...en-6-6-big	0.65911	0.004
25	ntrex128	compare	gmw-eng/opus...2024-08-17	0.63808	facebook/nllb-200-3.3B	0.65075	-0.013
26	tatoeba-test-v2020-07-28	compare	de-en/opus-2020-02-26	0.70658	facebook/nllb-200-3.3B	0.71833	-0.012
27	tatoeba-test-v2021-03-30	compare	de-en/opus-2020-02-26	0.69634	facebook/nllb-200-3.3B	0.71104	-0.015
28	tatoeba-test-v2021-08-07	compare	de-en/opus-2020-02-26	0.6862	facebook/nllb-200-3.3B	0.70124	-0.015
		average		0.619		0.624	-0.005