| ID | Benchmark (bleu) | Output | OPUS-MT | bleu | external | bleu | Diff |
|---|---|---|---|---|---|---|---|
| 0 | flores101-devtest | compare | spa-sla/opus...2022-09-15 | 18 | facebook/nllb-200-3.3B | 17.1 | 0.9 |
| 1 | flores200-devtest | compare | deu+eng+fra+...2024-05-30 | 18.6 | facebook/nllb-200-3.3B | 17.1 | 1.5 |
| 2 | newssyscomb2009 | compare | deu+eng+fra+...2024-05-30 | 24.2 | facebook/nll...illed-1.3B | 22.7 | 1.5 |
| 3 | newstest2008 | compare | deu+eng+fra+...2024-05-30 | 22.2 | facebook/nll...illed-600M | 18 | 4.2 |
| 4 | newstest2009 | compare | deu+eng+fra+...2024-05-30 | 22.8 | facebook/nll...illed-1.3B | 21.1 | 1.7 |
| 5 | newstest2010 | compare | deu+eng+fra+...2024-05-30 | 24.5 | facebook/nll...illed-1.3B | 23.2 | 1.3 |
| 6 | newstest2011 | compare | deu+eng+fra+...2024-05-30 | 23 | facebook/nll...illed-1.3B | 21.7 | 1.3 |
| 7 | newstest2012 | compare | deu+eng+fra+...2024-05-30 | 22.2 | facebook/nll...illed-1.3B | 21.2 | 1.0 |
| 8 | newstest2013 | compare | deu+eng+fra+...2024-05-30 | 25.7 | facebook/nll...illed-1.3B | 23.4 | 2.3 |
| 9 | ntrex128 | compare | deu+eng+fra+...2024-05-30 | 22.8 | facebook/nll...illed-600M | 18.1 | 4.7 |
| 10 | tatoeba-test-v2021-08-07 | compare | spa-ces/opus-2021-02-19 | 45.1 | facebook/nll...illed-1.3B | 43.2 | 1.9 |
| average | 24.5 | 22.4 | 2.0 |