| ID | Benchmark (bleu) | Output | OPUS-MT | bleu | external | bleu | Diff |
|---|---|---|---|---|---|---|---|
| 0 | flores101-devtest | compare | eng-mkd/opus...2021-04-10 | 32 | facebook/m2m100_1.2B | 32.6 | -0.6 |
| 1 | flores200-devtest | compare | deu+eng+fra+...2024-05-30 | 35.2 | facebook/m2m100_1.2B | 32.6 | 2.6 |
| 2 | ntrex128 | compare | deu+eng+fra+...2024-05-30 | 33.9 | facebook/nll...illed-1.3B | 31.5 | 2.4 |
| 3 | tatoeba-test-v2020-07-28 | compare | en-mk/opus-2019-12-18 | 57.5 | facebook/nll...illed-1.3B | 41.8 | 15.7 |
| 4 | tatoeba-test-v2021-03-30 | compare | en-mk/opus-2019-12-18 | 57.5 | facebook/nll...illed-1.3B | 41.8 | 15.7 |
| 5 | tatoeba-test-v2021-08-07 | compare | en-mk/opus-2019-12-18 | 57.5 | facebook/nll...illed-1.3B | 41.8 | 15.7 |
| average | 45.6 | 37.0 | 8.6 |