| ID | Benchmark (bleu) | Output | OPUS-MT | bleu | external | bleu | Diff |
|---|---|---|---|---|---|---|---|
| 0 | flores101-devtest | compare | eng-zho/opus...2022-05-14 | 1.8 | facebook/nll...illed-1.3B | 11.4 | -9.6 |
| 1 | flores200-devtest | compare | deu+eng+fra+...2024-05-30 | 1.9 | facebook/nll...illed-1.3B | 11.4 | -9.5 |
| 2 | tatoeba-test-v2020-07-28 | compare | sit-sit/opus-2021-02-18 | 2.9 | facebook/nllb-200-3.3B | 1.5 | 1.4 |
| 3 | tatoeba-test-v2021-03-30 | compare | sit-sit/opus-2021-02-18 | 2.9 | facebook/nll...illed-1.3B | 1.4 | 1.5 |
| 4 | tatoeba-test-v2021-08-07 | compare | eng-zho/opus-2021-02-23 | 2.8 | facebook/nllb-200-3.3B | 1.6 | 1.2 |
| average | 2.5 | 5.5 | -3.0 |