| ID | Benchmark (bleu) | Output | OPUS-MT | bleu | external | bleu | Diff |
|---|---|---|---|---|---|---|---|
| 0 | flores101-devtest | compare | eng-jpn/opus-2021-02-18 | 18.2 | staka/fugumt-en-ja | 31 | -12.8 |
| 1 | flores200-devtest | compare | eng-jpn/opus-2021-02-18 | 18.1 | staka/fugumt-en-ja | 31 | -12.9 |
| 2 | generaltest2022 | compare | eng-jpn/opus-2021-02-18 | 1.5 | facebook/nll...illed-600M | 2.7 | -1.2 |
| 3 | newstest2020 | compare | eng-jpn/opus-2021-02-18 | 15.1 | facebook/m2m100_1.2B | 21.3 | -6.2 |
| 4 | newstest2021 | compare | eng-jpn/opus-2021-02-18 | 15.4 | staka/fugumt-en-ja | 23.1 | -7.7 |
| 5 | ntrex128 | compare | jpx-jpx/opus-2020-10-04 | 0.2 | facebook/nll...illed-1.3B | 0.5 | -0.3 |
| 6 | tatoeba-test-v2020-07-28 | compare | eng-jpn/opus-2021-02-18 | 16.8 | staka/fugumt-en-ja | 34.3 | -17.5 |
| 7 | tatoeba-test-v2021-03-30 | compare | eng-jpn/opus-2021-02-18 | 16.7 | staka/fugumt-en-ja | 32.6 | -15.9 |
| 8 | tatoeba-test-v2021-08-07 | compare | eng-jpn/opus-2021-02-18 | 16.6 | staka/fugumt-en-ja | 31.4 | -14.8 |
| average | 13.2 | 23.1 | -9.9 |