Hello ! Is it possible to make youtokentome.BPE outputs compatible with official implementation of bpe_dropout ? (https://github.com/rsennrich/subword-nmt#how-implementation-differs-from-sennrich-et-al-2016) The matter is that when building the merge operations table we have the space symbol "▁" and not the end-of-word token “</w>”.
Hello !
Is it possible to make youtokentome.BPE outputs compatible with official implementation of bpe_dropout ? (https://github.com/rsennrich/subword-nmt#how-implementation-differs-from-sennrich-et-al-2016)
The matter is that when building the merge operations table we have the space symbol "▁" and not the end-of-word token “”.