I try to apply mtmsn in DRCD chinese corpus, and find out that "bert.tokenization.FullTokenizer" can't handle the chinese word tokenization. Is that why I can't use mtmsn in Chinese corpus ?
But I see the "bert.tokenization" can handle the chinese word, I dont know what the problem is.
Is there any help ?
Thanks.
I try to apply mtmsn in DRCD chinese corpus, and find out that "bert.tokenization.FullTokenizer" can't handle the chinese word tokenization. Is that why I can't use mtmsn in Chinese corpus ?
But I see the "bert.tokenization" can handle the chinese word, I dont know what the problem is.
Is there any help ?
Thanks.