Skip to content

Commit dd4573d

Browse files
committed
Fix issue caused by pretokenising of UTF8 strings (which get converted after the pretokenise step and cause the length to be wrong)
Even if length were correct, pretokenising the string stops the scan for UTF8 char patterns so the string wouldn't be treated correctly
1 parent 12f7edd commit dd4573d

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

core

0 commit comments

Comments
 (0)