SimpleTokenizer Special function does not account for string length correctly

[arturo-myota]

in lexer.go, line 164 the simple lexer can get a token reformatted/etc by a special function.
The special function returns the actual token to be used.

At line 185, the buffer is advanced by the length of the returned token from the Special function.
It should be advanced by the length of the consumed input.

Maybe Special should return the length it consumed?

[arnodel]

Yes you are correct - IIRC in my use cases the special function was returning the consumed input so it still behaved ok. Your suggestion seems sensible