Skip to content

UnicodeDecodeError in HTMLFormatter.quote with extended ASCII characters #17

@qqrs

Description

@qqrs

I am unable to see tracebacks for a character encoding error in a Flask app I am debugging. Flask uses weberror to show tracebacks and there appears to be a character encoding issue somewhere along the way.

Either the character encoding issue is in weberror as illustrated by the test case below, or weberror has an implied contract on the encoding it expects and this is not being respected by Flask.

$ pip show weberror

---
Metadata-Version: 2.0
Name: WebError
Version: 0.13.1
from weberror import formatter

hf = formatter.HTMLFormatter()

#s = "<Request 'http://example.com?abc=def\xc5' [GET]>"
s = "\xc5"
hf.quote(s)
Traceback (most recent call last):
  File "weberror_test_case.py", line 7, in <module>
    hf.quote(s)
  File "/home/vagrant/envs/web/local/lib/python2.7/site-packages/weberror/formatter.py", line 296, in quote
    s = s.encode('latin1', 'htmlentityreplace')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 0: ordinal not in range(128)

Quick hack fix weberror/formatter.py:

class HTMLFormatter(TextFormatter):

    def quote(self, s):
        if isinstance(s, str) and hasattr(self, 'frame'):
            s = s.decode(self.frame.source_encoding, 'replace')
        #s = s.encode('latin1', 'htmlentityreplace')
        s = s.decode('latin1').encode('latin1', 'htmlentityreplace')
        return html_quote(s)

Adding the .decode('latin1') works for my purposes in that I'm now able to see tracebacks in Flask.

It's not clear to me what the expected encoding is at this point so I don't know whether this would be a correct fix for the general case, or even whether this error is expected behavior (perhaps Flask should be decoding before passing to weberror).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions