FritzBox Parsing Issue crashes FritzConnection

[DerEichler]

I use the FritzConnection via the FritzTools in HomeAssistant.

I recently noticed the FritzTools integration crashes regularly. Via this issue home-assistant/core#151241 we figured out its related to a parsing error I seem to have in my error log after a reconnect of my internet connection.

Maybe we can find a fix to not let the FritzConnection crash on that parsing error.

I will open a case with AVM (Or Fritz! now) to get the parsing error resolved on their side.

[mib1185]

maybe switching to lxml and using their recover parser option could be the solution 🤔

[kbr]

Version < 1 has used lxml. For the next major version the parser implementation will change, but not the way it works – so no go back to lxml. I think in case of an error it is preferable to know whether the origin is in the library or somewhere else. Otherwise it would be hard to get a reason for an unexpected output.

[DerEichler]

My ticket with the Fritz! Support has so far returned a "not our problem, must be with your Internet provider".
I have answered them to please look into it further as I don't think the error message or better said the gibberish in the error message is coming from vodafone and rather from their end. Ticket is still ongoing.

[gunnicom]

from vodafone and rather from their end.

Then it must be the same problem with NetCologne which happens to be my provider. Happened after update of my 6660 to V8.20. Clearing the log did let me reconnect the FritBox Tools. I do not know how long it will last, just did it a few minutes ago.

[mib1185]

I think in case of an error it is preferable to know whether the origin is in the library or somewhere else. Otherwise it would be hard to get a reason for an unexpected output.

In this case the cause seems to be that the Fritzbox forwards data coming from the ISP directly into the log, without sanitising it (just a guess, it could also be an encoding issue on the box itself). Anyway, the cause are faulty data returned by the Fritzbox.
@kbr what do you think about sanitising the raw data in the lib before feeding them to the xml parser? Would this be an acceptable solution?

[kbr]

Currently the library seems to forward malformed data (or just gibberish) from another source (and crashing because of this). Sanitising would mean, that the user does not know whether data in the log (or elsewhere) are sanitised or not (are the data modified or not?). Or there is a defined sanitized message that gets forwarded. And a question will be: how must data look like, to get sanitised – and what about the original data? How to detect, that something was wrong? Applications on top of the library must implement some specific logic to deal with this.

An idea (and the pythonic way) could be to catch a crash, log the malformed data, and do something else, i.e. raising an exception, so the calling stack is informed that something is not working the way it should. But that would just be a variation of the current situation: instead of some kind of FritzConnectionError an xml.etree.ElementTree.ParseError is raised. Applications would have to handle this anyway.

[mib1982]

Things have been quiet around this issue for a while, but it’s still present.

I’ve noticed that the malformed data is always the same (at least for me): an IPv6 prefix warning line followed by something that looks like binary garbage. Since FritzBox firmware updates tend to be quite slow, I was wondering if a small workaround for this specific pattern would be possible.

For example, could fritzconnection sanitize or skip just this type of entry before it triggers a parse error, and log that it did so? That would keep the Home Assistant integration (which makes use of fritzconnection) from crashing while still making it clear that some data was filtered.

Is something like this feasible with the way fritzconnection currently processes the event log?

[kbr]

It is the same argument as before: if the router returns invalid data, where to do a fix? There are two options: in the router itself or in the end-application to handle a bug in an application specific way. But not in between.

Chances are that AVM has fixed this meanwhile (home-assistant/core#151241 (comment)). A newer version of Fritz!OS (-> 8.21) has been released recently (at least for my 7590).

[DerEichler]

My FritzBox 6591 Cable just got an update to 8.21.
I had the parsing problem again yesterday and had my fritzconnect break due to that. I will monitor if the problem comes up again.