observe stopped logging data

[technicallyfeasible]

About a week ago I stopped seeing taptap data events in homeassistant. I use the mqtt wrapper around taptap:
https://github.com/litinoveweedle/taptap-mqtt

When I investigated I ran the taptap executable locally connecting directly to the cca serial port.

If I run taptap observe --serial COM1 I see no data being received.

However, there is data on the bus and I receive these frames:

taptap peek-frames --serial COM1

Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 255, 24, 64, 135] }
Frame { address: To(GatewayID(0x1201)), frame_type: Type::RECEIVE_REQUEST, payload: [0, 1, 237, 24, 4] }
Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 255, 24, 64, 137] }
Frame { address: To(GatewayID(0x1201)), frame_type: Type::RECEIVE_REQUEST, payload: [0, 1, 237, 24, 4] }
Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 255, 24, 64, 139] }
Frame { address: To(GatewayID(0x1201)), frame_type: Type::RECEIVE_REQUEST, payload: [0, 1, 237, 24, 4] }
Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 255, 24, 64, 141] }
Frame { address: To(GatewayID(0x1201)), frame_type: Type::RECEIVE_REQUEST, payload: [0, 1, 237, 24, 4] }
Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 255, 24, 64, 143] }
Frame { address: To(GatewayID(0x1201)), frame_type: Type::RECEIVE_REQUEST, payload: [0, 1, 237, 24, 4] }
Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 255, 24, 64, 145] }
Frame { address: To(GatewayID(0x1201)), frame_type: Type::RECEIVE_REQUEST, payload: [0, 1, 237, 24, 4] }
Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 255, 24, 64, 147] }
Frame { address: To(GatewayID(0x1201)), frame_type: Type::RECEIVE_REQUEST, payload: [0, 1, 237, 24, 4] }
Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 255, 24, 64, 149] }
Frame { address: To(GatewayID(0x1201)), frame_type: Type::RECEIVE_REQUEST, payload: [0, 1, 237, 24, 4] }
Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 255, 24, 64, 151] }
Frame { address: To(GatewayID(0x1201)), frame_type: Type::RECEIVE_REQUEST, payload: [0, 1, 237, 24, 4] }
Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 224, 0, 14, 0, 1, 2, 0, 237, 24, 64, 153] }
Frame { address: To(GatewayID(0x1201)), frame_type: Type::RECEIVE_REQUEST, payload: [0, 1, 237, 24, 4] }
Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 255, 24, 64, 156] }
Frame { address: To(GatewayID(0x1201)), frame_type: Type::RECEIVE_REQUEST, payload: [0, 1, 237, 24, 4] }
Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 255, 24, 64, 158] }
Frame { address: To(GatewayID(0x1201)), frame_type: Type::RECEIVE_REQUEST, payload: [0, 1, 237, 24, 4] }
Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 255, 24, 64, 160] }
Frame { address: To(GatewayID(0x1201)), frame_type: Type::RECEIVE_REQUEST, payload: [0, 1, 237, 24, 4] }
Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 255, 24, 64, 162] }
Frame { address: To(GatewayID(0x1201)), frame_type: Type::RECEIVE_REQUEST, payload: [0, 1, 237, 24, 4] }
Frame { address: From(GatewayID(0x1201)), frame_type: Type::RECEIVE_RESPONSE, payload: [65, 255, 24, 64, 164] }

Looking through the protocol description it does look like the frames use a different identifier and maybe can't be decoded anymore. I don't know whether there was a CCA update or something else happened as this is all very intransparent.
Tigo data transmission to the cloud is working normally and I can see the usage data there.

[litinoveweedle]

Could you please post the output of the peek-bytes mode?

[technicallyfeasible]

Here is an excerpt:

00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 FF 7A C4 2D 9E 51 7E 08
FF 00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 FF 7A C4 2F 8C 72 7E 08
FF 00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 FF 7A C4 31 73 8B 7E 08
FF 00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 FF 7A C4 33 61 A8 7E 08
FF 00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 FF 7A C4 35 57 CD 7E 08
FF 00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 FF 7A C4 37 45 EE 7E 08
FF 00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 FF 7A C4 39 3B 07 7E 08
FF 00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 E0 00 0E 00 01 02 00 3A 7A C4 3B 03 9E 7E 08
FF 00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 FF 7A C4 3E 84 73 7E 08
FF 00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 FF 7A C4 40 7D E9 7E 08
FF 00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 FF 7A C4 42 6F CA 7E 08
FF 00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 FF 7A C4 44 59 AF 7E 08
FF 00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 FF 7A C4 45 D0 BE 7E 08
FF 00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 FF 7A C4 47 C2 9D 7E 08
FF 00 FF FF 7E 07 12 01 01 48 00 01 3A 7A 04 84 59 7E 08
FF 7E 07 92 01 01 49 01 FF 7A C4 49 BC 74 7E 08

[litinoveweedle]

Can you please check FW version on the Tigo CCA? This seems to be similar to the issue discussed here #17 . (the unconfirmed root cause seems to be CCA upgrade FW to v4.0.0)

[litinoveweedle]

Would it be possible to get ~1 min long peek-bytes capture during the day (you shall see longer messages - including power reports)

[technicallyfeasible]

Can you please check FW version on the Tigo CCA? This seems to be similar to the issue discussed here #17 . (the unconfirmed root cause seems to be CCA upgrade FW to v4.0.0)

I was looking for a way to do this. Where can I find the FW version?

Here is a ~1min capture of bytes data

bytes.txt

[technicallyfeasible]

Regarding firmware, in the Tigo app when I go to settings then I now get a button "Device update required"but nowhere does it show the current and new versions. I haven't clicked it yet.

Edit: I found it using the device api (/cgi-bin/mobile_api?cmd=DEVICE_INFO&dev=1&ver=1). I am on 4.0.1-ct

[litinoveweedle]

Thank you, your findings support the working theory of FW 4.X.X breaking the taptap.

[technicallyfeasible]

If I can help in any way to fix this let me know. I can also run more captures.
I also saw the discussion about node addresses being parsed incorrectly. I have tried to get these in the past running observe for days and never seeing any node tables, now that makes sense.

[litinoveweedle]

Thank you, I hope that @willglynn will be able to take a look and if there will be anything we will ask.

P.S.: I fixed the node addreses/serial/barodes parsing in my fork of taptap, it will be delivered inside of the latest HA-addon very soon, probably today, and there is PR #19 open to get this fix back to this upstream. Unfortunately this will not help you right now, as first the protocol change need to be addresses.

[willglynn]

This doesn't look too bad. Zooming in on line 234, adding newlines for clarity:

FF 7E 07 92 01 01 49 81 FE 04 CA E0 DB
31 00 0C 00 5A 00 0F 38 91 C2 FF 4D 10 E6 92 00 64 DF 40 B2 02 00
31 00 0C 00 5A 00 0F 38 61 C0 FF 4D F0 E6 92 00 64 E0 D0 B2 02 00
31 00 07 00 18 08 0F 38 D1 C4 FF 4E 00 E6 92 00 64 E0 D0 AC 02 00
31 00 07 00 18 08 0F 38 F1 C5 FF 4D 20 E6 92 00 64 DF 40 AC 02 00
2B FB 7E 08

This is a link layer 01 49 receive response which is different, but in a way that's completely unambiguous: status type is 81 FE rather than 00 E0…00 FF. Don't know what 04 CA E0 DB…2B mean yet, but in the middle we have four normal PV network layer packets, all type 31:

         Packet: 31 00 0C 00 5A 00 0F 38 91 C2 FF 4D 10 E6 92 00 64 DF 40 B2 02 00
           Type: 31
     PV node ID:    00 0C
  Short address:          00 5A
            DSN:                00
    Data length:                   0F
           Data:                      38 91 C2 FF 4D 10 E6 92 00 64 DF 40 B2 02 00

Type 31 is a power report which is expected. These power reports are 15 bytes rather than 13 bytes as before, and it looks to me like they were extended but are otherwise compatible:

                    38 91 C2 FF 4D 10 E6 92 00 64 DF 40 B2 02 00
        Voltage in: 38 9                                         0x2B6 = 905 * 0.05V = 45.25V
       Voltage out:     1 C2                                     0x1C2 = 450 * 0.10V = 45.0V
  DC-DC duty cycle:          FF                                  0xFF = 255 = 100%
        Current in:             4D 1                             0x4D1 = 1233 * 0.005A = 6.165A
       Temperature:                 0 E6                         0x0E6 = 230 * 0.1ºC = 23.0ºC
               ???:                      92 00 64                
      Slot counter:                               DF 40          timestamp referenced to gateway
              RSSI:                                     B2       0xB2 = 178
               ???:                                        02 00

[litinoveweedle]

So we have some change on the PV level (so far seen only in power report) and than we have something added to the link layer. To me there needs to be some reason behind it. You would not go for the firmware upgrade of the CCA and gateway just for fun. It can break on some installations, you would have to support it, RMAs... so what is the reason?

[litinoveweedle]

Also user already reported, that there is already other firmware update waiting... so fix, or something needed to be upgraded in two phases?

[willglynn]

I think the second byte of the read response status type might be the same as before, since when there's no frames being exchanged, there's a familiar FF, FF, FF, …, E0 cadence. The first byte of the status type had always been zero, but now it goes from 81 to C1 partway through.

Note how the slot counter approaching EE DF then wrapping around to 00 00 corresponds to 81 incrementing to C1:

                     vv                            vv vv
FF 7E 07 92 01 01 49 81 E0 00 0E 00 01 02 00 1E 9A EE 97 F8 75 7E 08
FF 7E 07 92 01 01 49 81 E0 00 0E 00 01 02 00 1E 9A EE B7 FA 54 7E 08
FF 7E 07 92 01 01 49 81 E0 00 0E 00 01 02 00 1E 9A EE D6 75 26 7E 08
FF 7E 07 92 01 01 49 C1 E0 00 0E 00 01 02 00 1E 9A 00 18 EE 2D 7E 08
FF 7E 07 92 01 01 49 C1 E0 00 0E 00 01 02 00 1E 9A 00 39 65 1D 7E 08
                     ^^                            ^^ ^^

I would benefit from a longer capture, but I also think it's worth trying to update taptap to decode the data we have here.

[litinoveweedle]

Yes I would agree that it would be nice to update taptap to be able to decode what we know - so users can use it. Shall modify something on the PR so you can merge it into the new release as well?

[litinoveweedle]

@technicallyfeasible could we please ask you for the additional logs?

1. longer report during the day (as requested by @willglynn)
2. could you please try to run my latest version of the taptap in the observe mode during the night to confirm, that it will produce 'infrastructure event' - including barcodes/serials. Please note that you have to be patient for those, those are reported only time to time and at least in my case strictly after sun set.