uartex - crc as a rx_footer - Crc16_mcrf4xx()

[rgumd]

Hello

I have unique situation when crc sum is a frame footer, there is no static frame termination.
Looks like crc sum should be calculated on every byte received (after the header) and then set as a rx_footer.
I assume that if I do not define rx_footer I can still receive data for sensors (after rx_timeout)?
I have (below) sample of code for crc calculation. Can somebody help with implementation?

I'm not receiving now any data in sensor objects (sensor/text_sensor). Will uartex work without defined rx_footer?

• frame dictionary: https://dzien.net/tech/index.php?baza=slownik
• frame decoder: https://dzien.net/tech/index.php

example frames:

0226FFF81699000015810000178C2805178D0000178E00000405000016EF000116C20001167F003216C3630A161400C11624000115AC00001680000116B90000170A000016E100000367000002F7000016EF00000218B3EB
0226FFF81699000015810000178C2805178D0000178E00000405000016EF000116C20001167F003216C3630A161400C31624000115AC002E1680000116B90000170A000016E100000367000002F7000016EF00000218E7F0
0226FFF81699000015810000178C2805178D0000178E00000405000015A7002316FF000515B7F830169E5035169F4B28157C0021157D00C8157E003C166E01A01616002D183C00001681F83016C100BB15CD000316200A3716210003158B00001587000015880000159B0000183D000016840000158900000218AD1E
0226FFF81699000015810000178C2805178D0000178E00000405000015A7002316FF000515B7F830169E5035169F4B28157C0021157D00CA157E003C166E01A11616002D183C00001681F83016C100BB15CD000316200A3816210003158B00001587000015880000159B0000183D0000168400001589000002185E35

my code:

uartex:
  tx_delay: 50ms
  tx_timeout: 500ms
  tx_retry_cnt: 3
  #tx_header: [0x42, 0x4D]
  #tx_checksum: add
  #tx_footer: []

  rx_timeout: 20ms
  rx_header: [0x02, 0x26, 0xFF, 0xF8]
  #rx_checksum: add
  #rx_footer: []

  version:
    disabled: True
  error:
    disabled: False
  log:
    disabled: True
text_sensor:
  - platform: uartex
    name: "Pump work type"
    icon: "mdi:pump"
    state: [0x15, 0xCD, 0x00]
    lambda: |-
      if (data[3] == 0x00) return "home heating";
      if (data[3] == 0x01) return "hot water";
      if (data[3] == 0x02) return "water/heating";
      if (data[3] == 0x03) return "summer";
      return "unknown";
sensor:
  - platform: uartex
    name: "water temperature"
    unit_of_measurement: "°C"
    device_class: "temperature"
    icon: "mdi:coolant-temperature"
    state: [0x16, 0x6E]
    state_number:
      offset: 2
      length: 2
      precision: 0

crc code example:

typedef unsigned short int ushort;

ushort Crc16_mcrf4xx(ushort *frame, int len)
{
    int i;
    ushort crc = 0xFFFF;

    while (len--)
    {
        crc ^= *frame++;
        for (i=0; i<8; i++)
        {
            if (crc & 0x0001) crc = (crc >> 1) ^ 0x8408;
            else crc = (crc >> 1);
        }
    }
    return crc;
};

[eigger]

If you register the checksum, the footer is not required.
I've added the checksum below — would you like to test it with this configuration?

  rx_checksum2: !lambda |-
    uint16_t crc = 0xFFFF;
    crc = crc16(crc, 0x8408, 0x02);
    crc = crc16(crc, 0x8408, 0x26);
    crc = crc16(crc, 0x8408, 0xFF);
    crc = crc16(crc, 0x8408, 0xF8);
    return  crc16_checksum(crc, 0x8408, data, len);

[rgumd]

Applied and tested.

All sensors returns: NA
Diagnostic -> Error: Checksum Error

I have UART debug enabled additionally.

My logs:

13:31:33[W][uartex:344]
[Read] Checksum error: 0226FFF81699000015810000178C2805178D0000178E00000405000015A7002316FF000515B7F830169E5035169F4B28157C0021157D00C9157E003C166E01E31616002D183C00001681F83016C100BB15CD000316200B0C16210003158B00001587000015880000159B0000183D0000168400001589000002181E26(124)
13:31:33[W][component:237]
Component uartex took a long time for an operation (184 ms).
13:31:33[W][component:238]
Components should block for at most 30 ms.
13:31:33[D][uart_debug:114]
<<< 02 26 FF F8 16 99 00 00 15 81 00 00 17 8C 28 05 17 8D 00 00 17 8E 00 00 04 05 00 00 15 A7 00 23 16 FF 00 05 15 B7 F8 30 16 9E 50 35 16 9F 4B 28 15 7C 00 21 15 7D 00 C9 15 7E 00 3C 16 6E 01 E3 16 16 00 2D 18 3C 00 00 16 81 F8 30 16 C1 00 BB 15 CD 00 03 16 20 0B 0C 16 21 00 03 15 8B 00 00 15 87 00 00 15 88 00 00 15 9B 00 00 18 3D 00 00 16 84 00 00 15 89 00 00 02 18 1E 26
13:31:36[W][uartex:344]
[Read] Checksum error: 0226FFF81699000015810000178C2805178D0000178E00000405000001F60000028E000002980000029900000245000004B50000159100001592000102090000020A000016F1000016F20000021844FD(80)
13:31:36[W][component:237]
Component uartex took a long time for an operation (113 ms).
13:31:36[W][component:238]
Components should block for at most 30 ms.
13:31:39[W][uartex:344]
[Read] Checksum error: 0226FFF81699000015810000178C2805178D0000178E000004050000021824B5(32)
13:31:39[W][component:237]
Component uartex took a long time for an operation (57 ms).
13:31:39[W][component:238]
Components should block for at most 30 ms.
13:31:45[W][uartex:344]
[Read] Checksum error: 0226FFF81699000015810000178C2805178D0000178E00000405000016EF000116C20001167F003216C3630A161400C51624000115AC00001680000116B90000170A000016E100000367000002F7000016EF000002184AB5(88)
13:31:45[W][component:237]
Component uartex took a long time for an operation (128 ms).
13:31:46[W][component:238]
Components should block for at most 30 ms.
13:31:46[D][uart_debug:114]
<<< 02 26 FF F8 16 99 00 00 15 81 00 00 17 8C 28 05 17 8D 00 00 17 8E 00 00 04 05 00 00 16 EF 00 01 16 C2 00 01 16 7F 00 32 16 C3 63 0A 16 14 00 C5 16 24 00 01 15 AC 00 00 16 80 00 01 16 B9 00 00 17 0A 00 00 16

[eigger]

It was found that 0218 before the checksum is not included in the checksum calculation. Please test again.

  rx_checksum2: !lambda |-
    uint16_t crc = 0xFFFF;
    crc = crc16(crc, 0x8408, 0x02);
    crc = crc16(crc, 0x8408, 0x26);
    crc = crc16(crc, 0x8408, 0xFF);
    crc = crc16(crc, 0x8408, 0xF8);
    return  crc16_checksum(crc, 0x8408, data, len - 2);

[rgumd]

When I have changed:
return crc16_checksum(crc, 0x8408, data, len);
to:
return crc16_checksum(crc, 0x8408, data, len - 2);
then my esphome on esp8266 is very instable, rebooting every few seconds and no logs are displayed
I have catched: "Last reset occurred too quickly; safe mode will be invoked in 5 restarts"

[eigger]

I'll test on the dummy device to see what issues might occur tomorrow.

[eigger]

I noticed the CRC16 calculation was implemented as the standard MSB-first CRC16, so I’ve updated it to the reflected LSB-first CRC16 (poly=0x8408). Would you like to test it again?

rx_checksum2: !lambda |-
    uint16_t crc = 0xFFFF;
    crc = crc16_reflected(crc, 0x8408, 0x02);
    crc = crc16_reflected(crc, 0x8408, 0x26);
    crc = crc16_reflected(crc, 0x8408, 0xFF);
    crc = crc16_reflected(crc, 0x8408, 0xF8);
    return crc16_reflected_checksum(crc, 0x8408, data, len - 2);

[rgumd]

nothing has changed
looks like a reboot every time a package is received
no logs
someone uses Crc16_mcrf4xx() with success, maybe it will help
example of project communicating with similar hardware: https://github.com/agirilovich/Tech_RS-mqtt-gateway

[eigger]

Please share your full configuration and more sample packet.
When I tested it, the sample packet you previously provided worked correctly.

0226FFF81699000015810000178C2805178D0000178E00000405000016EF000116C20001167F003216C3630A161400C11624000115AC00001680000116B90000170A000016E100000367000002F7000016EF00000218B3EB
0226FFF81699000015810000178C2805178D0000178E00000405000016EF000116C20001167F003216C3630A161400C31624000115AC002E1680000116B90000170A000016E100000367000002F7000016EF00000218E7F0
0226FFF81699000015810000178C2805178D0000178E00000405000015A7002316FF000515B7F830169E5035169F4B28157C0021157D00C8157E003C166E01A01616002D183C00001681F83016C100BB15CD000316200A3716210003158B00001587000015880000159B0000183D000016840000158900000218AD1E
0226FFF81699000015810000178C2805178D0000178E00000405000015A7002316FF000515B7F830169E5035169F4B28157C0021157D00CA157E003C166E01A11616002D183C00001681F83016C100BB15CD000316200A3816210003158B00001587000015880000159B0000183D0000168400001589000002185E35

[rgumd]

My full code for testing purposes and example frames.
when rx_checksum2 is defined then esp rebooting quickly end enters safe mode.
when rx_checksum2 is commented out I have starting point situation - sensors not receinving any data and have 'NA' state

esphome:
  name: techtest

esp8266:
  board: nodemcuv2

logger:
  level: DEBUG

wifi:
  fast_connect: true
  networks:
    - ssid: !secret wifi_networks_ssid
      password: !secret wifi_networks_password

ota:
  password: !secret ota_password
  platform: esphome

captive_portal:
web_server:
  port: 80
  version: 3

status_led:
  pin:
    number: GPIO2
    inverted: true

external_components:
  - source: github://eigger/espcomponents@latest
    components: [ uartex ]
    refresh: always

uart:
  id: uart_bus
  rx_pin: D5
  tx_pin: D6
  baud_rate: 9600

uartex:
  tx_delay: 50ms
  tx_timeout: 500ms
  tx_retry_cnt: 3
  rx_timeout: 20ms
  rx_header: [0x02, 0x26, 0xFF, 0xF8]
  rx_checksum2: !lambda |-
    uint16_t crc = 0xFFFF;
    crc = crc16_reflected(crc, 0x8408, 0x02);
    crc = crc16_reflected(crc, 0x8408, 0x26);
    crc = crc16_reflected(crc, 0x8408, 0xFF);
    crc = crc16_reflected(crc, 0x8408, 0xF8);
    return crc16_reflected_checksum(crc, 0x8408, data, len - 2);

  version:
    disabled: True
  error:
    disabled: False
  log:
    disabled: False

binary_sensor:
  - platform: uartex
    name: "water pump"
    state: [0x15, 0x8B]
    state_on:
      offset: 2
      data: [0x01]
    state_off:
      offset: 2
      data: [0x00]

text_sensor:
  - platform: uartex
    name: "Pump work type"
    state: [0x15, 0xCD]
    lambda: |-
      if (data[3] == 0x00) return "home heating";
      if (data[3] == 0x01) return "hot water";
      if (data[3] == 0x02) return "water/heating";
      if (data[3] == 0x03) return "summer";
      return "unknown";

sensor:
  - platform: uartex
    name: "water temperature"
    state: [0x16, 0x6E]
    state_number:
      offset: 2
      length: 2
      precision: 0
  - platform: uartex
    name: "valve temperature"
    state: [0x16, 0x14]
    state_number:
      offset: 2
      length: 2
    

0226FFF81699000015810000178C2805178D0000178E00000405000015A7002316FF000515B7F830169E5035169F4B28157C0021157D00C9157E003C166E01C01616002D183C00001681F83016C100BB15CD000316200E1716210003158B00001587000015880000159B0000183D000016840000158900000218E8E8
0226FFF81699000015810000178C2805178D0000178E00000405000015A7002316FF000515B7F830169E5035169F4B28157C0021157D00C8157E003C166E01C51616002D183C00001681F83016C100BC15CD000316200E1B16210003158B00001587000015880000159B0000183D0000168400001589000002185285
0226FFF81699000015810000178C2805178D0000178E00000405000015A7002316FF000515B7F830169E5035169F4B28157C0021157D00C8157E003C166E01C31616002D183C00001681F83016C100BB15CD000316200E1C16210003158B00001587000015880000159B0000183D000016840000158900000218B765
0226FFF81699000015810000178C2805178D0000178E00000405000015A7002316FF000515B7F830169E5035169F4B28157C0021157D00C9157E003D166E01C61616002D183C00001681F83016C100BB15CD000316200E1C16210003158B00001587000015880000159B0000183D000016840000158900000218A005
0226FFF81699000015810000178C2805178D0000178E00000405000015A7002316FF000515B7F830169E5035169F4B28157C0021157D00C8157E003D166E01C51616002D183C00001681F83016C100BB15CD000316200E1E16210003158B00001587000015880000159B0000183D000016840000158900000218ACD1
0226FFF81699000015810000178C2805178D0000178E00000405000016EF000116C20001167F003216C3630A161400C51624000115AC00001680000116B90000170A000016E100000367000002F7000016EF000002184AB5
0226FFF81699000015810000178C2805178D0000178E00000405000016EF000116C20001167F003216C3630A161400C61624000115AC00001680000116B90000170A000016E100000367000002F7000016EF000002188C45
0226FFF81699000015810000178C2805178D0000178E00000405000001F60000028E000002980000029900000245000004B50000159100001592000102090000020A000016F1000016F20000021844FD
0226FFF81699000015810000178C2805178D0000178E00000405000016EF000316C2000016EF0000021832AA

[eigger]

I still don't know exactly why your ESP8266 keeps rebooting in a loop.
For now, try using it this way as a temporary workaround.

uartex:
  tx_delay: 50ms
  tx_timeout: 500ms
  tx_retry_cnt: 3
  rx_timeout: 20ms
  rx_header: [0x02, 0x26, 0xFF, 0xF8]
  rx_footer: [0x00, 0x00, 0x02, 0x18]

return crc16_reflected_checksum(crc, 0x8408, data, len); is fine,
but I don't understand why return crc16_reflected_checksum(crc, 0x8408, data, len - 2); is causing an issue.
I'll take a closer look.

[rgumd]

when I'll set the rx_footer then crc16_reflected_checksum(crc, 0x8408, data, len - 2) NOT causing esp to reset.
but both cases (len or len-2) returns Checksum Error and sensors have NA values.
I don't know why I didn't see before that there is a frame footer defined.

For testing, when I'll ignore rx_checksum2 and set proper rx_header and rx_footer I expecting to have working solution.
No errors are reported but sensors have NA value
Below example shows nothing, what is strange, should always show a or b.
Frame data is not processed?

text_sensor:

• platform: uartex
  state: [0x15, 0x81]
  name: "test"
  lambda: |-
  if (data[2] == 0x01) return "a";
  return "b";

[eigger]

This condition is used to parse packets with a specific pattern.
0226FFF81581xxxx

When analyzing actual packets, you need to apply an additional offset.
0226FFF8169900001581xxxx

header offset 0...1....2...3...4
02 26 FF F8 16 99 00 00 15 81 xxxx

text_sensor:
  - platform: uartex
    state: 
      data: [0x15, 0x81]
      offset: 4
    name: "test"
    lambda: |-
      if (data[6] == 0x01) return "a";
      return "b";

or

text_sensor:
  - platform: uartex
    state: 
      data: [0x00, 0x00, 0x00, 0x00, 0x15, 0x81]
      mask: [0x00, 0x00, 0x00, 0x00, 0xFF, 0xFF]
    name: "test"
    lambda: |-
      if (data[6] == 0x01) return "a";
      return "b";

[rgumd]

The examples given on small data frames are misleading.
"offset" always counts from the beginning of the whole frame (counted from right after rx_header), I mistakenly assumed that offset is relative to each time defined "data".
Now I have already successfully read the first data values.
I'll move to esp32 in a while, I'm betting that my reboot problems are due to small resources in esp8266.

Thank you for your help!
Your project is extremely versatile and useful.

[eigger]

I'm glad it worked out. Let me know if you have any other questions.