fix: deduplicate MeshCore client retransmissions in room server#4
Open
fix: deduplicate MeshCore client retransmissions in room server#4
Conversation
Clients retry a message when they don't receive an ack fast enough (common for distant nodes with weak signal). Without deduplication every retry lands as a separate message in the room. Add an in-memory `_recent_posts` cache keyed on (pubkey_hex, message_text). Any identical (author, text) pair arriving within DEDUP_WINDOW_SECS (30s) is silently dropped. Cache is evicted when it grows past 500 entries to bound memory usage. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pymc_core decodes incoming bytes with errors="replace", so invalid
trailing bytes (common in C-string firmware payloads) become U+FFFD (□).
The MeshCore client retransmits a message up to 3 times when no ACK is
received; without this fix "Test" and "Test□" were treated as different
strings and all three retransmissions were stored as separate messages.
Extend the existing rstrip("\x00") to also strip "�" so all
retransmissions of the same message produce identical normalised text
before being handed to add_post() for dedup comparison.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
7 tests in TestAddPostDedup: - First post is stored normally - Identical (author, text) within 30s window is dropped - Same text accepted again after the window expires - Different text from the same author always accepted - Same text from different authors both stored independently - sender_timestamp=0 (web-API path) dedup within window - sender_timestamp=0 (web-API path) accepted after window Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…t.py
The rstrip("\x00�").rstrip() was only applied in text.py, which
handles radio-received packets. Messages posted via the web UI go
directly to add_post() and were never normalised, so trailing newlines
from the textarea were persisted to the DB and rendered as visible
whitespace in the room messages UI.
Move the normalisation to the top of add_post() so it applies to every
code path: radio, web UI, companion, and CLI.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pymc_coredecodes bytes witherrors="replace", turning invalid trailing bytes into U+FFFD (□). The existingrstrip("\x00")didn't strip □, so "Test" and "Test□" had different dedup keystext.py): extend rstrip to also strip□so all retransmissions normalise to identical textroom_server.py): in-memory_recent_postscache keyed on(author_pubkey_hex, normalised_text), identical (author, text) within 30 s is droppedTest plan
pytest tests/test_room_server_dedup.py: 7 tests, all pass