Support Multiple Comments in Packet Object #4798

bkayranci · 2025-07-18T21:55:14Z

The pcapng format specification explicitly allows for multiple comments to be associated with a packet via repeated opt_comment fields. This change brings Scapy closer to that standard, making it easier to work with multi-comment pcapng traces and enabling richer annotations in packet analysis.

Checklist:

If you are new to Scapy: I have checked CONTRIBUTING.md (esp. section submitting-pull-requests)
I squashed commits belonging together
I added unit tests or explained why they are not relevant
I executed the regression tests (using cd test && ./run_tests or tox)
If the PR is still not finished, please create a Draft Pull Request

…r consistency

…riter classes for back compatibility

bkayranci · 2025-07-20T04:32:53Z

Hi @gpotter2 , this is my first contribution to the repo. Could you please review this PR? Let me know if there's anything I should fix. Thanks!

codecov · 2025-07-21T05:39:13Z

Codecov Report

❌ Patch coverage is 91.66667% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.91%. Comparing base (b54fdb7) to head (6f2593c).
⚠️ Report is 19 commits behind head on master.

Files with missing lines	Patch %	Lines
scapy/utils.py	86.95%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4798      +/-   ##
==========================================
+ Coverage   80.08%   80.91%   +0.83%     
==========================================
  Files         365      366       +1     
  Lines       89027    90147    +1120     
==========================================
+ Hits        71296    72942    +1646     
+ Misses      17731    17205     -526

Files with missing lines	Coverage Δ
scapy/packet.py	`84.66% <100.00%> (+0.10%)`	⬆️
scapy/utils.py	`73.54% <86.95%> (-0.33%)`	⬇️

... and 48 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

bkayranci · 2025-07-24T05:03:04Z

Hi again @gpotter2, just following up on this PR. We have a need for this feature in our project, so I’d really appreciate a review when you have the time. Please let me know if any changes are needed. Thanks in advance!

gpotter2 · 2025-07-30T01:34:18Z

scapy/utils.py


-    def __init__(self, filename, fdesc=None, magic=None):  # type: ignore
-        # type: (str, IO[bytes], bytes) -> None
+    def __init__(self, filename, fdesc=None, magic=None, comments=None):  # type: ignore


This looks weird. I don't understand why you would be passing comments to the reader?

This was likely my oversight — it’s not actually necessary. I’ll go ahead and remove it.

gpotter2 · 2025-07-30T01:34:47Z

scapy/utils.py

@@ -1802,7 +1803,12 @@ def _read_options(self, options):
                        "%d !" % len(options))
                raise EOFError
            if code != 0 and 4 + length <= len(options):
-                opts[code] = options[4:4 + length]
+                if code in [1, 2988, 2989, 19372, 19373]:


What are those?

These are option codes defined in the pcapng file format docs, and it's possible for multiple codes to be present.

Got it. Could you add the link you just sent as a comment? Thanks.

Added it as a comment just now 👍

gpotter2 · 2025-07-30T01:35:46Z

scapy/utils.py

-        p.comment = comment
+        p.comments = comments
+        if p.comments is None:
+            p.comment = comment


I don't get this. I would expect to only set comments (?)

This part is included for backward compatibility. If comments is None, we assign comment = comment to ensure existing usage doesn't break.

If I understand it well you're already using a property for comment to take the last value of comments. this should already take care of compatibility, so I don't understand why it's necessary to put the two.

You're right. this isn't necessary anymore, so I'm removing it. Thanks for pointing it out!

gpotter2 · 2025-07-30T01:36:40Z

Thanks for the PR and sorry for the delay of review. I have a few questions regarding the code.
Cheers !

bkayranci · 2025-08-01T06:22:44Z

Thanks for the PR and sorry for the delay of review. I have a few questions regarding the code.
Cheers !

Let me know if there's anything else I should take care of before this can be merged. Appreciate your time!

gpotter2 · 2025-08-01T11:44:00Z

scapy/utils.py

@@ -2211,7 +2225,8 @@ def write_packet(self,
            comment=comment,


gpotter2 · 2025-08-01T11:44:18Z

scapy/utils.py

@@ -2194,6 +2207,7 @@ def write_packet(self,
            wirelen = caplen

        comment = getattr(packet, "comment", None)


gpotter2 · 2025-08-01T11:44:37Z

scapy/utils.py

@@ -2366,6 +2381,7 @@ def _write_packet(self,
                      comment=None,  # type: Optional[bytes]


gpotter2 · 2025-08-01T11:44:56Z

scapy/utils.py

@@ -2600,6 +2617,7 @@ def _write_packet(self,  # type: ignore
                      comment=None,  # type: Optional[bytes]


gpotter2 · 2025-08-01T11:46:36Z

I also personally would have kept comments in the same argument index as comment, rather than adding it at the end.

bkayranci · 2025-08-02T06:32:14Z

I also personally would have kept comments in the same argument index as comment, rather than adding it at the end.

That's a fair point, and I understand the value of maintaining positional consistency. In this case, since we changed both the name and the type (comment: Optional[str] to comments: Optional[List[bytes]]), we felt that reusing the same argument position might introduce subtle bugs or make mistakes harder to catch during refactoring. especially if any code still relies on the previous positional usage. Placing comments at the end was a cautious choice to reduce the risk of silent errors. Open to refactor if you think it’s worth aligning for clarity.

…tead

bkayranci · 2025-08-16T20:29:42Z

@gpotter2 I hope all is well. I wanted to gently follow up on my comment on the PR. I'd be grateful for your thoughts whenever you get a chance.

guedou

Thanks for the PR. I made a second review.

scapy/packet.py

scapy/utils.py

gpotter2 · 2025-08-22T06:05:43Z

LGTM although tests don't pass.

A change is still needed

bkayranci · 2025-08-22T11:14:33Z

LGTM although tests don't pass.

I fixed typing and linting issues.

I noticed a few # type: ignore lines — that makes sense here since mypy can’t infer the types, but according to the pcapng spec we know they are bytes or List[bytes]

…acking in PcapNgReader

guedou · 2025-08-28T15:42:21Z

scapy/utils.py

-                                             "comment", "ifname", "direction",
-                                             "process_information"])
+                                             "comments", "ifname", "direction",
+                                             "process_information", ])


Suggested change

"process_information", ])

"process_information"])

guedou · 2025-08-28T15:43:12Z

scapy/utils.py

@@ -1802,7 +1802,13 @@ def _read_options(self, options):
                        "%d !" % len(options))
                raise EOFError
            if code != 0 and 4 + length <= len(options):
-                opts[code] = options[4:4 + length]
+                # https://www.ietf.org/archive/id/draft-tuexen-opsawg-pcapng-05.html#name-options-format
+                if code in [1, 2988, 2989, 19372, 19373]:


Suggested change

if code in [1, 2988, 2989, 19372, 19373]:

if code in [1, 2988, 2989, 19372, 19373]: # opt_custom

guedou · 2025-08-28T15:44:27Z

scapy/utils.py

@@ -1876,11 +1888,13 @@ def _read_block_epb(self, block, size):

        process_information = {}
        for code, value in options.items():
-            if code in [0x8001, 0x8003]:  # PCAPNG_EPB_PIB_INDEX, PCAPNG_EPB_E_PIB_INDEX
+            # PCAPNG_EPB_PIB_INDEX, PCAPNG_EPB_E_PIB_INDEX


Please revert changes from line 1891 to 1898 as they are not needed for your PR.

guedou · 2025-08-28T15:44:52Z

scapy/utils.py

        epb_flags_raw = options.get(2, None)
-        if epb_flags_raw:
+        if epb_flags_raw and isinstance(epb_flags_raw, bytes):


Is this change needed?

guedou · 2025-08-28T15:45:55Z

@bkayranci please keep comments at the same location as comment in the methods.

bkayranci added 4 commits July 18, 2025 22:56

Rename 'comment' to 'comments' in Packet class and related classes fo…

5f55c44

…r consistency

Add support for 'comment' parameter in GenericPcapWriter and RawPcapW…

e1cc6f5

…riter classes for back compatibility

fix typo

0a17862

Add test and support for 'comment' parameter for back compatibility

eb68ff6

Refactor type annotations to comply with mypy type checking

f5e5c07

gpotter2 reviewed Jul 30, 2025

View reviewed changes

bkayranci added 3 commits July 30, 2025 09:46

Remove unnecessary code related to comments definition

38f490f

Remove redundant fallback to comment for compatibility handling

3b25874

add comment for option codes

e695901

gpotter2 reviewed Aug 1, 2025

View reviewed changes

Remove legacy 'comment' field when writing packet; use 'comments' ins…

89fd522

…tead

guedou reviewed Aug 17, 2025

View reviewed changes

remove casting and update comment property to take first item of list

b36cf2e

guedou reviewed Aug 21, 2025

View reviewed changes

scapy/utils.py Show resolved Hide resolved

gpotter2 previously approved these changes Aug 22, 2025

View reviewed changes

fix linting and typing issues

1fb4657

Reorder fields in PacketMetadata tuple for consistency and update unp…

6f2593c

…acking in PcapNgReader

guedou reviewed Aug 28, 2025

View reviewed changes

		@@ -2211,7 +2225,8 @@ def write_packet(self,
		comment=comment,

		@@ -2194,6 +2207,7 @@ def write_packet(self,
		wirelen = caplen

		comment = getattr(packet, "comment", None)

		@@ -2366,6 +2381,7 @@ def _write_packet(self,
		comment=None, # type: Optional[bytes]

		@@ -2600,6 +2617,7 @@ def _write_packet(self, # type: ignore
		comment=None, # type: Optional[bytes]

	if code in [1, 2988, 2989, 19372, 19373]:
	if code in [1, 2988, 2989, 19372, 19373]: # opt_custom

Uh oh!

Support Multiple Comments in Packet Object #4798

Are you sure you want to change the base?

Support Multiple Comments in Packet Object #4798

Conversation

bkayranci commented Jul 18, 2025

Uh oh!

bkayranci commented Jul 20, 2025

Uh oh!

codecov bot commented Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

bkayranci commented Jul 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gpotter2 commented Jul 30, 2025

Uh oh!

bkayranci commented Aug 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gpotter2 commented Aug 1, 2025

Uh oh!

bkayranci commented Aug 2, 2025

Uh oh!

bkayranci commented Aug 16, 2025

Uh oh!

guedou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gpotter2 commented Aug 22, 2025

Uh oh!

bkayranci commented Aug 22, 2025

Uh oh!

guedou Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Jul 21, 2025 •

edited

Loading

guedou Aug 28, 2025 •

edited

Loading