http module and pcap from FireEye appliance

[deresz]

I am trying to extract http objects via 'http -p 8023|http_extractor' on a packet capture from a proxy server. I pass the right port (8023) yet I can't get any content - as if no HTTP traffic was inside the pcap. Is this a bug or am my missing something ?

[Mraoul]

What's your full invocation? From what you've said this was data captured on a proxy server? If that's the case the proxy server should be making outbound connections to regular ports (port 80). Have you tried running it with the default port 80 or with both ports ( http -p "80,8023")?

[deresz]

Not really on the proxy server. It's data captured going towards the proxy server so there won't be any regular ports in there. And yes, I tried this one:

$ chopshop -f file.pcap 'http -p "80,8023"|http_extractor'
Starting ChopShop (Created by MITRE)
Initializing Modules ...
Initializing module 'http'
WARNING: No Parent for http providing sslim data
Initializing module 'http_extractor'
Running Modules ...
Shutting Down Modules ...
Shutting Down http
Shutting Down http_extractor
Module Shutdown Complete ...
ChopShop Complete

And yet there is HTTP data inside ...

[Mraoul]

Hmm, have you verified whether the http data is 1.0 or 1.1? I don't think the back-end code (libhtp) supports 2.0. If that's not it, it could be a bug somewhere -- If you'd be willing to share a pcap that exhibits the issue, I could take a look.

[wxsBSD]

Libhtp, and thus hype, do not support HTTP/2. If this is indeed HTTP/2 then there is nothing we can do.

[deresz]

Heh, I think i found the problem. Looks like it's not a proxy issue. I am working on PCAP files coming from FireEye alerts and, to my surprise, they don't contain any SYN packets. Yet, Wireshark does just fine with File -> Export Objects -> HTTP. I guess in this case there is not much you can do ?

[Mraoul]

:( libnids (what chopshop uses in the backend to process packets) needs to see the handshake. It's a known and very annoying limitation ...

[wxsBSD]

Someone(TM) should move us to a better implementation there. 😉