Idiotic results from httpx

[TheTechromancer]

When BBOT is set to proxy through Burpsuite, it seems that httpx discovers HTTPS URLs but not HTTP ones.

[liquidsec]

Having trouble replicating, do you have an example query?

[liquidsec]

On hold until issue can be replicated

[TheTechromancer]

Unable to replicate, closing.

[liquidsec]

Was able to replicate this, there seems to be two separate, but related issues:

1. False reporting of port 80 on https sites: https://www.blacklanternsecurity.com:80/
2. HTTP sites not being detected at all when running through burp

[TheTechromancer]

Running some tests this week, and starting to realize how bad this is. Httpx consistently misses URLs any time the web server issues a valid HTTP response on an HTTPS port (which practically every modern server and WAF does).

When given a target, it will output idiotic results:

[URL]               	http://aliexpress.com/	httpx	(dir, http-title-301-moved-permanently, in-scope, ip-47-246-173-237, status-301)
[URL]               	http://aliexpress.com:443/	httpx	(dir, http-title-301-moved-permanently, in-scope, ip-47-246-173-237, status-301)

http://aliexpress.com:443/?? There is no excuse for this. We should prioritize replacing httpx.

@liquidsec