Further false positive detection

[BBerastegui]

Hi!

I just wanted to drop here another use case that it'll be great to drop out from the results, marking it as a false positive.

During the nonexistent paths detection, it'll be great to test a random file with different extensions, as I've seen several cases where the response varies depending only on the extension appended. E.g.:

$ curl -s -o /dev/null -w "%{size_download}" http://[REDACTED]/error/1.html
14
$ curl -s -o /dev/null -w "%{size_download}" http://[REDACTED]/error/1.php
60

In this example, any request that ends in .html will have a size of 14 bytes, and any request that finishes in .php will have a size of 60 bytes.

It'll be great if the nonexistent detection routine could handle these cases too.

My two cents!

[BBerastegui]

For now, I've made a small script that re-processes the json output and groups the urls that return the same code, size, and flags into an array. The end result will look like this:

{
  "code": 200,
  "size": 3317,
  "is_directory": false,
  "is_listable": false,
  "found_from_listable": false,
  "url": [
    "http://[REDACTED]:80/aaa",
    "http://[REDACTED]:80/bbb",
    "http://[REDACTED]:80/ccc",
    "http://[REDACTED]:80/ddd",
..snip..

Just in case implementing a small "patch" like this will help and may be easier to do for starter.