How to make tamper.dev show all request headers?

[p-]

Hi @sirdarckcat

First of all: thanks for Tamper Dev!

I quickly tested tamper.dev and did not find out why it's not showing all expected request headers:

Using an up-to-date Chrome on macOS and the newest Tamper Dev version (2.0.7), Tamper Dev is listing only 4 request headers when visiting github.com (scrolling included):

• Accept
• Cookie
• Upgrade-Insecure-Requests
• User-Agent

The browser itself sends a lot more headers of course (e.g. Cache-Control, Connection, Accept-Encoding etc.)

Is this some kind of UI issue?

Greetings

@p-

[sirdarckcat]

Hi!

@p- I think this is the scrollbar hiding the other requests, can you check what happens when you click on User-Agent and then press down arrow?

[sirdarckcat]

if this is the problem, follow #138 for the UI change that would fix this

[sirdarckcat]

#156 should hopefully resolve this

[p-]

Hi @sirdarckcat

I think I've found the problem. Tamper Dev is not receiving those headers from the Chrome API:

From the chrome.webRequest Doc:

The following headers are currently not provided to the onBeforeSendHeaders event.
This list is not guaranteed to be complete nor stable.

• Authorization
• Cache-Control
• Connection
• Content-Length
  [...]

Accept-Encoding is not listed but seems also to be excluded.

[sirdarckcat]

I'm using the debugger API, it's a different one. I see more than 4 headers on github.com

[sirdarckcat]

You tried this on https://github.com right?

[p-]

I'm using the debugger API, it's a different one. I see more than 4 headers on github.com

Ah yes, I see I found the reference to webRequest.onBeforeSendHeaders in the v1-Folder of tamperchrome, but the recent source code of Tamper Dev is in the v2-folder I assume?
Do you see the Connection, Cache-Control etc headers?

You tried this on https://github.com right?

Yes. In this screenshot I set a break point very early. You can see that the url is https://github.com, but the debugee seems to receive only 4 headers.

I also tried it with a recent Chromium build from download-chromium. With that version I see two additional user agent hints related headers sec-ch-ua and sec-ch-ua-mobile, but still no more than 6 headers.

[sirdarckcat]

Oh, maybe it's a cached response. Can you try reloading (hard reload)?

[sirdarckcat]

ah, nevermind. I just noticed you have the non-logged-in version of github in your screenshot. I can reproduce now :-)

indeed the browser only seems to send 6 request headers, and I reproduced by going to https://httpbin.org/headers and comparing both, the ones the server sees and the ones TamperDev sees. There is indeed a discrepancy.

[sirdarckcat]

httpb.in says:

{
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9", 
    "Accept-Encoding": "gzip, deflate, br", 
    "Accept-Language": "en-GB,en;q=0.9,de-CH;q=0.8,de;q=0.7,en-US;q=0.6,es-MX;q=0.5,es;q=0.4,zh-CN;q=0.3,zh;q=0.2", 
    "Cache-Control": "max-age=0", 
    "Host": "httpbin.org", 
    "Sec-Ch-Ua": "\"Google Chrome\";v=\"87\", \" Not;A Brand\";v=\"99\", \"Chromium\";v=\"87\"", 
    "Sec-Ch-Ua-Mobile": "?0", 
    "Sec-Fetch-Dest": "document", 
    "Sec-Fetch-Mode": "navigate", 
    "Sec-Fetch-Site": "none", 
    "Sec-Fetch-User": "?1", 
    "Upgrade-Insecure-Requests": "1", 
    "User-Agent": "Mozilla/5.0 (X11; CrOS x86_64 13505.73.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.109 Safari/537.36", 
  }
}

DevTools says:

accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
accept-encoding: gzip, deflate, br
accept-language: en-GB,en;q=0.9,de-CH;q=0.8,de;q=0.7,en-US;q=0.6,es-MX;q=0.5,es;q=0.4,zh-CN;q=0.3,zh;q=0.2
cache-control: max-age=0
sec-ch-ua: "Google Chrome";v="87", " Not;A Brand";v="99", "Chromium";v="87"
sec-ch-ua-mobile: ?0
sec-fetch-dest: document
sec-fetch-mode: navigate
sec-fetch-site: none
sec-fetch-user: ?1
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (X11; CrOS x86_64 13505.73.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.109 Safari/537.36

and TamperDev says:

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; CrOS x86_64 13505.73.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.109 Safari/537.36
sec-ch-ua: "Google Chrome";v="87", " Not;A Brand";v="99", "Chromium";v="87"
sec-ch-ua-mobile: ?0

I suspect the reason is because TamperDev is injected early on in the request, so the browser doesn't know yet it will send those other headers, but it seems possible to edit those headers:

    "Accept-Encoding": "foobar", 

Host doesn't seem to be editable, though. I think this isn't really possible to change because Chrome doesn't know yet what is going to happen to the request later on in the stack. On the upside, we can notify the user that these headers are sent (after they are sent, though). Not sure if that's useful, but we can make it happen.

[sirdarckcat]

Track this in #159

[p-]

Thanks for checking! I'm happy it's not just my browser 😉

ah, nevermind. I just noticed you have the non-logged-in version of github in your screenshot. I can reproduce now :-)

Yeah a sample with httpbin probably would have been better 😄 .
But at first I thought it was just a simple UI/configuration issue.

but it seems possible to edit those headers

That's true. So it's not a big issue.

On the upside, we can notify the user that these headers are sent (after they are sent, though). Not sure if that's useful, but we can make it happen.

Yeah I'm also not sure if that would be useful. 🤔
Maybe a note on the bottom of the request header list that says "not all headers sent are displayed". Because that's what confused me at first: "Why are only 4 request headers shown, sure there must be more?"
I think it's OK if there are limitations but the users typically want to know them. I once almost missed a vulnerability because the tool I used to perform requests encoded the URLs differently than it displayed them.

Host doesn't seem to be editable, though.

I think it's absolutely OK that host is not editable. Seems out of scope for most tamper use cases. (But it's also good if the users know about this limitation.)

[sirdarckcat]

URLs are also likely encoded differently in the extension :-).

Will think about how to present it.

[p-]

Yeah, I kinda expect that from a browser-based tool as the browser itself normalizes URLs.
If one needs an URL path that's not altered it's best to use something like curl's --path-As-Is. (helps for testing path traversal vulnerabilities etc.)