feat: implement https server support #602

bliuchak · 2025-10-08T10:51:02Z

This PR implements HTTPS server support to proxy-chain.

flowchart LR
    subgraph HTTPS["Desired State"]
        direction LR
        C2[Client] -->|Encrypted HTTPS| P2[HTTPS Proxy] -->|HTTP/HTTPS| T2[Target]
    end
    subgraph HTTP["Current State"]
        direction LR
        C1[Client] -->|Cleartext HTTP| P1[HTTP Proxy] -->|HTTP/HTTPS| T1[Target]
    end

Readiness Checklist

Add ability to create either HTTP or HTTPS proxy server
Implement TLS overhead bytes counting for HTTPS proxy
Add tests (see Tests section)
Document limitations (see Known Limitations section)

Tests

TLS Overhead Analysis

Investigate the current state of TLS overhead in proxy-chain.

Investigate where TLS overhead might be possible (both in legacy and https implementation)
- Client -> Proxy TLS (HTTPS Main Proxy)
- Proxy -> Upstream TLS (HTTPS upstream)
- Client -> Target TLS (End-to-End via CONNECT); no explicit tracking needed since it's end-to-end
Verify TLS overhead tracked correctly for these cases
- ~~Forward via HTTPS upstream~~ (not implemented in master)
- ~~Chain via HTTPS upstream~~ (not implemented in master)
- Confirmed that TLS overhead for both cases tracked wrongly. See test: missing tls overhead for https upstream #607 for more details.

Places Where TLS Overhead Possible:

Client - Proxy TLS (HTTPS Main Proxy):
- direct.ts - CONNECT to HTTPS target, no upstream
- chain.ts - CONNECT via HTTP/HTTPS upstream
- chain_socks.ts - CONNECT via SOCKS upstream
- forward.ts - GET/POST to HTTP target (with or without upstream)
- forward_socks.ts - GET/POST via SOCKS upstream
- custom_response.ts - Custom response
- custom_connect.ts - Custom CONNECT connect
Proxy - HTTPS Upstream TLS (for both HTTP and HTTPS Main Proxy):
- chain.ts - CONNECT via HTTPS upstream
- forward.ts - GET/POST/etc via HTTPS upstream

Bugs

Extra bugs found and fixed during work on this PR.

e6ea53b

Known Limitations

Can't test HTTPS with IPv6 scenarios because of got-scraping limitations.
1. Scope: HTTPS implementation with IPv6 only. It's implemented but not tested. HTTP with IPv6 is both implemented and tested.
2. Impact: Only tests are affected.
3. Proposal A: replace got-scraping library (dev-dependency used only for testing) to something better which supports both Socks and IPv6
4. Proposal B: OR keep it as-is and fix this in future patch
TLS overhead for HTTPS upstreams. Should be fixed separately as legacy issue.
1. Scope: We're missing TLS overhead count for HTTPS upstreams. It's legacy issues and exists in current master. I've implemented TLS overhead count for HTTP upstreams in current PR.
2. Impact: LOW, proxy-chain can fully work with HTTP upstreams including routing traffic to HTTPS targets using CONNECT. Only HTTPS upstreams will be missing TLS overhead bytes in final statistic (rare usage).
3. Proposal: Merge this with TLS overhead count for HTTP upstreams (our main use-case). Solve TLS overhead for HTTPS upstream separately in future.

- Fix a datarace for error handler - Add a regression test that verify datarace fix - Add TLS defaults for better security

jirimoravcik · 2025-10-09T09:16:00Z

Also fixed:

a datarace for error event when we might log same events

fix for usage statistics trackings

Could you please point me to the changes that are related to the fixes? Thanks. Also, what was wrong with the statistics?

bliuchak · 2025-10-09T09:32:13Z

Also fixed:

a datarace for error event when we might log same events

fix for usage statistics trackings

Could you please point me to the changes that are related to the fixes? Thanks. Also, what was wrong with the statistics?

Also, what was wrong with the statistics?

Don't remember right now for 100%, but few tests failed for https scenarios. I believe there was some issues related with undefined values for statistics.

jirimoravcik

Looks good, had a few comments.
In addition to that, could you please:

Bump the package version
Describe all the new things in README.md (which serves as the primary user-facing documentation)
Thanks

test/utils/certificate_generator.js

src/server.ts

tsconfig.json

src/server.ts

Aslo revert changes in eslint and tsconfig

bliuchak · 2025-10-21T15:29:19Z

@jirimoravcik @lewis-wow Guys, I've added main logic for TLS overhead bytes. Please take a look 🙏

Gonna polish tests in meantime and push 'em ASAP.

src/server.ts

lewis-wow

Nice! Have nothing to add.

jirimoravcik

Found a few more points for discussion

jirimoravcik · 2025-10-27T08:53:37Z

src/server.ts

+        if (options.serverType === 'https') {
+            if (!options.httpsOptions) {
+                throw new Error('httpsOptions is required when serverType is "https"');
+            }
+
+            // Apply secure TLS defaults (user options can override)
+            // This prevents users from accidentally configuring insecure TLS settings
+            const secureDefaults: https.ServerOptions = {
+                ...HTTPS_DEFAULTS,
+                honorCipherOrder: true, // Server chooses cipher (prevents downgrade attacks)
+                ...options.httpsOptions, // User options override defaults
+            };
+
+            this.server = https.createServer(secureDefaults);
+            this.serverType = 'https';
+        } else {
+            this.server = http.createServer();
+            this.serverType = 'http';
+        }


Maybe validate if options.serverType is one of http, https? It would make it consistent with the type. I'd also set it to http by default in the constructor parameter. That way you could just do this.serverType = options.serverType

Good point. Gonna add this.

jirimoravcik · 2025-10-27T08:55:49Z

src/server.ts

+            socket.proxyChainErrorHandled = true;
+
+            // Log errors only if there are no user-provided error handlers
+            if (this.listenerCount('error') === 0) {


There was === 1 before, why is it === 0 now?

If I understand it correctly for previous condition this.listenerCount('error') === 1 (which count server-listeners, not socket) app log error only when there is one server handler. Right?

If I'm correct, then in this case the error will be handled by that one server-handler. For us it might be useful to log here when there are no other server-handlers this.listenerCount('error') === 0.

What do you think?

So it was broken before, right?

Yes, it was broken before. But I was wrong in my assumptions.

Original code should check socket.listenerCount('error') === 1 instead for this (server). Because socker and server error events aren't interconnected.

Also, we're attaching additional error listeners in our handlers (e.g. direct).

So, basically the flow should be like that:

onConnection() - attach out early handler - socket.listenerCount('error') === 1

onReqeust() / onConnect() - prepareRequestHandling

direct() function called - attaches sourceSocket error handler

Now socket.listenerCount('error') === 2

If error occurs after step 3: then the direct() handler will log instead our early handler.

If error occurs between steps 1-2: then our early handler will log it.

I'm gonna prepare fix for this.

@jirimoravcik fixed here e6ea53b

src/server.ts

test/https_edge_cases.js

+
+        it('handles upstream HTTPS proxy with expired certificate', async () => {
+            const expiredCert = loadCertificate('expired');
+            const validCert = loadCertificate('valid');


bliuchak added 4 commits October 8, 2025 10:52

feat: add https server implementation

313f535

fix: add onTLSClientError handler

e6adb19

- Fix a datarace for error handler - Add a regression test that verify datarace fix - Add TLS defaults for better security

fix: add https edge case tests

0158e0c

fix: resolve broken test with raw TCP and TLS

44fc388

bliuchak requested review from jirimoravcik and lewis-wow October 8, 2025 10:51

github-actions bot assigned bliuchak Oct 8, 2025

github-actions bot added this to the 125th sprint - Core Services team milestone Oct 8, 2025

github-actions bot added t-core-services Issues with this label are in the ownership of the core services team. tested Temporary label used only programatically for some analytics. labels Oct 8, 2025

bliuchak added the t-unblocking Issues with this label are in the ownership of the unblocking team. label Oct 8, 2025

bliuchak added 2 commits October 8, 2025 14:36

fix: skip fixture integrity checks for older node versions

8893eca

fix: beter dockerfile

57cb93e

jirimoravcik requested changes Oct 9, 2025

View reviewed changes

test/utils/certificate_generator.js Show resolved Hide resolved

src/server.ts Outdated Show resolved Hide resolved

src/server.ts Show resolved Hide resolved

lewis-wow requested changes Oct 9, 2025

View reviewed changes

tsconfig.json Outdated Show resolved Hide resolved

src/server.ts Outdated Show resolved Hide resolved

bliuchak added 5 commits October 10, 2025 15:40

chore: bump minor version

9956192

chore: extract https defaults into separate const

e5a1171

docs: add info about https server and other related stuff

8a90e7e

refactor: rewrite https example to js for consistency

de12161

Aslo revert changes in eslint and tsconfig

fix: export few user facing interfaces

4c036a0

bliuchak requested review from jirimoravcik and lewis-wow October 13, 2025 14:27

jirimoravcik approved these changes Oct 14, 2025

View reviewed changes

lewis-wow approved these changes Oct 14, 2025

View reviewed changes

fix: add tls overhead bytes into final statistic

934693a

bliuchak requested review from jirimoravcik and lewis-wow October 21, 2025 15:27

fix: better comment

597b942

jirimoravcik reviewed Oct 21, 2025

View reviewed changes

src/server.ts Show resolved Hide resolved

lewis-wow approved these changes Oct 22, 2025

View reviewed changes

jirimoravcik requested changes Oct 27, 2025

View reviewed changes

lewis-wow mentioned this pull request Oct 31, 2025

feat: add requestFinished event for per request tracking #604

Draft

jirimoravcik approved these changes Nov 4, 2025

View reviewed changes

fix: log error message on socket only when there are no other handlers

e6ea53b

github-code-quality bot found potential problems Nov 6, 2025

View reviewed changes

test/https_edge_cases.js

it('handles upstream HTTPS proxy with expired certificate', async () => {

const expiredCert = loadCertificate('expired');

const validCert = loadCertificate('valid');

bliuchak mentioned this pull request Nov 18, 2025

Add HTTPS Test Coverage for HTTP Agent Pooling #620

Open

scutuatua-crypto approved these changes Nov 21, 2025

View reviewed changes

feat: implement https server support #602

Are you sure you want to change the base?

feat: implement https server support #602

Conversation

bliuchak commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Readiness Checklist

Tests

TLS Overhead Analysis

Bugs

Known Limitations

Uh oh!

jirimoravcik commented Oct 9, 2025

Uh oh!

bliuchak commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jirimoravcik left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bliuchak commented Oct 21, 2025

Uh oh!

Uh oh!

lewis-wow left a comment

Choose a reason for hiding this comment

Uh oh!

jirimoravcik left a comment

Choose a reason for hiding this comment

Uh oh!

jirimoravcik Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

bliuchak Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

jirimoravcik Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

bliuchak Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

jirimoravcik Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

bliuchak Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

bliuchak Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

bliuchak commented Oct 8, 2025 •

edited

Loading

bliuchak commented Oct 9, 2025 •

edited

Loading