v2.4.0rc7 (#1010)

* Fix README instructions for embedded mode

* Expose sleep_loop

* [SshTunnel] WIP (#992)

[SshTunnel] WIP

* [Middleware] Capability in the core to allow custom client connection classes (#993)

* Move all TCP server related flags within `tcp_server.py` and also move the encryption functionality within TCP base server

* Templatize `BaseTcpServerHandler` which now expects a client connection object bound to `TcpClientConnection`.  This will allow for custom `HttpClientConnection` object in future to be used by `HttpProtocolHandler`

* Pass necessary flags to allow self-signed certificates

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix https integration tests

* Affected by #994

* Fix docs

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* [Middleware] `HttpClientConnection` preparation (#995)

* Turn usual suspects to warnings, not error

* Add `HttpClientConnection` skeleton

* Fix doc build

* Update references in http tests

* Make `work` core agnostic to work object construction by adding an abstract static method to `Work` interface called `create`

* Make mypy happy

* Fix tests broken due to change in how work objects are now constructed

* Doc ko bhi happy karo

* Bail out early for non-HTTP but HTTP looking protocols (#972)

* Add support in `Url` to parse all types of schemes

* .

* Guard handler against http looking protocol but not web or proxy requests

* Fix condition for web server protocol detection

* doc happy

* Update flags and type check imports only

* npm: bump eslint-plugin-import from 2.25.3 to 2.25.4 in /dashboard (#1005)

Bumps [eslint-plugin-import](https://github.com/import-js/eslint-plugin-import) from 2.25.3 to 2.25.4.
- [Release notes](https://github.com/import-js/eslint-plugin-import/releases)
- [Changelog](https://github.com/import-js/eslint-plugin-import/blob/main/CHANGELOG.md)
- [Commits](import-js/eslint-plugin-import@v2.25.3...v2.25.4)

---
updated-dependencies:
- dependency-name: eslint-plugin-import
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* npm: bump ws from 8.4.0 to 8.4.2 in /dashboard (#1007)

Bumps [ws](https://github.com/websockets/ws) from 8.4.0 to 8.4.2.
- [Release notes](https://github.com/websockets/ws/releases)
- [Commits](websockets/ws@8.4.0...8.4.2)

---
updated-dependencies:
- dependency-name: ws
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix broken `--local-executor` logic for windows ever since it was made default (#1008)

Co-authored-by: sowmyasudhasingh <sowmyasudhasingh@gmail.com>

* [Windows] `--threaded` mode integration tests works locally but fails on GHA (#1009)

* Enable remote threadless and threaded integration test for windows

* Run only threaded on windows

* Use powershell for execution of integration script on Windows

* Update test_integration.py

* Update test_integration.py

Co-authored-by: sowmyasudhasingh <sowmyasudhasingh@gmail.com>
Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com>

* Restrict request handling to `DEFAULT_ALLOWED_URL_SCHEMES` (#1002)

* Raise `HttpProtocolException` if request line scheme do not match `DEFAULT_ALLOWED_URL_SCHEMES`

* ignore WPS329

* Fix tests

* Pin to 4.3.2

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Test coverage for exception handling

* type ignore

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* [Doc] Threadless Remote vs Local Execution Mode (#1011)

* [Coverage] For newly added components (#1014)

* Add newly added code cov

* Fix spelling

* [Devtools] Build as part of GHA workflow (#1015)

* Fix devtools build

* Build devtools as part of GHA workflows

* [isort] Lib modules (#1016)

* isort `proxy.py` main class

* isort init and main

* isort common

* pre-commit fix

* isort dashboard and testing

* isort plugins

* isort core

* Only sort top level http py files

* isort http exception and websocket

* Remove proxy auth plugin from proxy package exports and force discover `PLUGIN_PROXY_AUTH` flags

* isort parser and web server

* no setattr

* isort all

* Enable pre-commit isort hook

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sowmya Sudha Singh <83529764+sowmya-jaxl@users.noreply.github.com>
Co-authored-by: sowmyasudhasingh <sowmyasudhasingh@gmail.com>

Author: 5 people

.github/workflows/test-library.yml

@@ -703,6 +703,11 @@ jobs:
         cd dashboard
         npm run build
         cd ..
+    - name: Build Devtools
+      run: |
+        cd dashboard
+        npm run devtools
+        cd ..
 
   developer:
     runs-on: ${{ matrix.os }}-latest

.gitignore

@@ -16,6 +16,7 @@ proxy.py.iml
 .vscode/*
 !.vscode/settings.json
 
+*.dot
 *.pyc
 *.egg-info
 *.csr
@@ -31,4 +32,5 @@ htmlcov
 dist
 build
 
+pyreverse.png
 profile.svg

.pre-commit-config.yaml

@@ -7,12 +7,12 @@ repos:
     args:
     - --py36-plus
 
-# - repo: https://github.com/timothycrosley/isort.git
-#   rev: 5.10.0
-#   hooks:
-#   - id: isort
-#     args:
-#     - --honor-noqa
+- repo: https://github.com/timothycrosley/isort.git
+  rev: 5.10.0
+  hooks:
+  - id: isort
+    args:
+    - --honor-noqa
 
 - repo: https://github.com/Lucas-C/pre-commit-hooks.git
   rev: v1.1.7
@@ -24,8 +24,7 @@ repos:
         helper/proxy\.pac|
         Makefile|
         proxy/common/pki\.py|
-        README\.md|
-        .+\.(plist|pbxproj)
+        README\.md
       $
 
 - repo: https://github.com/pre-commit/pre-commit-hooks.git
@@ -36,7 +35,6 @@ repos:
     exclude: |
       (?x)
       ^
-        \.github/workflows/codeql-analysis\.yml|
         dashboard/src/core/plugins/inspect_traffic\.json
       $
   - id: check-merge-conflict

Makefile

@@ -57,7 +57,7 @@ sign-https-certificates:
 	python -m proxy.common.pki sign_csr \
 		--csr-path $(HTTPS_CSR_FILE_PATH) \
 		--crt-path $(HTTPS_SIGNED_CERT_FILE_PATH) \
-		--hostname example.com \
+		--hostname localhost \
 		--private-key-path $(CA_KEY_FILE_PATH) \
 		--public-key-path $(CA_CERT_FILE_PATH)
 
@@ -169,11 +169,17 @@ lib-speedscope:
 			--open-file-limit 65536 \
 			--log-file /dev/null
 
+lib-pyreverse:
+	rm -f proxy.proxy.Proxy.dot pyreverse.png
+	pyreverse -ASmy -c proxy.proxy.Proxy proxy
+	dot -Tpng proxy.proxy.Proxy.dot > pyreverse.png
+	open pyreverse.png
+
 devtools:
-	pushd dashboard && npm run devtools && popd
+	pushd dashboard && npm install && npm run devtools && popd
 
 dashboard:
-	pushd dashboard && npm run build && popd
+	pushd dashboard && npm install && npm run build && popd
 
 dashboard-clean:
 	if [[ -d dashboard/public ]]; then rm -rf dashboard/public; fi