Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
243 commits
Select commit Hold shift + click to select a range
e47d09d
Start for bpmux replacement
patuwwy Jan 10, 2023
3a8104e
tc-verser Switch to connect method
patuwwy Jan 10, 2023
6a839fd
Rename to tecemux
patuwwy Jan 10, 2023
de371b8
DecodedFrame type
patuwwy Jan 11, 2023
f2ab883
Flags parsers
patuwwy Jan 12, 2023
e8796c6
Flags utils
patuwwy Jan 13, 2023
9c2baf8
push data to channel
patuwwy Jan 15, 2023
ef4060e
Duplex communication
patuwwy Jan 16, 2023
0bb7f96
Send stdin over frames
patuwwy Jan 16, 2023
ea3cd11
Issue, stream paused
patuwwy Jan 16, 2023
258d9db
Debug AbortError
patuwwy Jan 17, 2023
6c90a6c
Add handlers
patuwwy Jan 18, 2023
1f6bb57
Ack on, alternate stdin to cli/srv
MichalCz Jan 18, 2023
9becfad
Issue. Decoder receives more than 1 frame
patuwwy Jan 18, 2023
5deef5a
Read buffer of frames to decode
patuwwy Jan 18, 2023
097e632
Cleanup, common sequenceNumber for TeceMux instance
patuwwy Jan 19, 2023
dedf56d
Send FIN
patuwwy Jan 19, 2023
d6a636e
Replace BPMux with TeCeMux
patuwwy Jan 20, 2023
d4d9cea
Cleanup
patuwwy Jan 20, 2023
ae9f94b
Playground pass file
patuwwy Jan 25, 2023
a3a8b27
Pass big file
patuwwy Jan 25, 2023
26079ec
Calc/check checksum
patuwwy Jan 28, 2023
8300d04
FramesKeeper
patuwwy Jan 29, 2023
1408c72
Check for ACK of last frames send
patuwwy Jan 30, 2023
63f1057
Fix send FIN
patuwwy Jan 31, 2023
21842ea
Send file both directions
patuwwy Jan 31, 2023
540dedf
Debug w/o FrameKeeper
patuwwy Feb 2, 2023
08b0b54
FramesKeeper generator
patuwwy Feb 9, 2023
d991a6c
Cleanup
patuwwy Feb 10, 2023
ca0a494
Tests
patuwwy Feb 14, 2023
2809cc5
Replace BPMux with TeCeMux
patuwwy Jan 20, 2023
03bd54c
R->STH. WIP
patuwwy Mar 23, 2023
7d3be52
R<->STH
patuwwy Mar 28, 2023
c285ad1
TeCeMux wip
patuwwy Apr 3, 2023
d613203
fix channels _id
patuwwy Apr 6, 2023
d89275a
Test defer
patuwwy Apr 6, 2023
3d4535d
CI test
patuwwy Apr 11, 2023
ab72593
Remove debug listeners
patuwwy Apr 11, 2023
78a7f7d
Runner host-client destroy
patuwwy Apr 12, 2023
25b58f7
nextTick before sequence call
patuwwy Apr 12, 2023
81ce3f7
TeceMuxChannel class
patuwwy Apr 13, 2023
9ed55fa
Disable nagle in R-H
patuwwy Apr 13, 2023
b912447
Simplify TecemuxChannel
patuwwy Apr 19, 2023
56bbb26
Merge pull request #848 from scramjetorg/feature/demux-sth-r2
patuwwy Apr 19, 2023
34636c8
Fix lint
patuwwy Apr 20, 2023
15841a9
Restore TCM channel implementation for older nodejs
patuwwy Apr 20, 2023
16c05fb
Fix M-S connection
patuwwy Apr 26, 2023
daf4851
Initial changes
gierwialo May 17, 2023
fbc48f4
Logger with coroutine support
gierwialo May 17, 2023
af3bc1c
TCP/IP packet decode/encode class
gierwialo May 17, 2023
e3cb4b0
Tests for TCP/IP classes
gierwialo May 17, 2023
f6696e3
First python Runner tests
gierwialo May 17, 2023
e3d4a5b
Main loop for Tecemux
gierwialo May 18, 2023
b36db37
Tecemux class in external file
gierwialo May 18, 2023
8394d6b
First version of channel data forward
gierwialo May 19, 2023
dd426ad
Tests for channel forwarding
gierwialo May 19, 2023
3e390eb
Tecemux communication between endpoints
gierwialo May 19, 2023
ed81a03
Fix bug with forward multiple packets in one chunk
gierwialo May 22, 2023
31ee129
Add platform args to STH
patuwwy Feb 20, 2023
27b891a
si space accesskey generate|revoke
patuwwy Mar 3, 2023
c3208a8
Disable nagle R-H
patuwwy Apr 20, 2023
830bc27
Start for bpmux replacement
patuwwy Jan 10, 2023
235c22a
tc-verser Switch to connect method
patuwwy Jan 10, 2023
cd8b599
Rename to tecemux
patuwwy Jan 10, 2023
f622e03
DecodedFrame type
patuwwy Jan 11, 2023
534ec8c
Flags parsers
patuwwy Jan 12, 2023
732461b
Flags utils
patuwwy Jan 13, 2023
f597140
push data to channel
patuwwy Jan 15, 2023
48151d5
Duplex communication
patuwwy Jan 16, 2023
2c4b6c9
Send stdin over frames
patuwwy Jan 16, 2023
3e429fc
Issue, stream paused
patuwwy Jan 16, 2023
0fa423c
Debug AbortError
patuwwy Jan 17, 2023
3413937
Add handlers
patuwwy Jan 18, 2023
8b133ad
Ack on, alternate stdin to cli/srv
MichalCz Jan 18, 2023
e6818eb
Issue. Decoder receives more than 1 frame
patuwwy Jan 18, 2023
97f0900
Read buffer of frames to decode
patuwwy Jan 18, 2023
6556e1a
Cleanup, common sequenceNumber for TeceMux instance
patuwwy Jan 19, 2023
7a6512e
Send FIN
patuwwy Jan 19, 2023
472f0cb
Replace BPMux with TeCeMux
patuwwy Jan 20, 2023
6d15716
Cleanup
patuwwy Jan 20, 2023
5a92353
Playground pass file
patuwwy Jan 25, 2023
8ad0709
Pass big file
patuwwy Jan 25, 2023
dd7b3e1
Calc/check checksum
patuwwy Jan 28, 2023
3a20fc0
FramesKeeper
patuwwy Jan 29, 2023
c3332cb
Check for ACK of last frames send
patuwwy Jan 30, 2023
d160314
Fix send FIN
patuwwy Jan 31, 2023
a41e00d
Send file both directions
patuwwy Jan 31, 2023
96b19eb
Debug w/o FrameKeeper
patuwwy Feb 2, 2023
4cfedde
FramesKeeper generator
patuwwy Feb 9, 2023
b05ba1a
Cleanup
patuwwy Feb 10, 2023
eb7b66b
Tests
patuwwy Feb 14, 2023
d311f1c
Replace BPMux with TeCeMux
patuwwy Jan 20, 2023
ad3a5a5
R->STH. WIP
patuwwy Mar 23, 2023
fbdcb0d
R<->STH
patuwwy Mar 28, 2023
f068c5b
TeCeMux wip
patuwwy Apr 3, 2023
d96995b
fix channels _id
patuwwy Apr 6, 2023
377f1b7
Test defer
patuwwy Apr 6, 2023
83183ef
CI test
patuwwy Apr 11, 2023
f8bc943
Remove debug listeners
patuwwy Apr 11, 2023
79d202e
Runner host-client destroy
patuwwy Apr 12, 2023
426ad3b
nextTick before sequence call
patuwwy Apr 12, 2023
fe211e7
TeceMuxChannel class
patuwwy Apr 13, 2023
ee0fe1e
Disable nagle in R-H
patuwwy Apr 13, 2023
162d412
Simplify TecemuxChannel
patuwwy Apr 19, 2023
337389b
Fix lint
patuwwy Apr 20, 2023
82a770f
Restore TCM channel implementation for older nodejs
patuwwy Apr 20, 2023
f666dfc
Fix M-S connection
patuwwy Apr 26, 2023
2de58a5
Rebase
patuwwy May 22, 2023
6e06ff0
Problem with wrong data cut resolved
gierwialo May 22, 2023
e061c0e
TecemuxStreamReader and TecemuxStreamWriter wrappers for asyncio.Stream
gierwialo May 22, 2023
f1082ac
Pause resume tecemux channel
patuwwy May 25, 2023
69072bf
Fix: version bump of deprecated github actions
tomekcrm Apr 24, 2023
57b6fb1
Fix "sequence unpack failed" error with kubernetes (#852)
S4adam Apr 26, 2023
d28bba5
Fix KeyError in data
gierwialo Apr 24, 2023
a06ba53
Fix: readme help command
tomekcrm May 12, 2023
88397ec
added description and tags to sth
piotrek6641 May 19, 2023
59ec0fd
Example sth config
patuwwy Oct 24, 2022
c751cc6
Fix verser connection path
patuwwy Oct 23, 2022
e470496
Platform token
patuwwy Oct 22, 2022
b99bd90
Delete duplicated config
patuwwy Feb 15, 2023
15492cc
connect over https to MW
patuwwy Feb 17, 2023
cfc34d9
Add platform args to STH
patuwwy Feb 20, 2023
4964a15
Use daughter key, change addhub url
patuwwy Feb 27, 2023
58af95b
Add hub type, org:space param
patuwwy Feb 28, 2023
d971abf
Update params
patuwwy Feb 28, 2023
e577aa2
si space accesskey
patuwwy Mar 2, 2023
adc45a2
si space accesskey generate|revoke
patuwwy Mar 3, 2023
5f323cf
ManagerClient disconnect method
patuwwy Mar 7, 2023
e98bd15
CLI revoke key
patuwwy Mar 8, 2023
f2b889c
Notify hub about disconnect
patuwwy Mar 8, 2023
a536c48
Add si space access create "description"
patuwwy Mar 10, 2023
ea307c5
Add disconnect payload types
patuwwy Mar 14, 2023
543e561
Disconnected due to limit
patuwwy Mar 14, 2023
73cd7af
M Disconnect response type
patuwwy Mar 15, 2023
fd363a9
si space <spc> disconnect
patuwwy Mar 20, 2023
54b556b
Get HTTP platform connection status
patuwwy Mar 27, 2023
4a4bd49
Fix dropping STH by id
patuwwy Mar 27, 2023
bf2460d
Add platform args to STH
patuwwy Feb 20, 2023
a537a69
si space accesskey generate|revoke
patuwwy Mar 3, 2023
8f87419
R<->STH
patuwwy Mar 28, 2023
0913bb4
fix channels _id
patuwwy Apr 6, 2023
0cad9e4
Remove debug listeners
patuwwy Apr 11, 2023
f7813d7
Disable nagle in R-H
patuwwy Apr 13, 2023
562a68e
Rebase
patuwwy May 22, 2023
b00b855
Pause resume tecemux channel
patuwwy May 25, 2023
a484281
Sending TCP segment during open/close channel request
gierwialo Jun 1, 2023
858344e
Fix missing typings and minor refactor
gierwialo Jun 1, 2023
ef0aadb
Add missing changes
gierwialo Jun 2, 2023
13da4b3
Sequence number & Pause & Resume channel support
gierwialo Jun 6, 2023
b302e1c
Python runner with TeceMux support
gierwialo Jun 7, 2023
2594af6
First try to communitate with STH
gierwialo Jun 7, 2023
3c54c63
TCP Options support
gierwialo Jun 9, 2023
5440345
Tecemux protocol minor changes
gierwialo Jun 9, 2023
1ae7b94
Fix wrong TCP header length calculation
gierwialo Jun 12, 2023
bfb94d9
Big/Litte-endianess switch
gierwialo Jun 12, 2023
64e8b2e
To/From buffer with pseudo tcp header
gierwialo Jun 12, 2023
76820e3
Tecemux with TCP pseudoheaders
gierwialo Jun 13, 2023
7ed419e
Runner with Tecemux
gierwialo Jun 14, 2023
0e06560
Tecemux with only native asyncio.Stream
gierwialo Jun 14, 2023
70f2eae
calculateChecksum modulo problem fix
gierwialo Jun 16, 2023
4d03558
Calc checksum match with Transform Hub
gierwialo Jun 16, 2023
7638ede
Send ACK for PSH packet
gierwialo Jun 16, 2023
464dfb5
Fix wrong indent
gierwialo Jun 16, 2023
0a7b349
Add write_eof method
gierwialo Jun 16, 2023
b88590a
Sequence Order as Singleton
gierwialo Jun 16, 2023
2580f18
Documentation for inet
gierwialo Jun 16, 2023
4555cda
Tecemux docstrings
gierwialo Jun 19, 2023
be3276e
Merge branch 'feature/demux-sth-r3' into feature/python-runner-tecemux
gierwialo Jun 19, 2023
a48f6e0
Merge pull request #885 from scramjetorg/feature/python-runner-tecemux
patuwwy Jun 19, 2023
89f3e8a
Merge branch 'devel' into feature/demux-sth-r3
patuwwy Jun 19, 2023
d24f2e7
Fix linter issues
patuwwy Jun 19, 2023
668746f
Remove pydevd
gierwialo Jun 19, 2023
4f16ba6
Remove init.py
gierwialo Jun 19, 2023
b53ae90
Add attrs to requirements.txt
gierwialo Jun 19, 2023
ca61c78
Bump base image version for Docker container python runner
gierwialo Jun 20, 2023
a63e032
Additional assets for python runner
gierwialo Jun 20, 2023
b60ebdb
Stream write as wrapper for internal async
gierwialo Jun 21, 2023
a39c1e9
Excepiton details are redirected to stderr channel
gierwialo Jun 22, 2023
e6ec7d7
Erorrs and debug messages shows correctly
gierwialo Jun 23, 2023
276ff08
Minor changes
gierwialo Jun 23, 2023
9be82a6
Sync barrier & simple write method
gierwialo Jun 26, 2023
46fe7e4
anext in __ChannelContext works with Scramjet Framework to list
gierwialo Jun 27, 2023
c5727d1
pytest Tecemux test with pipe
gierwialo Jun 28, 2023
ed28e2e
Return to asyncio.wait_for
gierwialo Jun 28, 2023
df222fe
Tecemux IN & OUT channels works
gierwialo Jun 29, 2023
0c652be
Exception handling
gierwialo Jun 30, 2023
8d37089
Output stream ends correctly
gierwialo Jun 30, 2023
e3cb9ca
Runner finish improvements
gierwialo Jun 30, 2023
a0b2bef
Better cleanup
gierwialo Jul 3, 2023
ecc6786
Better cleanup #2
gierwialo Jul 3, 2023
4e24000
Close communication improvements
gierwialo Jul 5, 2023
b7ab9ac
Docs for new methods
gierwialo Jul 5, 2023
7067fcb
Merge branch 'devel' into feature/demux-sth-r3
gierwialo Jul 5, 2023
00163a9
Add missing barrier.py
gierwialo Jul 5, 2023
d20ca15
Fix readuntil EOF state
gierwialo Jul 6, 2023
2db3978
Fix readuntil EOF state #2
gierwialo Jul 6, 2023
8555271
Fix health check in tests
gierwialo Jul 6, 2023
9e29847
Fix problems with events and handlers
gierwialo Jul 7, 2023
06dcc5d
Longer timeout
gierwialo Jul 7, 2023
9e6d00c
Sync monitoring
gierwialo Jul 7, 2023
017017c
Longer timeout
gierwialo Jul 7, 2023
3d19b21
Add wait for in tests
gierwialo Jul 7, 2023
c50997b
Fix for topics
gierwialo Jul 7, 2023
8cca7ce
Merge branch 'devel' into feature/demux-sth-r3
gierwialo Jul 7, 2023
81e062c
Merge branch 'devel' into feature/demux-sth-r3
patuwwy Jul 11, 2023
ff1281c
Replace bpmux with tecemux in context.hub
patuwwy Aug 7, 2023
e959d62
Merge branch 'devel' into feature/demux-sth-r3
patuwwy Aug 8, 2023
8a8fbb1
Nested tecemux in tecemux
gierwialo Aug 10, 2023
4de3915
Docstring for new function
gierwialo Aug 10, 2023
623340e
Removing the nested TeceMux. Requests from the sequence as channels c…
patuwwy Aug 16, 2023
53d4cc3
Remove nested channels support
gierwialo Aug 25, 2023
f8b160a
Extra channels support
gierwialo Aug 25, 2023
99fc58a
Extra channel support
gierwialo Aug 28, 2023
f023abe
HTTPProxy minimal implementation
gierwialo Aug 29, 2023
85ebdff
HTTPProxy minimal tests implementation
gierwialo Aug 29, 2023
8696cde
HTTPProxy works with aiohttp
gierwialo Aug 30, 2023
594e464
Split tests into separated files
gierwialo Aug 31, 2023
a019f26
Add missing pytest.mark.asyncio
gierwialo Aug 31, 2023
af88fb6
Fix error with extra_channel creation
gierwialo Aug 31, 2023
57e9352
Add missing docstrings
gierwialo Aug 31, 2023
4465cac
ChannelGuard as manager for open/close channels
gierwialo Sep 1, 2023
79f9026
Test calculation of unused channel id
gierwialo Sep 1, 2023
795cab0
Divide tecemux.py into smaller files
gierwialo Sep 1, 2023
9962668
Move Python tecemux tests
gierwialo Sep 1, 2023
e8ee060
Format code
gierwialo Sep 4, 2023
0028afe
Format code #2
gierwialo Sep 4, 2023
7270e66
Protocol to Multiplexer name change
gierwialo Sep 4, 2023
6d39e17
Format code #3
gierwialo Sep 4, 2023
c611a11
streams sync() fix
Sep 5, 2023
75617b1
Merge branch 'devel' into feature/demux-sth-r3
S4adam Sep 5, 2023
33b700b
leftover - rename protocol to multiplexer
Sep 5, 2023
5a03ca4
Setup.py for tecemux
gierwialo Sep 5, 2023
116e0b9
[py] Proxy url handling
Sep 19, 2023
1c83e75
Move Tecemux from Verser
patuwwy Nov 3, 2023
e1c4d3b
Update deps
patuwwy Nov 3, 2023
ac6eaf8
Merge branch 'devel' into feature/demux-sth-r3
patuwwy Nov 3, 2023
e71a8e3
Fix lint issues
patuwwy Nov 3, 2023
d03c754
Merge branch 'feature/demux-sth-r3' of github.com:scramjetorg/transfo…
patuwwy Nov 3, 2023
cda80d7
Fix max chunk length, remove console.logs
patuwwy Nov 14, 2023
20ac958
Tecemux test server playground
patuwwy Nov 16, 2023
2dbd2d1
PY: prepare runner&Tecemux for Hub custom connector
Dec 4, 2023
dbb85b8
Merge branch 'devel' into feature/demux-sth-r3
patuwwy Jan 15, 2024
e6a8f36
After devel merge fix
patuwwy Jan 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,8 @@ bdd/*-test-result.txt

# pre-runner sample-package
packages/pre-runner/sample-package/

packages/python-runner/tecemux/build/
*.egg-info
# sample packages
*.tar.gz

Expand Down
14 changes: 13 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,17 @@
"markdown",
"latex",
"plaintext"
]
],
"python.testing.pytestArgs": [
"packages"
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true,
"python.linting.flake8Enabled": true,
"python.linting.enabled": true,
"flake8.args": [
"--max-line-length=120",
"--ignore=F401",
],

}
3 changes: 3 additions & 0 deletions bdd/features/e2e/E2E-014-python.feature
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ Feature: Test our shiny new Python runner
Given host is running
When sequence "../packages/python-unhealthy-sequence.tar.gz" loaded
And instance started
And wait for "3000" ms
Then instance health is "false"
And host is still running

Expand All @@ -52,9 +53,11 @@ Feature: Test our shiny new Python runner
When I execute CLI with "seq send ../packages/python-topic-producer.tar.gz"
When I execute CLI with "seq start - --output-topic names3"
Then I send input data "topic test input" with options "--end"
And wait for "100" ms
When I execute CLI with "seq send ../packages/python-topic-consumer.tar.gz"
When I execute CLI with "seq start - --input-topic names3"
And I execute CLI with "inst output -" without waiting for the end
And wait for "100" ms
Then I confirm data named "python-topics" will be received
And host is still running

Expand Down
4 changes: 4 additions & 0 deletions bdd/features/e2e/E2E-015-unified.feature
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Feature: Test our shiny new Python runner
When find and upload sequence "hello.tar.gz"
And instance started
And send "python runner" to input
And wait for "100" ms
Then "output" is "Hello python runner!"
And host is still running

Expand All @@ -15,6 +16,7 @@ Feature: Test our shiny new Python runner
When find and upload sequence "stdinout.tar.gz"
And instance started
And send "python runner" to stdin
And wait for "100" ms
Then "stdout" is "Got on stdin: python runner"
And host is still running

Expand Down Expand Up @@ -62,6 +64,7 @@ Feature: Test our shiny new Python runner
When find and upload sequence "topic-producer.tar.gz"
And instance started
And send "topic test input" to input
And wait for "100" ms
And find and upload sequence "topic-consumer.tar.gz"
And instance started
Then "output" will be data named "python-topics"
Expand All @@ -73,6 +76,7 @@ Feature: Test our shiny new Python runner
When find and upload sequence "events.tar.gz"
And instance started
And send event "test-event" to instance with message "foo"
And wait for "100" ms
Then instance emits event "test-response" with body
"""
"reply to foo"
Expand Down
Loading