congure: initial import of a congestion control framework by miri64 · Pull Request #15951 · RIOT-OS/RIOT

miri64 · 2021-02-08T20:47:20Z

Contribution description

This provides CongURE (Congestion Control Utilizing Reusable Elements, pronounced like "conjure"), a framework to implement congestion control and use it in a reusable manner with a number of protocols.

The original motivation was to provide congestion control with 6LoWPAN selective fragment recovery and allow for the congestion control mechanism to be interchangeable. However, now that there is a framework with multiple well tested implementations of congestion control, it can also be used for other protocols, such as TCP or QUIC, as I kept the units for the congestion window up to the user (TCP and QUIC, e.g., uses bytes, SFR, e.g., uses number of fragments). It is also designed as such that multiple instances of the same implementation can run on the same image (e.g. for when TCP and SFR want to use the same CC). In that case, I can imagine a lot of ROM can be saved, since there is only one CC code for two layers.

A test framework for the CongURE framework is also provided in form of shell commands to trigger the API functions of CongURE, as well as tests for said framework (a framework test framework test if you want 😉). If preferred, I can move those to a separate PR.

As to why this is in sys/ and not sys/net/: There are no networking components to this framework, it just provides the functions to calculate the size of a congestion window. seq e.g. is also in sys/, though sequence number arithmetic mostly shows up in networking. Not sure this could help with local queue congestion problems, but when thinking out of the box, one might think as such :-P.

Testing procedure

A test framework including tests for it is provided. riotctrl (at least v0.2.2) is required for them to run. If installed,

make -j -C tests/congure_test flash test

should succeed.

Issues/PRs references

Follow-up PRs for actual implementations of congestion control mechanisms will follow, as well as integration PRs for GNRC's 6LoWPAN SFR and maybe TCP as well :-). When merged, I will adapt the test framework to use the turo framework introduced in #15950, which was the original idea to begin with (which is why it is already printing JSON strings ;-)).

MrKevinWeiss · 2021-02-09T09:39:53Z

Is there any whitelist/black list needed for the boards or features required? A quick test on the nucleo-f411re has timeouts (even with just plain term).

Details

BOARD=nucleo-f411re make term -C tests/congure_test/
make: Entering directory '/home/weiss/wd/RIOT/tests/congure_test'
/home/weiss/wd/RIOT/dist/tools/pyterm/pyterm -p "/dev/ttyACM0" -b "115200"  
Twisted not available, please install it if you want to use pyterm's JSON capabilities
2021-02-09 10:33:17,797 # Connect to serial port /dev/ttyACM0
Welcome to pyterm!
Type '/exit' to exit.
help
2021-02-09 10:33:19,835 # Command              Description
2021-02-09 10:33:19,838 # ---------------------------------------
2021-02-09 10:33:19,844 # state                Prints current CongURE state object as JSON
2021-02-09 10:33:19,847 # reboot               Reboot the node
2021-02-09 10:33:19,852 # version              Prints current RIOT_VERSION
2021-02-09 10:33:19,857 # pm                   interact with layered PM subsystem
2021-02-09 10:33:19,861 # cong_clear           Clears CongURE state object
2021-02-09 10:33:19,867 # cong_setup           Calls the setup function for the CongURE state object
2021-02-09 10:33:19,873 # cong_init            Calls init method of the CongURE state object
2021-02-09 10:33:19,881 # cong_imi             Calls inter_message_interval method of the CongURE state object
2021-02-09 10:33:19,887 # cong_report          Calls a report_* method of the CongURE state object
> state
2021-02-09 10:33:23,914 #  {"driver":"0x00000000","ctx":"0x00000000","cwnd":0,"init":{"calls":0,"last_args":{"c":"0x00000000","ctx":"0x00000000"}},"inter_msg_interval":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_sent":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_discarded":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_timeout":{"calls":0,"last_args":{"c":"0x00000000","msgs":[]}},"report_msg_lost":{"calls":0,"last_args":{"c":"0x00000000","msgs":[]}},"report_msg_acked":{"calls":0,"last_args":{"c":"0x00000000","msg":null,"ack":null}},"report_ecn_ce":{"calls":0,"last_args":{"c":"0x00000000","time":0,}},}
pm
2021-02-09 10:33:29,816 #  Usage:
2021-02-09 10:33:29,821 # 	pm show: display current blockers for each power mode
2021-02-09 10:33:29,827 # 	pm set <mode>: manually set power mode (lasts until WFI returns)
2021-02-09 10:33:29,831 # 	pm block <mode>: manually block power mode
2021-02-09 10:33:29,835 # 	pm unblock <mode>: manually unblock power mode
2021-02-09 10:33:29,837 # 	pm off: call pm_off()
> version
2021-02-09 10:33:31,524 #  2021.04-devel-474-g90b0f-congure/feat/initial
> cong_clear
2021-02-09 10:34:02,223 #  > main(): This is RIOT! (Version: 2021.04-devel-474-g90b0f-congure/feat/initial)
help
2021-02-09 10:34:07,452 #  Command              Description
2021-02-09 10:34:07,455 # ---------------------------------------
2021-02-09 10:34:07,462 # state                Prints current CongURE state object as JSON
2021-02-09 10:34:07,464 # reboot               Reboot the node
2021-02-09 10:34:07,469 # version              Prints current RIOT_VERSION
2021-02-09 10:34:07,474 # pm                   interact with layered PM subsystem
2021-02-09 10:34:07,478 # cong_clear           Clears CongURE state object
2021-02-09 10:34:07,484 # cong_setup           Calls the setup function for the CongURE state object
2021-02-09 10:34:07,490 # cong_init            Calls init method of the CongURE state object
2021-02-09 10:34:07,498 # cong_imi             Calls inter_message_interval method of the CongURE state object
2021-02-09 10:34:07,504 # cong_report          Calls a report_* method of the CongURE state object
> cong_init
2021-02-09 10:34:16,036 #  {"error":"State object not set up"}
> cong_setup
2021-02-09 10:34:35,819 #  {"setup":"0x20000284"}
> cong_init
2021-02-09 10:34:39,189 #  {"error":"`ctx` argument expected"}
> cong_init a
2021-02-09 10:34:42,053 #  {"error":"`ctx` expected to be hex"}
> cong_init 0
2021-02-09 10:35:36,993 # Exiting Pyterm

miri64 · 2021-02-09T11:10:46Z

Is there any whitelist/black list needed for the boards or features required?

Shouldn't be, which is why so far, I only tested on native. Will try to reproduce on one of the platforms I have at hand.

miri64 · 2021-02-09T11:11:29Z

Where do I see the timeout in your output?

miri64 · 2021-02-09T11:14:20Z

(cong_init does not print anything on success, if that is what you mean...)

MrKevinWeiss · 2021-02-09T11:15:42Z

It doesn't show a prompt and locks or something compared to native.

miri64 · 2021-02-09T11:59:13Z

It doesn't show a prompt and locks or something compared to native.

Mh... two problems, I did not check the length of the argument, before accessing the array, which I fixed. However, it seems like the prompt is not flushed when a command has no output in general:

2021-02-09 12:53:02,690 #  main(): This is RIOT! (Version: 2021.04-devel-474-g90b0f-congure/feat/initial)
> cong_setup
2021-02-09 12:53:06,004 #  {"setup":"0x20000284"}
> cong_init 0x0
help
2021-02-09 12:53:18,596 #  > shell: command not found: he
> cong_setup
2021-02-09 12:53:31,069 #  {"setup":"0x20000284"}
> cong_init 0x0
help
2021-02-09 12:53:38,535 #  > Command              Description
2021-02-09 12:53:38,539 # ---------------------------------------
2021-02-09 12:53:38,545 # state                Prints current CongURE state object as JSON
2021-02-09 12:53:38,548 # reboot               Reboot the node
2021-02-09 12:53:38,552 # version              Prints current RIOT_VERSION
2021-02-09 12:53:38,557 # pm                   interact with layered PM subsystem
2021-02-09 12:53:38,562 # cong_clear           Clears CongURE state object
2021-02-09 12:53:38,568 # cong_setup           Calls the setup function for the CongURE state object
2021-02-09 12:53:38,574 # cong_init            Calls init method of the CongURE state object
2021-02-09 12:53:38,582 # cong_imi             Calls inter_message_interval method of the CongURE state object
2021-02-09 12:53:38,588 # cong_report          Calls a report_* method of the CongURE state object
2021-02-09 12:54:13,481 # Exiting Pyterm

Note the > prompt showing up in the table head of the help command. Will dig a bit why.

miri64 · 2021-02-09T12:20:49Z

Seems to be an issue with pyterm. When using RIOT_TERMINAL=socat, this is not a problem:

cong_setup
{"setup":"0x20000284"}
> cong_init 0x0
> help
Command              Description
---------------------------------------
state                Prints current CongURE state object as JSON
reboot               Reboot the node
version              Prints current RIOT_VERSION
pm                   interact with layered PM subsystem
cong_clear           Clears CongURE state object
cong_setup           Calls the setup function for the CongURE state object
cong_init            Calls init method of the CongURE state object
cong_imi             Calls inter_message_interval method of the CongURE state object
cong_report          Calls a report_* method of the CongURE state object

Will have a look if I find a fix.

miri64 · 2021-02-09T19:41:57Z

Rebased to include #15962.

miri64 · 2021-02-09T20:08:26Z

Great... now when hitting enter on an empty command, does not work anymore...

miri64 · 2021-02-09T20:15:48Z

Ah, RIOTCtrl needs to use cleanterm instead of term for this to work :-).

miri64 · 2021-02-23T18:24:23Z

It seems like pytest tests/congure_test/tests/01-run.py just times out...

Ah! You have to execute it within the test directory. So

cd tests/congure_test
pytest tests/01-run.py

Should I add a fix for that to be possible from any directory, or should we leave it as is, since this is not integrated anyways (yet, but when it will be, it will be used via make anyways where we have CURDIR)

MrKevinWeiss · 2021-02-24T08:24:44Z

Should I add a fix for that to be possible from any directory, or should we leave it as is, since this is not integrated anyways (yet, but when it will be, it will be used via make anyways where we have CURDIR)

Maybe just add a note on the readme.

MrKevinWeiss · 2021-02-24T08:26:02Z

miri64 · 2021-02-24T09:35:09Z

Should I add a fix for that to be possible from any directory, or should we leave it as is, since this is not integrated anyways (yet, but when it will be, it will be used via make anyways where we have CURDIR)

Maybe just add a note on the readme.

Done

MrKevinWeiss

Tests work with make and pytest on both native and nucleo-f411re.

It seems like there are some kconfig issues:

Details

make clean
weiss@mobiweiss:~/wd/RIOT/tests/congure_test$ TEST_KCONFIG=1 make all
=== [ATTENTION] Testing Kconfig dependency modelling ===
[genconfig.py]:ERROR-=> The parameter KCONFIG_USEMODULE_SHELL could not be set to y.
[genconfig.py]:ERROR-   Check the following unmet dependencies: USEMODULE_SHELL (=n), (!TEST_KCONFIG) (=n)

[genconfig.py]:ERROR-=> The parameter SHELL_NO_ECHO could not be set to y.
[genconfig.py]:ERROR-   Check the following unmet dependencies: ((KCONFIG_USEMODULE_SHELL && !TEST_KCONFIG) || (MODULE_SHELL && TEST_KCONFIG)) (=n)

make: *** No rule to make target '/home/weiss/wd/RIOT/tests/congure_test/bin/native/generated/out.config', needed by 'check-kconfig-errors'.  Stop.

Maybe @leandrolanzieri can give some feedback.

MrKevinWeiss · 2021-02-25T09:37:07Z

same thing if I say make menuconfig

miri64 · 2021-02-25T09:37:40Z

It seems like there are some kconfig issues:

Mhhh I thought I solved that already :-/ Will have another look, but I think it's because with TEST_KCONFIG=1 it still tries to load the config for app.config.

miri64 · 2021-02-25T09:38:59Z

@leandrolanzieri would it make sense that with TEST_KCONFIG=1 we only consider *.config.test files, while with !TEST_KCONFIG we only consider *.config files in the application?

miri64 · 2021-02-25T09:40:49Z

It seems like there are some kconfig issues:

Mhhh I thought I solved that already :-/ Will have another look, but I think it's because with TEST_KCONFIG=1 it still tries to load the config for app.config.

Yepp, if I remove the app.config file in the test, it works:

$ TEST_KCONFIG=1 make -C tests/congure_test/ -j
make: Entering directory '/home/mlenders/Repositories/RIOT-OS/RIOT/tests/congure_test'
=== [ATTENTION] Testing Kconfig dependency modelling ===
[genconfig.py]:ERROR-=> The parameter KCONFIG_USEMODULE_SHELL could not be set to y.
[genconfig.py]:ERROR-   Check the following unmet dependencies: USEMODULE_SHELL (=n), (!TEST_KCONFIG) (=n)

make: *** No rule to make target '/home/mlenders/Repositories/RIOT-OS/RIOT/tests/congure_test/bin/native/generated/out.config', needed by 'check-kconfig-errors'.  Stop.
make: *** Waiting for unfinished jobs....
make: Leaving directory '/home/mlenders/Repositories/RIOT-OS/RIOT/tests/congure_test'
$ rm tests/congure_test/app.config
$ TEST_KCONFIG=1 make -C tests/congure_test/ -j
make: Entering directory '/home/mlenders/Repositories/RIOT-OS/RIOT/tests/congure_test'
=== [ATTENTION] Testing Kconfig dependency modelling ===
=== [ATTENTION] Testing Kconfig dependency modelling ===
Building application "tests_congure_test" for "native" with MCU "native".

"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/boards/native
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/core
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/cpu/native
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/drivers
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/sys
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/boards/native/drivers
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/sys/auto_init
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/sys/congure
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/sys/fmt
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/cpu/native/periph
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/drivers/periph_common
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/cpu/native/stdio_native
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/sys/shell
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/sys/shell/commands
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/sys/test_utils/interactive_sync
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/sys/congure/mock
"make" -C /home/mlenders/Repositories/RIOT-OS/RIOT/sys/congure/test
/usr/bin/ld: /home/mlenders/Repositories/RIOT-OS/RIOT/tests/congure_test/bin/native/cpu/tramp.o: warning: relocation against `_native_saved_eip' in read-only section `.text'
/usr/bin/ld: warning: creating DT_TEXTREL in a PIE
   text	   data	    bss	    dec	    hex	filename
  47915	    772	  48004	  96691	  179b3	/home/mlenders/Repositories/RIOT-OS/RIOT/tests/congure_test/bin/native/tests_congure_test.elf
make: Leaving directory '/home/mlenders/Repositories/RIOT-OS/RIOT/tests/congure_test'

MrKevinWeiss · 2021-02-25T09:41:20Z

Just got there.. yup, it looks like the app.config is being loaded when TEST_KCONFIG=1

MrKevinWeiss · 2021-02-25T09:43:16Z

Could it have something to do with #16064?

MrKevinWeiss · 2021-02-25T09:44:17Z

probably not...

miri64 · 2021-02-25T09:47:57Z

Could it have something to do with #16064?

No, I don't think so. This is purely about app.config being included all the time:

RIOT/makefiles/kconfig.mk

Lines 58 to 65 in f3871c0

    
           # Add configurations that only work when running the Kconfig test so far, 
        
           # because they activate modules. 
        
           ifeq (1,$(TEST_KCONFIG)) 
        
             MERGE_SOURCES += $(wildcard $(APPDIR)/app.config.test) 
        
           endif 
        
           MERGE_SOURCES += $(wildcard $(KCONFIG_APP_CONFIG)) 
        
           MERGE_SOURCES += $(wildcard $(KCONFIG_USER_CONFIG))

miri64 · 2021-02-25T09:54:07Z

Oh, I did not even push the app.config.test yet, I just noticed Oo

MrKevinWeiss · 2021-02-25T09:56:53Z

I wonder if we just drop trying to support the old way and only keep the post migration way...

I guess it depends on if app.config would be able to be used with app.config.test (like it is now) or if it should be either or.

miri64 · 2021-02-25T10:06:34Z

I wonder if we just drop trying to support the old way and only keep the post migration way...

I guess it depends on if app.config would be able to be used with app.config.test (like it is now) or if it should be either or.

This will make full migration a pain, I believe, since we then have to port all the app.configs in one big effort. It would be better if they could exist side by side for a while IMHO. See #16092

MrKevinWeiss · 2021-02-26T08:52:19Z

Please rebase and I will retest with the app.config.test fix

miri64 · 2021-02-26T09:08:33Z

Please rebase and I will retest with the app.config.test fix

Rebased and squashed. Remember the clean step inbetween though ;-)

miri64 · 2021-02-26T09:15:17Z

Rebased and squashed (forgot to push)

MrKevinWeiss

I think things look good to me. Kconfig is checked, the idea is documented, test work for a board and native. ACK!

miri64 · 2021-02-26T11:47:13Z

Thanks for the review and all the help with Kconfig migration and RIOTCtrl deployment!

miri64 added Type: new feature The issue requests / The PR implemements a new feature for RIOT Area: sys Area: System labels Feb 8, 2021

miri64 requested review from MrKevinWeiss, benpicco, bergzand, leandrolanzieri and maribu February 8, 2021 20:47

miri64 requested review from MichelRottleuthner, aabadie, cgundogan, fjmolinas, jia200x and smlng as code owners February 8, 2021 20:47

miri64 added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: run tests If set, CI server will run tests on hardware for the labeled PR labels Feb 8, 2021

This was referenced Feb 8, 2021

congure_quic: initial import of QUIC congestion control #15952

Merged

congure_reno: initial import of TCP Reno congestion control #15953

Merged

This was referenced Feb 9, 2021

pyterm: read space after prompt into prompt #15962

Merged

congure_abe: initial import of TCP Alternative Backoff with ECN for CongURE #15968

Merged

miri64 force-pushed the congure/feat/initial branch from 1a67d29 to dd0b5c8 Compare February 9, 2021 19:38

MrKevinWeiss requested changes Feb 25, 2021

View reviewed changes

miri64 mentioned this pull request Feb 25, 2021

Kconfig: make app.config and app.config.test mutually exclusive #16092

Merged

miri64 added 4 commits February 26, 2021 10:08

congure: initial import of a congestion control framework

2f2a009

uncrustify: whitelist congure

28592c2

congure_test: initial import of CongURE test framework

e65fee4

congure_mock: initial import of a CongURE mock implementation

f5bae1a

tests: initial import of tests for CongURE and its test framework

90a1e4d

miri64 force-pushed the congure/feat/initial branch from 62585ab to 90a1e4d Compare February 26, 2021 09:15

MrKevinWeiss approved these changes Feb 26, 2021

View reviewed changes

miri64 merged commit fc66276 into RIOT-OS:master Feb 26, 2021

miri64 deleted the congure/feat/initial branch February 26, 2021 11:47

Conversation

miri64 commented Feb 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution description

Testing procedure

Issues/PRs references

Uh oh!

MrKevinWeiss commented Feb 9, 2021

Uh oh!

miri64 commented Feb 9, 2021

Uh oh!

miri64 commented Feb 9, 2021

Uh oh!

miri64 commented Feb 9, 2021

Uh oh!

MrKevinWeiss commented Feb 9, 2021

Uh oh!

miri64 commented Feb 9, 2021

Uh oh!

miri64 commented Feb 9, 2021

Uh oh!

miri64 commented Feb 9, 2021

Uh oh!

miri64 commented Feb 9, 2021

Uh oh!

miri64 commented Feb 9, 2021

Uh oh!

miri64 commented Feb 23, 2021

Uh oh!

MrKevinWeiss commented Feb 24, 2021

Uh oh!

MrKevinWeiss commented Feb 24, 2021

Uh oh!

miri64 commented Feb 24, 2021

Uh oh!

MrKevinWeiss left a comment

Choose a reason for hiding this comment

Uh oh!

MrKevinWeiss commented Feb 25, 2021

Uh oh!

miri64 commented Feb 25, 2021

Uh oh!

miri64 commented Feb 25, 2021

Uh oh!

miri64 commented Feb 25, 2021

Uh oh!

MrKevinWeiss commented Feb 25, 2021

Uh oh!

MrKevinWeiss commented Feb 25, 2021

Uh oh!

MrKevinWeiss commented Feb 25, 2021

Uh oh!

miri64 commented Feb 25, 2021

Uh oh!

miri64 commented Feb 25, 2021

Uh oh!

MrKevinWeiss commented Feb 25, 2021

Uh oh!

miri64 commented Feb 25, 2021

Uh oh!

MrKevinWeiss commented Feb 26, 2021

Uh oh!

miri64 commented Feb 26, 2021

Uh oh!

miri64 commented Feb 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MrKevinWeiss left a comment

Choose a reason for hiding this comment

Uh oh!

miri64 commented Feb 26, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

miri64 commented Feb 8, 2021 •

edited

Loading

miri64 commented Feb 26, 2021 •

edited

Loading