congure_test: make shell commands shorter#16111
Conversation
301ce15 to
f19c8ba
Compare
|
the flasher looks pretty cool now! |
|
It seems like the flash process screws up the test BOARD=nucleo-f411re make -C tests/congure_test/ -j flash test
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE......but running just test without flash after seems to work. Is this a problem for the scope of this PR or should the hardening of the tests be moved to another PR? |
|
Shouldn't have any influence on the tests. The |
|
Will test at my end however. |
|
Could not reproduce with $ BOARD=samr21-xpro make -C tests/congure_test -j flash test
[…]
✓ Flashing done! (programmer: 'edbg' - duration: 6.02s)
(for full programmer output add PROGRAMMER_QUIET=0 or QUIET=0 to the make command line)
.....................................
----------------------------------------------------------------------
Ran 37 tests in 20.375s
OK
make: Leaving directory '/home/mlenders/Repositories/RIOT-OS/RIOT/tests/congure_test'
$ BOARD=iotlab-m3 make -C tests/congure_test -j flash test
[…]
✓ Flashing done! (programmer: 'openocd' - duration: 3.01s)
(for full programmer output add PROGRAMMER_QUIET=0 or QUIET=0 to the make command line)
.....................................
----------------------------------------------------------------------
Ran 37 tests in 21.629s
OK
make: Leaving directory '/home/mlenders/Repositories/RIOT-OS/RIOT/tests/congure_test' |
|
Could you please run your error case with diff --git a/tests/congure_test/tests/01-run.py b/tests/congure_test/tests/01-run.py
index ea58fc8d90..7494674d4f 100755
--- a/tests/congure_test/tests/01-run.py
+++ b/tests/congure_test/tests/01-run.py
@@ -17,7 +17,7 @@ from riotctrl.shell.json import RapidJSONShellInteractionParser, rapidjson
class TestCongUREBase(unittest.TestCase):
- DEBUG = False
+ DEBUG = True
@classmethod
def setUpClass(cls):and provide the output here? |
DetailsBOARD=nucleo-f411re make -C tests/congure_test/ -j flash test
make: Entering directory '/home/weiss/wd/RIOT/tests/congure_test'
Building application "tests_congure_test" for "nucleo-f411re" with MCU "stm32".
"make" -C /home/weiss/wd/RIOT/boards/nucleo-f411re
"make" -C /home/weiss/wd/RIOT/core
"make" -C /home/weiss/wd/RIOT/cpu/stm32
"make" -C /home/weiss/wd/RIOT/drivers
"make" -C /home/weiss/wd/RIOT/sys
"make" -C /home/weiss/wd/RIOT/boards/common/nucleo
"make" -C /home/weiss/wd/RIOT/drivers/periph_common
"make" -C /home/weiss/wd/RIOT/sys/auto_init
"make" -C /home/weiss/wd/RIOT/cpu/cortexm_common
"make" -C /home/weiss/wd/RIOT/sys/congure
"make" -C /home/weiss/wd/RIOT/cpu/stm32/periph
"make" -C /home/weiss/wd/RIOT/sys/fmt
"make" -C /home/weiss/wd/RIOT/cpu/stm32/stmclk
"make" -C /home/weiss/wd/RIOT/sys/isrpipe
"make" -C /home/weiss/wd/RIOT/cpu/stm32/vectors
"make" -C /home/weiss/wd/RIOT/sys/malloc_thread_safe
"make" -C /home/weiss/wd/RIOT/sys/newlib_syscalls_default
"make" -C /home/weiss/wd/RIOT/sys/pm_layered
"make" -C /home/weiss/wd/RIOT/sys/shell
"make" -C /home/weiss/wd/RIOT/cpu/cortexm_common/periph
"make" -C /home/weiss/wd/RIOT/sys/shell/commands
"make" -C /home/weiss/wd/RIOT/sys/congure/mock
"make" -C /home/weiss/wd/RIOT/sys/stdio_uart
"make" -C /home/weiss/wd/RIOT/sys/congure/test
"make" -C /home/weiss/wd/RIOT/sys/test_utils/interactive_sync
"make" -C /home/weiss/wd/RIOT/sys/tsrb
text data bss dec hex filename
22332 132 2536 25000 61a8 /home/weiss/wd/RIOT/tests/congure_test/bin/nucleo-f411re/tests_congure_test.elf
✓ Flashing done! (programmer: 'openocd' - duration: 2.00s)
(for full programmer output add PROGRAMMER_QUIET=0 or QUIET=0 to the make command line)
socat - open:/dev/ttyACM0,b115200,echo=0,raw
main(): This is RIOT! (Version: 2021.04-devel-906-gf19c8b-congure_test/enh/shorter-shell-cmd)
> main(): This is RIOT! (Version: 2021.04-devel-906-gf19c8b-congure_test/enh/shorter-shell-cmd)
>
cong_clear
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Esocat - open:/dev/ttyACM0,b115200,echo=0,raw
main(): This is RIOT! (Version: 2021.04-devel-906-gf19c8b-congure_test/enh/shorter-shell-cmd)
>
> cong_clear
{"success":null}
> cong_init 0x12345
{"error":"State object not set up"}
> .cong_clear
{"success":null}
> cong_imi 689
{"error":"State object not set up"}
> .cong_clear
{"success":null}
> state
{"driver":"0x00000000","ctx":"0x00000000","cwnd":0,"init":{"calls":0,"last_args":{"c":"0x00000000","ctx":"0x00000000"}},"inter_msg_interval":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_sent":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_discarded":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msgs_timeout":{"calls":0,"last_args":{"c":"0x00000000","msgs":[]}},"report_msgs_lost":{"calls":0,"last_args":{"c":"0x00000000","msgs":[]}},"report_msg_acked":{"calls":0,"last_args":{"c":"0x00000000","msg":null,"ack":null}},"report_ecn_ce":{"calls":0,"last_args":{"c":"0x00000000","time":0,}},}
> .cong_clear
{"success":null}
> cong_report
{"error":"State object not set up"}
> .cong_clear
{"success":null}
> state
{"driver":"0x00000000","ctx":"0x00000000","cwnd":0,"init":{"calls":0,"last_args":{"c":"0x00000000","ctx":"0x00000000"}},"inter_msg_interval":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_sent":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_discarded":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msgs_timeout":{"calls":0,"last_args":{"c":"0x00000000","msgs":[]}},"report_msgs_lost":{"calls":0,"last_args":{"c":"0x00000000","msgs":[]}},"report_msg_acked":{"calls":0,"last_args":{"c":"0x00000000","msg":null,"ack":null}},"report_ecn_ce":{"calls":0,"last_args":{"c":"0x00000000","time":0,}},}
> cong_setup
{"success":"0x20000284"}
> cong_setup 0
{"success":"0x20000284"}
> .cong_clear
{"success":null}
> state
{"driver":"0x00000000","ctx":"0x00000000","cwnd":0,"init":{"calls":0,"last_args":{"c":"0x00000000","ctx":"0x00000000"}},"inter_msg_interval":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_sent":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_discarded":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msgs_timeout":{"calls":0,"last_args":{"c":"0x00000000","msgs":[]}},"report_msgs_lost":{"calls":0,"last_args":{"c":"0x00000000","msgs":[]}},"report_msg_acked":{"calls":0,"last_args":{"c":"0x00000000","msg":null,"ack":null}},"report_ecn_ce":{"calls":0,"last_args":{"c":"0x00000000","time":0,}},}
> cong_setup abcd
{"error":"`id` expected to be integer"}
> cong_setup 1
{"error":"`id` is invalid"}> .
======================================================================
ERROR: test_init_arg_not_hex (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_init_no_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_init_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_inter_msg_interval_no_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_inter_msg_interval_not_int (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_inter_msg_interval_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_ecn_ce_no_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_ecn_ce_not_int (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_ecn_ce_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msg_acked_delay_not_uint16 (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msg_acked_msg_args_not_int (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msg_acked_no_msg (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msg_acked_not_enough_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msg_acked_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msg_acked_wnd_not_wnd_size (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msg_discarded_no_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msg_discarded_not_int (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msg_discarded_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msg_sent_no_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msg_sent_not_int (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msg_sent_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msgs_lost_exceed_msg_pool_size (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msgs_lost_msg_args_not_int (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msgs_lost_not_enough_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msgs_lost_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msgs_timeout_args_not_int (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msgs_timeout_exceed_msg_pool_size (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msgs_timeout_not_enough_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_msgs_timeout_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_no_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
======================================================================
ERROR: test_report_unknown_command (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base
----------------------------------------------------------------------
Ran 37 tests in 13.097s
FAILED (errors=31)
make: *** [/home/weiss/wd/RIOT/makefiles/tests/tests.inc.mk:22: test] Error 1
make: Leaving directory '/home/weiss/wd/RIOT/tests/congure_test'
|
|
It looks like there is a sleep before the port opens but the test runs right away. Normally there is the interactive sync or something in the tests. |
That is done by |
See RIOT-OS/riotctrl#21 (change could have been done in tests, but doing it in |
|
RIOT-OS/riotctrl#21 was merged an riotctrl v0.4.1 was released so please update your local |
|
Yup that did it! Details |
|
Maybe bump the requirements? |
|
Also does anyone else have any problems with this? |
@fjmolinas said he wanted to test some other ST boards. @fjmolinas, anything to report? |
|
|
fine with me then. |
It's passing on all |
Not sure its related though, checking. |
Actually, the test doesn't boot on that BOARD (an msp430) |
then it probably also does in |
Its not about it being non-resettable, I jumped to that conclusion initially, but then realized I get no output from the board when flashing this test (I do with other apps). |
Still, I don't see how this PR would change the situation in current master. Can you please check, if this is a regression introduced in this PR by checking if the same happens on master? |
Ah yes sorry, its unrelated to this PR in that sense. |
MrKevinWeiss
left a comment
There was a problem hiding this comment.
I think this PR fixes problems, all other issues found should probably be dealt with in further PRs. This is still a new feature and it would be good if we can get improvements and bug fixes in before the next release.
ACK.
|
I will wait a bit before clicking the merge! |
I would say GO! |
|
🎉 thanks for the review! |
Contribution description
The current version of
congure_testsuses quite a lot of arguments which can be problematic, if e.g. the UART-USB-converter on a board does not support such long inputs. This splits up of those commands, by filling a central argument list. Since this framework is meant to be used with a test script only anyways, I don't think this extra step would make a problem.Testing procedure
$ make -C tests/congure_test -j flash testshould still succeed for any board.
Issues/PRs references
#15953 (comment)
Required for #15952 and #15953.