Skip to content

tools/testrunner: add variable for customizing a delay before reset#13070

Merged
fjmolinas merged 2 commits intoRIOT-OS:masterfrom
aabadie:pr/tools/testrunner_reset_delay
Jan 10, 2020
Merged

tools/testrunner: add variable for customizing a delay before reset#13070
fjmolinas merged 2 commits intoRIOT-OS:masterfrom
aabadie:pr/tools/testrunner_reset_delay

Conversation

@aabadie
Copy link
Contributor

@aabadie aabadie commented Jan 10, 2020

Contribution description

This PR adds a new variable in the testrunner that allows for setting a custom delay before resetting a board under test.
The default value is set to 0, e.g. no delay but for the hifive1b it's set to 1s. On this board, when calling reset too early after flashing, the firmware crashes (with the red led blinking).
The variable is only set with the test target.

Testing procedure

make BOARD=hifive1b -C tests/bench_msg_pingpong flash test

on master, the test fails and the firmware is in a broken state (red led blinking), with this PR, the test runs normally (and succeeds).

Issues/PRs references

Found while testing #12934

@aabadie
testrunner: allow setting custom delay before reset
c6a2b39 
On some boards, such as hifive1b, resetting right after flashing could crash the running firmware
@aabadie
boards/hifive1b: set custom reset delay of 1s for tests
267e273
@aabadie aabadie added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Area: CI Area: Continuous Integration of RIOT components Area: tools Area: Supplementary tools Area: boards Area: Board ports labels Jan 10, 2020
@aabadie aabadie requested a review from fjmolinas January 10, 2020 07:40
@fjmolinas
Copy link
Contributor

fjmolinas commented Jan 10, 2020

@aabadie don't have access to a board to test ATM, can you copy output of a failed case? and sucesfful?

@fjmolinas
Copy link
Contributor

fjmolinas commented Jan 10, 2020

Code is good otherwise.

@aabadie
Copy link
Contributor Author

aabadie commented Jan 10, 2020

output of failed (master) and successful (this PR) test:

master 
make BOARD=hifive1b -C tests/bench_msg_pingpong flash test
make: Entering directory '/work/riot/RIOT/tests/bench_msg_pingpong'
Building application "tests_bench_msg_pingpong" for "hifive1b" with MCU "fe310".

"make" -C /work/riot/RIOT/boards/hifive1b
"make" -C /work/riot/RIOT/core
"make" -C /work/riot/RIOT/cpu/fe310
"make" -C /work/riot/RIOT/cpu/fe310/nano
"make" -C /work/riot/RIOT/cpu/fe310/periph
"make" -C /work/riot/RIOT/cpu/fe310/vendor
"make" -C /work/riot/RIOT/drivers
"make" -C /work/riot/RIOT/drivers/periph_common
"make" -C /work/riot/RIOT/sys
"make" -C /work/riot/RIOT/sys/auto_init
"make" -C /work/riot/RIOT/sys/div
"make" -C /work/riot/RIOT/sys/isrpipe
"make" -C /work/riot/RIOT/sys/stdio_uart
"make" -C /work/riot/RIOT/sys/test_utils/interactive_sync
"make" -C /work/riot/RIOT/sys/tsrb
"make" -C /work/riot/RIOT/sys/xtimer
   text	   data	    bss	    dec	    hex	filename
  13890	    136	   3784	  17810	   4592	/work/riot/RIOT/tests/bench_msg_pingpong/bin/hifive1b/tests_bench_msg_pingpong.elf
/work/riot/RIOT/dist/tools/jlink/jlink.sh flash /work/riot/RIOT/tests/bench_msg_pingpong/bin/hifive1b/tests_bench_msg_pingpong.bin
### Flashing Target ###
### Flashing at base address 0x20010000 with offset 0 ###
SEGGER J-Link Commander V6.56d (Compiled Dec 12 2019 13:05:00)
DLL version V6.56d, compiled Dec 12 2019 13:04:51

J-Link Commander will now exit on Error

J-Link Command File read successfully.
Processing script file...

J-Link connection not established yet but required for command.
Connecting to J-Link via USB...O.K.
Firmware: J-Link OB-K22-SiFive compiled Dec 12 2019 16:26:28
Hardware version: V1.00
S/N: 979001207
VTref=3.300V
Target connection not established yet but required for command.
Device "FE310" selected.


Connecting to target via JTAG
ConfigTargetSettings() start
ConfigTargetSettings() end
TotalIRLen = 5, IRPrint = 0x01
JTAG chain detection found 1 devices:
 #0 Id: 0x20000913, IRLen: 05, Unknown device
Debug architecture:
  RISC-V debug: 0.13
  AddrBits: 7
  DataBits: 32
  IdleClks: 5
Memory access:
  Via system bus: No
  Via ProgBuf: Yes (16 ProgBuf entries)
DataBuf: 1 entries
  autoexec[0] implemented: Yes
Detected: RV32 core
CSR access via abs. commands: No
Temp. halted CPU for NumHWBP detection
HW instruction/data BPs: 8
Support set/clr BPs while running: No
HW data BPs trigger before execution of inst
RISC-V identified.
Halting CPU for downloading file.
Downloading file [/work/riot/RIOT/tests/bench_msg_pingpong/bin/hifive1b/tests_bench_msg_pingpong.bin]...
Comparing flash   [100%] Done.
Erasing flash     [100%] Done.
Programming flash [100%] Done.
Verifying flash   [100%] Done.
J-Link: Flash download: Bank 0 @ 0x20000000: 1 range affected (65536 bytes)
J-Link: Flash download: Total time needed: 0.563s (Prepare: 0.103s, Compare: 0.035s, Erase: 0.146s, Program: 0.226s, Verify: 0.035s, Restore: 0.015s)
O.K.

Reset delay: 0 ms
Reset type Normal: Resets core & peripherals using <ndmreset> bit in <dmcontrol> debug register.



Script processing completed.

r
/work/riot/RIOT/dist/tools/pyterm/pyterm -p "/dev/ttyACM0" -b "115200" --noprefix --no-repeat-command-on-empty-line
Connect to serial port /dev/ttyACM0
Welcome to pyterm!
Type '/exit' to exit.
r
r
r
r
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
    incoming = spawn.read_nonblocking(spawn.maxread, timeout)
  File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 482, in read_nonblocking
    raise TIMEOUT('Timeout exceeded.')
pexpect.exceptions.TIMEOUT: Timeout exceeded.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/work/riot/RIOT/tests/bench_msg_pingpong/tests/01-run.py", line 19, in <module>
    sys.exit(run(testfunc))
  File "/work/riot/RIOT/dist/pythonlibs/testrunner/__init__.py", line 27, in run
    logfile=sys.stdout if echo else None)
  File "/work/riot/RIOT/dist/pythonlibs/testrunner/spawn.py", line 71, in setup_child
    sync_child(child, env)
  File "/work/riot/RIOT/dist/pythonlibs/testrunner/spawn.py", line 97, in sync_child
    TEST_INTERACTIVE_DELAY)
  File "/work/riot/RIOT/dist/pythonlibs/testrunner/utils.py", line 24, in test_utils_interactive_sync
    child.expect_exact('READY', timeout=0)
  File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 418, in expect_exact
    return exp.expect_loop(timeout)
  File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 119, in expect_loop
    return self.timeout(e)
  File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 82, in timeout
    raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: Timeout exceeded.
<pexpect.pty_spawn.spawn object at 0x7f67d1843290>
command: /usr/bin/make
args: [b'/usr/bin/make', b'cleanterm']
buffer (last 100 chars): "mand-on-empty-line\r\nConnect to serial port /dev/ttyACM0\r\nWelcome to pyterm!\r\nType '/exit' to exit.\r\n"
before (last 100 chars): "mand-on-empty-line\r\nConnect to serial port /dev/ttyACM0\r\nWelcome to pyterm!\r\nType '/exit' to exit.\r\n"
after: <class 'pexpect.exceptions.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 19747
child_fd: 5
closed: False
timeout: 10
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_string:
    0: 'READY'
make: *** [/work/riot/RIOT/tests/bench_msg_pingpong/../../Makefile.include:735: test] Error 1
make: Leaving directory '/work/riot/RIOT/tests/bench_msg_pingpong'
this PR 
make BOARD=hifive1b -C tests/bench_msg_pingpong flash test
make: Entering directory '/work/riot/RIOT/tests/bench_msg_pingpong'
Building application "tests_bench_msg_pingpong" for "hifive1b" with MCU "fe310".

"make" -C /work/riot/RIOT/boards/hifive1b
"make" -C /work/riot/RIOT/core
"make" -C /work/riot/RIOT/cpu/fe310
"make" -C /work/riot/RIOT/cpu/fe310/nano
"make" -C /work/riot/RIOT/cpu/fe310/periph
"make" -C /work/riot/RIOT/cpu/fe310/vendor
"make" -C /work/riot/RIOT/drivers
"make" -C /work/riot/RIOT/drivers/periph_common
"make" -C /work/riot/RIOT/sys
"make" -C /work/riot/RIOT/sys/auto_init
"make" -C /work/riot/RIOT/sys/div
"make" -C /work/riot/RIOT/sys/isrpipe
"make" -C /work/riot/RIOT/sys/stdio_uart
"make" -C /work/riot/RIOT/sys/test_utils/interactive_sync
"make" -C /work/riot/RIOT/sys/tsrb
"make" -C /work/riot/RIOT/sys/xtimer
   text	   data	    bss	    dec	    hex	filename
  13922	    136	   3784	  17842	   45b2	/work/riot/RIOT/tests/bench_msg_pingpong/bin/hifive1b/tests_bench_msg_pingpong.elf
/work/riot/RIOT/dist/tools/jlink/jlink.sh flash /work/riot/RIOT/tests/bench_msg_pingpong/bin/hifive1b/tests_bench_msg_pingpong.bin
### Flashing Target ###
### Flashing at base address 0x20010000 with offset 0 ###
SEGGER J-Link Commander V6.56d (Compiled Dec 12 2019 13:05:00)
DLL version V6.56d, compiled Dec 12 2019 13:04:51

J-Link Commander will now exit on Error

J-Link Command File read successfully.
Processing script file...

J-Link connection not established yet but required for command.
Connecting to J-Link via USB...O.K.
Firmware: J-Link OB-K22-SiFive compiled Dec 12 2019 16:26:28
Hardware version: V1.00
S/N: 979001207
VTref=3.300V
Target connection not established yet but required for command.
Device "FE310" selected.


Connecting to target via JTAG
ConfigTargetSettings() start
ConfigTargetSettings() end
TotalIRLen = 5, IRPrint = 0x01
JTAG chain detection found 1 devices:
 #0 Id: 0x20000913, IRLen: 05, Unknown device
Debug architecture:
  RISC-V debug: 0.13
  AddrBits: 7
  DataBits: 32
  IdleClks: 5
Memory access:
  Via system bus: No
  Via ProgBuf: Yes (16 ProgBuf entries)
DataBuf: 1 entries
  autoexec[0] implemented: Yes
Detected: RV32 core
CSR access via abs. commands: No
Temp. halted CPU for NumHWBP detection
HW instruction/data BPs: 8
Support set/clr BPs while running: No
HW data BPs trigger before execution of inst
RISC-V identified.
Halting CPU for downloading file.
Downloading file [/work/riot/RIOT/tests/bench_msg_pingpong/bin/hifive1b/tests_bench_msg_pingpong.bin]...
Comparing flash   [100%] Done.
Erasing flash     [100%] Done.
Programming flash [100%] Done.
Verifying flash   [100%] Done.
J-Link: Flash download: Bank 0 @ 0x20000000: 1 range affected (65536 bytes)
J-Link: Flash download: Total time needed: 1.250s (Prepare: 0.103s, Compare: 0.320s, Erase: 0.147s, Program: 0.343s, Verify: 0.320s, Restore: 0.015s)
O.K.

Reset delay: 0 ms
Reset type Normal: Resets core & peripherals using <ndmreset> bit in <dmcontrol> debug register.



Script processing completed.

r
/work/riot/RIOT/dist/tools/pyterm/pyterm -p "/dev/ttyACM0" -b "115200" --noprefix --no-repeat-command-on-empty-line
Connect to serial port /dev/ttyACM0
Welcome to pyterm!
Bench Clock Reset Complete

ATE0-->OK
AT+BLEINIT=0-->OK
AT+CWMODE=0-->OK

Help: Press s to start test, r to print it is ready
r
READY
s
START
main(): This is RIOT! (Version: 2020.01-devel-1641-g267e2-pr/tools/testrunner_reset_delay)
main starting
{ "result" : 458123 }

make: Leaving directory '/work/riot/RIOT/tests/bench_msg_pingpong'

fjmolinas
fjmolinas approved these changes Jan 10, 2020
View reviewed changes
Copy link
Contributor

@fjmolinas fjmolinas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @aabadie ACK!

@fjmolinas fjmolinas merged commit 7f63b48 into RIOT-OS:master Jan 10, 2020
@aabadie aabadie deleted the pr/tools/testrunner_reset_delay branch January 10, 2020 12:38
@fjmolinas fjmolinas added this to the Release 2020.01 milestone Jan 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@fjmolinas fjmolinas fjmolinas approved these changes

Assignees

No one assigned

Labels

Area: boards Area: Board ports Area: CI Area: Continuous Integration of RIOT components Area: tools Area: Supplementary tools CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR

Projects

None yet

Milestone

Release 2020.01

Development

Successfully merging this pull request may close these issues.

2 participants

@aabadie @fjmolinas