Makefiles: add support to generate both `.hex` and `.bin` file and add FLASHFILE variable by cladmi · Pull Request #8838 · RIOT-OS/RIOT

cladmi · 2018-03-27T13:22:25Z

Contribution description

I need to be able to generate a .bin file to flash nucleo boards by drag-and-dropping the firmware.
The main goal of this PR main is to add a target for the bin file. This would allow asking for building the bin file from the command line but not creating it for every build.

Remove the HEXFILE = $(ELFFILE:.elf=.bin) hack
Add a FLASHFILE variable which fixes the hack in murdock: initial hardware-in-the-loop support #8801
Change flasher script that where getting the flashfile from environment variable.

Integrated in sub PRs:

While looking into this I tried fixing other problems to not propagate hack somewhere else:

It has the side effect to not generate the '.hex' or '.bin' file if not required for the flasher, which I think is better

This ended up in a big PR that I will split.

Verifying after rebase:

Things I should look into after rebasing the PR in case new boards where added:

git grep FFLAGS | grep -v FLASHFILE
git grep OFLAGS |  grep -e '-O'
git grep HEXFILE | grep -v FLASHFILE

Issues/PRs references

Its re-introducing make: Make make more flexible #1098 (reverted by make: fix rebuilding on changes without clean or -B #1198)
Will remove the hack for https://github.com/RIOT-OS/RIOT/pull/8801/files
~~It will need rebase after make: replace curly braces with parenthesis #8822 is merged~~

cladmi · 2018-03-27T14:15:38Z

The commits are normally all working and can be reviewed individually to see the whole process.

kaspar030 · 2018-03-27T14:31:03Z

My gut feeling is that splitting this up would speed up reviewing.

kaspar030 · 2018-03-27T14:31:44Z

My gut feeling is that splitting this up would speed up reviewing.

Ah, seems like you're already doing that. ;)

miri64 · 2018-03-28T11:44:10Z

Rebase required (unless there are more PRs this is based on)

cladmi · 2018-03-28T12:37:00Z

I plan to do another sub-PR I yet did not have time to describe alone. But will rebase.

cladmi · 2019-06-14T13:33:37Z

Re-run check to see what has not been updated recently.

cladmi · 2019-07-16T13:55:21Z

This pull request is now not waiting for other pull requests anymore and I added the end of migration commit.

The test would prevent compilation:

FLASHFILE= make --no-print-directory -C examples/hello-world/
/home/harter/work/git/RIOT/examples/hello-world/../../Makefile.include:447: *** FLASHFILE is not defined for this board: .  Stop.

In the current form, it would ignore compiling it in murdock as info-toolchains-supported would return nothing (and not fail).
I could change it to not error when info-toolchains-supported is used if requested, but not sure if it makes it more maintainable.

FLASHFILE= make --no-print-directory -C examples/hello-world/ info-toolchains-supported
/home/harter/work/git/RIOT/examples/hello-world/../../Makefile.include:447: *** FLASHFILE is not defined for this board: .  Stop.

I re-did the "Verifying after rebase" and found one remaining instance of setting OFLAGS format. It is split in #11849 but not part of this one as it is now just a duplication cleanup.

tcschmidt · 2019-08-16T23:52:06Z

@jia200x @leandrolanzieri how about this? Can this move?

Update the documentation to say it can also be 'ELFFILE' The first version was setting it to EFLFILE by default but the default behavior was removed to prevent hiding errors. It can also be overwritten for application specific needs, like when using 'riotboot'.

* Remove the transition documentation. * assert that FLASHFILE is now defined

cladmi · 2019-08-19T14:36:46Z

I rebased to master to simplify testing in case new boards would be broken.

And currently all boards correctly define FLASHFILE as this goes to the end:

for board in $(make info-boards); do BOARD=${board} make --no-print-directory -C examples/hello-world/ info-debug-variable-FLASHFILE || break; done 2>&1 | sed "s|${PWD}/||" | tee output

No error on any boards

examples/hello-world/bin/acd52832/hello-world.bin
examples/hello-world/bin/airfy-beacon/hello-world.elf
examples/hello-world/bin/arduino-due/hello-world.bin
examples/hello-world/bin/arduino-duemilanove/hello-world.hex
examples/hello-world/bin/arduino-leonardo/hello-world.hex
examples/hello-world/bin/arduino-mega2560/hello-world.hex
examples/hello-world/bin/arduino-mkr1000/hello-world.bin
examples/hello-world/bin/arduino-mkrfox1200/hello-world.bin
examples/hello-world/bin/arduino-mkrzero/hello-world.bin
examples/hello-world/bin/arduino-nano/hello-world.hex
examples/hello-world/bin/arduino-uno/hello-world.hex
examples/hello-world/bin/arduino-zero/hello-world.bin
examples/hello-world/bin/avsextrem/hello-world.hex
examples/hello-world/bin/b-l072z-lrwan1/hello-world.elf
examples/hello-world/bin/b-l475e-iot01a/hello-world.elf
examples/hello-world/bin/blackpill/hello-world.elf
examples/hello-world/bin/bluepill/hello-world.elf
examples/hello-world/bin/calliope-mini/hello-world.elf
Warning: no PORT set!
examples/hello-world/bin/cc2538dk/hello-world.bin
examples/hello-world/bin/cc2650-launchpad/hello-world.elf
examples/hello-world/bin/cc2650stk/hello-world.elf
examples/hello-world/bin/chronos/hello-world.hex
examples/hello-world/bin/ek-lm4f120xl/hello-world.elf
ESP32_SDK_DIR should be defined as /path/to/esp-idf directory
ESP32_SDK_DIR is set by default to /opt/esp/esp-idf
examples/hello-world/bin/esp32-mh-et-live-minikit/hello-world.elf
ESP32_SDK_DIR should be defined as /path/to/esp-idf directory
ESP32_SDK_DIR is set by default to /opt/esp/esp-idf
examples/hello-world/bin/esp32-olimex-evb/hello-world.elf
ESP32_SDK_DIR should be defined as /path/to/esp-idf directory
ESP32_SDK_DIR is set by default to /opt/esp/esp-idf
examples/hello-world/bin/esp32-wemos-lolin-d32-pro/hello-world.elf
ESP32_SDK_DIR should be defined as /path/to/esp-idf directory
ESP32_SDK_DIR is set by default to /opt/esp/esp-idf
examples/hello-world/bin/esp32-wroom-32/hello-world.elf
ESP32_SDK_DIR should be defined as /path/to/esp-idf directory
ESP32_SDK_DIR is set by default to /opt/esp/esp-idf
examples/hello-world/bin/esp32-wrover-kit/hello-world.elf
ESP8266_NEWLIB_DIR should be defined as /path/to/newlib directory
ESP8266_NEWLIB_DIR is set by default to /opt/esp/newlib-xtensa
ESP8266_SDK_DIR should be defined as /path/to/sdk directory
ESP8266_SDK_DIR is set by default to /opt/esp/esp-open-sdk/sdk
examples/hello-world/bin/esp8266-esp-12x/hello-world.elf
ESP8266_NEWLIB_DIR should be defined as /path/to/newlib directory
ESP8266_NEWLIB_DIR is set by default to /opt/esp/newlib-xtensa
ESP8266_SDK_DIR should be defined as /path/to/sdk directory
ESP8266_SDK_DIR is set by default to /opt/esp/esp-open-sdk/sdk
examples/hello-world/bin/esp8266-olimex-mod/hello-world.elf
ESP8266_NEWLIB_DIR should be defined as /path/to/newlib directory
ESP8266_NEWLIB_DIR is set by default to /opt/esp/newlib-xtensa
ESP8266_SDK_DIR should be defined as /path/to/sdk directory
ESP8266_SDK_DIR is set by default to /opt/esp/esp-open-sdk/sdk
examples/hello-world/bin/esp8266-sparkfun-thing/hello-world.elf
examples/hello-world/bin/f4vi1/hello-world.bin
examples/hello-world/bin/feather-m0/hello-world.bin
examples/hello-world/bin/firefly/hello-world.bin
examples/hello-world/bin/fox/hello-world.elf
examples/hello-world/bin/frdm-k22f/hello-world.elf
examples/hello-world/bin/frdm-k64f/hello-world.elf
examples/hello-world/bin/frdm-kw41z/hello-world.elf
examples/hello-world/bin/hamilton/hello-world.bin
examples/hello-world/bin/hifive1/hello-world.elf
examples/hello-world/bin/hifive1b/hello-world.bin
examples/hello-world/bin/i-nucleo-lrwan1/hello-world.elf
examples/hello-world/bin/ikea-tradfri/hello-world.bin
examples/hello-world/bin/iotlab-a8-m3/hello-world.elf
examples/hello-world/bin/iotlab-m3/hello-world.elf
examples/hello-world/bin/jiminy-mega256rfr2/hello-world.hex
examples/hello-world/bin/limifrog-v1/hello-world.elf
examples/hello-world/bin/lobaro-lorabox/hello-world.bin
examples/hello-world/bin/lsn50/hello-world.elf
examples/hello-world/bin/maple-mini/hello-world.elf
examples/hello-world/bin/mbed_lpc1768/hello-world.bin
examples/hello-world/bin/mega-xplained/hello-world.hex
examples/hello-world/bin/microbit/hello-world.elf
examples/hello-world/bin/msb-430/hello-world.hex
examples/hello-world/bin/msb-430h/hello-world.hex
examples/hello-world/bin/msba2/hello-world.hex
examples/hello-world/bin/msbiot/hello-world.elf
examples/hello-world/bin/mulle/hello-world.elf
examples/hello-world/bin/native/hello-world.elf
examples/hello-world/bin/nrf51dk/hello-world.elf
examples/hello-world/bin/nrf51dongle/hello-world.bin
examples/hello-world/bin/nrf52832-mdk/hello-world.hex
examples/hello-world/bin/nrf52840-mdk/hello-world.hex
examples/hello-world/bin/nrf52840dk/hello-world.bin
examples/hello-world/bin/nrf52dk/hello-world.bin
examples/hello-world/bin/nrf6310/hello-world.bin
examples/hello-world/bin/nucleo-f030r8/hello-world.elf
examples/hello-world/bin/nucleo-f031k6/hello-world.elf
examples/hello-world/bin/nucleo-f042k6/hello-world.elf
examples/hello-world/bin/nucleo-f070rb/hello-world.elf
examples/hello-world/bin/nucleo-f072rb/hello-world.elf
examples/hello-world/bin/nucleo-f091rc/hello-world.elf
examples/hello-world/bin/nucleo-f103rb/hello-world.elf
examples/hello-world/bin/nucleo-f207zg/hello-world.elf
examples/hello-world/bin/nucleo-f302r8/hello-world.elf
examples/hello-world/bin/nucleo-f303k8/hello-world.elf
examples/hello-world/bin/nucleo-f303re/hello-world.elf
examples/hello-world/bin/nucleo-f303ze/hello-world.elf
examples/hello-world/bin/nucleo-f334r8/hello-world.elf
examples/hello-world/bin/nucleo-f401re/hello-world.elf
examples/hello-world/bin/nucleo-f410rb/hello-world.elf
examples/hello-world/bin/nucleo-f411re/hello-world.elf
examples/hello-world/bin/nucleo-f412zg/hello-world.elf
examples/hello-world/bin/nucleo-f413zh/hello-world.elf
examples/hello-world/bin/nucleo-f429zi/hello-world.elf
examples/hello-world/bin/nucleo-f446re/hello-world.elf
examples/hello-world/bin/nucleo-f446ze/hello-world.elf
examples/hello-world/bin/nucleo-f722ze/hello-world.elf
examples/hello-world/bin/nucleo-f746zg/hello-world.elf
examples/hello-world/bin/nucleo-f767zi/hello-world.elf
examples/hello-world/bin/nucleo-l031k6/hello-world.elf
examples/hello-world/bin/nucleo-l053r8/hello-world.elf
examples/hello-world/bin/nucleo-l073rz/hello-world.elf
examples/hello-world/bin/nucleo-l152re/hello-world.elf
examples/hello-world/bin/nucleo-l432kc/hello-world.elf
examples/hello-world/bin/nucleo-l433rc/hello-world.elf
examples/hello-world/bin/nucleo-l452re/hello-world.elf
examples/hello-world/bin/nucleo-l476rg/hello-world.elf
examples/hello-world/bin/nucleo-l496zg/hello-world.elf
examples/hello-world/bin/nucleo-l4r5zi/hello-world.elf
examples/hello-world/bin/nz32-sc151/hello-world.bin
examples/hello-world/bin/opencm904/hello-world.bin
examples/hello-world/bin/openmote-b/hello-world.hex
examples/hello-world/bin/openmote-cc2538/hello-world.bin
examples/hello-world/bin/particle-argon/hello-world.hex
examples/hello-world/bin/particle-boron/hello-world.hex
examples/hello-world/bin/particle-xenon/hello-world.hex
examples/hello-world/bin/pba-d-01-kw2x/hello-world.elf
examples/hello-world/bin/phynode-kw41z/hello-world.elf
examples/hello-world/bin/pic32-clicker/hello-world.hex
examples/hello-world/bin/pic32-wifire/hello-world.hex
examples/hello-world/bin/pyboard/hello-world.bin
examples/hello-world/bin/reel/hello-world.elf
examples/hello-world/bin/remote-pa/hello-world.bin
examples/hello-world/bin/remote-reva/hello-world.bin
examples/hello-world/bin/remote-revb/hello-world.bin
examples/hello-world/bin/ruuvitag/hello-world.bin
examples/hello-world/bin/samd21-xpro/hello-world.bin
examples/hello-world/bin/same54-xpro/hello-world.bin
examples/hello-world/bin/saml10-xpro/hello-world.bin
examples/hello-world/bin/saml11-xpro/hello-world.bin
examples/hello-world/bin/saml21-xpro/hello-world.bin
examples/hello-world/bin/samr21-xpro/hello-world.bin
examples/hello-world/bin/samr30-xpro/hello-world.bin
examples/hello-world/bin/seeeduino_arch-pro/hello-world.hex
examples/hello-world/bin/sensebox_samd21/hello-world.bin
examples/hello-world/bin/slstk3401a/hello-world.bin
examples/hello-world/bin/slstk3402a/hello-world.bin
examples/hello-world/bin/sltb001a/hello-world.bin
examples/hello-world/bin/slwstk6000b/hello-world.bin
examples/hello-world/bin/slwstk6220a/hello-world.bin
examples/hello-world/bin/sodaq-autonomo/hello-world.bin
examples/hello-world/bin/sodaq-explorer/hello-world.bin
examples/hello-world/bin/sodaq-one/hello-world.bin
examples/hello-world/bin/sodaq-sara-aff/hello-world.bin
examples/hello-world/bin/spark-core/hello-world.bin
examples/hello-world/bin/stk3600/hello-world.bin
examples/hello-world/bin/stk3700/hello-world.bin
examples/hello-world/bin/stm32f0discovery/hello-world.elf
examples/hello-world/bin/stm32f3discovery/hello-world.elf
examples/hello-world/bin/stm32f429i-disc1/hello-world.elf
examples/hello-world/bin/stm32f4discovery/hello-world.elf
examples/hello-world/bin/stm32f769i-disco/hello-world.elf
examples/hello-world/bin/stm32l0538-disco/hello-world.elf
examples/hello-world/bin/stm32l476g-disco/hello-world.elf
examples/hello-world/bin/teensy31/hello-world.hex
examples/hello-world/bin/telosb/hello-world.hex
examples/hello-world/bin/thingy52/hello-world.bin
examples/hello-world/bin/ublox-c030-u201/hello-world.elf
examples/hello-world/bin/udoo/hello-world.bin
examples/hello-world/bin/usb-kw41z/hello-world.elf
examples/hello-world/bin/waspmote-pro/hello-world.hex
examples/hello-world/bin/wsn430-v1_3b/hello-world.hex
examples/hello-world/bin/wsn430-v1_4/hello-world.hex
examples/hello-world/bin/yunjia-nrf51822/hello-world.elf
examples/hello-world/bin/z1/hello-world.hex

MrKevinWeiss

It seems like everything is up to date. If not I guess murdock will complain with this PR.

ACK!

cladmi · 2019-08-19T16:26:18Z

I think it would not complain for the boards in
TEST_BOARDS_LLVM_COMPILE:="iotlab-m3 native nrf52dk mulle nucleo-f401re samr21-xpro slstk3402a" if FLASHFILE is not defined as it would be removed by

RIOT/.murdock

Lines 120 to 124 in 552e425

    
               if is_in_list "${board}" "${TEST_BOARDS_LLVM_COMPILE}"; then 
        
                   toolchains="$(make -s --no-print-directory -C${appdir} BOARD=${board} \ 
        
                                 info-toolchains-supported 2> /dev/null | grep -o -e "llvm" -e "gnu")" 
        
               fi 
        
               echo "${toolchains}"

But the test from #8838 (comment) would have reported an issue.
And any local compilation would fail.

cladmi · 2019-08-20T09:57:14Z

As I was asked precision about my comment IRL.

For the boards in TEST_BOARDS_LLVM_COMPILE, if the FLASHFILE was not defined, the reported supported toolchains would be empty, and I think it would be ignored in murdock compilation.

FLASHFILE= BOARD=iotlab-m3 make -s --no-print-directory -C examples/hello-world/  info-toolchains-supported 2> /dev/null | grep -o -e "llvm" -e "gnu"
# empty answer

However, the test in my #8838 (comment) comment would have reported error if a current board was not setting FLASHFILE with the default PROGRAMMER configuration.

If a new board is added, or boards are refactored, a local compilation should also be done which would report the missing FLASHFILE.

I could add an handling for this that detects if the reported toolchain is empty. However, for non default PROGRAMMER, we would would also only see it when trying to compile for them as it will not be checked by murdock.

Also, any other errors during the makefile (that is not info-boards-supported) would make the board ignored by murdock so it is not specific to this.

MrKevinWeiss · 2019-08-20T14:06:22Z

Thanks update. I still think this is good to go!

cladmi · 2019-08-20T14:17:19Z

Thank you for the review.

Took 1 year and a half to get rid of such a bad pattern D: Finally done.

cladmi added Area: build system Area: Build system Type: cleanup The issue proposes a clean-up / The PR cleans-up parts of the codebase / documentation State: waiting for other PR State: The PR requires another PR to be merged first labels Mar 27, 2018

cladmi force-pushed the pr/big/cleanup/add_flashfile branch 2 times, most recently from 967ed6b to 9c0eebf Compare March 27, 2018 13:51

cladmi added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Mar 27, 2018

cladmi mentioned this pull request Mar 27, 2018

Makefile.include: refactor command present check #8839

Merged

cladmi requested review from aabadie, jia200x, jnohlgard, kaspar030 and miri64 March 27, 2018 14:08

cladmi mentioned this pull request Mar 27, 2018

makefiles: FIX boards generating a binary '.hex' file #8840

Merged

miri64 added State: waiting for other PR State: The PR requires another PR to be merged first and removed State: waiting for other PR State: The PR requires another PR to be merged first labels Mar 27, 2018

cladmi force-pushed the pr/big/cleanup/add_flashfile branch 2 times, most recently from e16e238 to 891ebdf Compare March 28, 2018 13:31

This was referenced Mar 28, 2018

Makefile.include: separate link in subtargets #8844

Merged

Makefile.include: add %.hex and %.bin rules #8845

Merged

cladmi force-pushed the pr/big/cleanup/add_flashfile branch 2 times, most recently from 0409eb5 to 9d48995 Compare April 9, 2018 16:02

cladmi removed the State: waiting for other PR State: The PR requires another PR to be merged first label Apr 9, 2018

cladmi force-pushed the pr/big/cleanup/add_flashfile branch from 9d48995 to 27a7b63 Compare April 9, 2018 16:25

cladmi mentioned this pull request Apr 9, 2018

robotis-loader.py: fix flake8 errors #8903

Merged

cladmi force-pushed the pr/big/cleanup/add_flashfile branch from 19453e8 to 72d6ab9 Compare April 9, 2018 16:47

cladmi force-pushed the pr/big/cleanup/add_flashfile branch 2 times, most recently from eec068e to 27f1b66 Compare June 5, 2019 12:40

cladmi mentioned this pull request Jun 6, 2019

cpu/esp*: use FLASHFILE for esp32 and esp8266 boards #11648

Merged

cladmi mentioned this pull request Jun 14, 2019

makefiles/pyocd.inc.mk use FLASHFILE #11696

Merged

cladmi force-pushed the pr/big/cleanup/add_flashfile branch from 27f1b66 to 8fb6418 Compare June 14, 2019 10:38

cladmi mentioned this pull request Jun 14, 2019

boards/pic32-xx: set FLASHFILE to HEXFILE #11699

Merged

cladmi force-pushed the pr/big/cleanup/add_flashfile branch from 8fb6418 to 795f150 Compare June 14, 2019 13:32

cladmi added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: run tests If set, CI server will run tests on hardware for the labeled PR labels Jun 14, 2019

This was referenced Jun 27, 2019

boards/native: define FLASHFILE #11752

Merged

makefiles/murdock.inc.mk: remove flashfile hack #11755

Merged

Makefile.include: do not build HEXFILE by default anymore #11757

Merged

This was referenced Jul 15, 2019

makefiles: documentation for hexfile binfile #11845

Merged

boards/hamilton: remove OFLAGS '-O binary' #11849

Merged

cladmi added 2 commits August 19, 2019 16:30

makefiles: end of 'FLASHFILE' transition phase

ce8d10c

* Remove the transition documentation. * assert that FLASHFILE is now defined

MrKevinWeiss approved these changes Aug 19, 2019

View reviewed changes

Conversation

cladmi commented Mar 27, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution description

Integrated in sub PRs:

Verifying after rebase:

Issues/PRs references

Uh oh!

cladmi commented Mar 27, 2018

Uh oh!

kaspar030 commented Mar 27, 2018

Uh oh!

kaspar030 commented Mar 27, 2018

Uh oh!

miri64 commented Mar 28, 2018

Uh oh!

cladmi commented Mar 28, 2018

Uh oh!

cladmi commented Jun 14, 2019

Uh oh!

cladmi commented Jul 16, 2019

Uh oh!

tcschmidt commented Aug 16, 2019

Uh oh!

cladmi commented Aug 19, 2019

Uh oh!

MrKevinWeiss left a comment

Choose a reason for hiding this comment

Uh oh!

cladmi commented Aug 19, 2019

Uh oh!

cladmi commented Aug 20, 2019

Uh oh!

MrKevinWeiss commented Aug 20, 2019

Uh oh!

cladmi commented Aug 20, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

cladmi commented Mar 27, 2018 •

edited

Loading