makefiles/blobs: Add rules for binary blob embedding. #11497
makefiles/blobs: Add rules for binary blob embedding. #11497jcarrano wants to merge 4 commits intoRIOT-OS:masterfrom
Conversation
ld is necessary for some low-level linking operation. For example, to generate an object file from a binary file.
This commit adds a new makefile module for embedding raw binary data into executables. These is useful for including resources such as graphics (for UIs), certificates, crypto keys, scripts, etc. Previously such a global rule did not exist and people had to resort to either provide a pre-generated C-file with an array or hand-craft a (header) file with xxd. The advantage of the new rules are: - No additional tools required (uses only the toolchain's tools) - Declared data types are easy to configure (no sed magic.) - No arrays in headers. - Output section is configurable (thismakes it possible to include code, not only data.)
Remove the xxd rules. Adding files to BLOBS looks way cleaner and also provides an example of how the BLOB function can be used.
jcarrano
added
Area: build system
|
Thanks for working on this!
Whats wrong with that? Its at least industry best practice, and the performance overhead is negligible. |
| # Note: _binary_$1_size is an "absolute" symbol. Because of relocation in | ||
| # virtual memory systems (i.e. native) this will not work in such systems. | ||
| # instead, we will have to hardcode the blob size. | ||
| define blob_header_template |
There was a problem hiding this comment.
Templating in a Makefile?
Why not implement the whole thing in a bash shell script?
|
|
||
| .INTERMEDIATE: $(BLOBS:%=$(BLOB_PATH)/%.o.tmp) | ||
|
|
||
| $(BLOBS:%=$(BLOB_PATH)/%.o.tmp): $(BLOB_PATH)/%.o.tmp: % | $(BLOB_PATH)/ |
There was a problem hiding this comment.
Why use ld at all? objcopy can also create .o from blobs with a similar naming scheme.
If the array was in a C file, then OK. But it is in a header, meaning that:
As for creating a C file with an array, I don't remember the exact details, but it didn't play so nice with our build system. The array approach does not allow one to embed code. Things like softdevice, for example, still require two images to be flashed. |
kaspar030
left a comment
kaspar030
left a comment
There was a problem hiding this comment.
I agree that we need proper blob handling, and I do see that using xxd is difficult mostly because it is part of vim and not generally available. If it would, the sed magic would be acceptable.
I don't like having the functionality implemented using Makefile functions. IMO it would make much more sense to have this as a tool that is just called from make.
I don't buy the "array is bad". Having blobs declared within a header as embedded binary might not be the most efficient cycle wise, but when it comes to handling, it is much simpler. No need for handling two files (generated header and generated .o, the latter having two creation steps), arch specific stuff whatsoever, anything. For 99% of the uses, specifying a variable (base)name and whether it should be const or not is sufficient. For 1% it might make sense to be able to specify the target section and change the data type.
why not |
|
#11870 was merged. Is this one still relevant ? I'm for closing it. @kaspar030 @miri64 what do you think ? |
4 participants
Contribution description
This PR adds a new makefile module for embedding raw binary data intoexecutables. These is useful for including resources such as graphics (for UIs), certificates, crypto keys, scripts, etc.
Previously such a global rule did not exist and people had to resort to either provide a pre-generated C-file with an array or hand-craft a (header) file with xxd. The advantage of the new rules are:
Testing procedure
I migrated the two Lua examples to the new system. They should compile and run without problems.
Issues/PRs references
Partial alternative to #9565.
See, for example, this file: https://github.com/RIOT-OS/RIOT/pull/10308/files#diff-09ce13303248a398e36e3febb371129a