pkg/nanopb: add nanoPb protocol buffers library package

[kaspar030]

Contribution description

This PR adds a package for easy usage of google protocol buffers.
It adds some logic to automatically generate *.c files from *.proto files within a module's source folder.

In order to do so, a prerequisite commit adds generic support for generated sources.

Testing procedure

See tests/pkg_nanopb/README.md for usage info. Specifically, in order to test this, you'll need to install the official protobuf compiler, it's python bindings and the nanopb generator plugin.

As that generator is not (yet) part of the build container, the test cannot be compiled on the CI.

This is an alternative to #11157. Differences:

• includes generic way to generate sources using tools like nanopb.
• uses official test code
• simpler package source compilation through using custom (one-line) Makefile instead of calling upstream cmake

Issues/PRs references

#11157

[benpicco]

I still want that package 😃
But I really think we should use the repo that gets downloaded through the pkg instead of having the user download it again manually.

If you don't have time for this right now I can also make a PR against your branch for this.

[benpicco]

Looks good now 😉
You can squash.

[kaspar030]

There's one issue here: CI is gonna be missing protoc. Please don't merge yet (the build should fail now when running the tests anyways)

[benpicco]

Will it be enough to add protobuf-compiler to requirements.txt in riotdocker?

[kaspar030]

Will it be enough to add protobuf-compiler to requirements.txt in riotdocker?

It should, apart from updating the murdock workers afterwards.

I guess at that point we need to make sure that building in docker ("BUILD_IN_DOCKER=1") works as expected...

BTW, thanks for helping out!

[benpicco]

Hm, is there a way to overwrite PKG_VERSION from the application?
I just realized that #11157 pointed to the master (soon to be 0.4.0) branch and it contains an API change from the 0.3.x branch.

But the package works as advertised :)

[miri64]

There is #11533 (comment). However, setting the PKG_VERSION from outside the build system would also be great (needs some kind of scoping, though)

[kaspar030]

We need do check if pkg/nanopb/Makefile.gensrc as is would work if multiple modules make use of GENSRC. Also I'm not sure as is if non-application modules can use protobuf files the way it is implemented now.

[kaspar030]

• squashed

@benpicco this might just make it into the release. 😉

[kaspar030]

@benpicco this might just make it into the release. wink

Hm seems like the python support is still missing... :(

[benpicco]

The code looks good and it works.
We're already pulling this branch into our development build, so it's being used already.

[kaspar030]

We're already pulling this branch into our development build, so it's being used already.

Perfect. I've added a commit to disable tests on CI for now.

[kaspar030]

Sorry @benpicco, python-protobuf is actually needed for building ;(

[kaspar030]

@benpicco I did a test run with the soon-to-be-live build container. It failed as we're installing the python 3 protobuf module, but nanopb's generator defaults to python2...

I've worked around that by patching the generator to use python3. Seems to work fine. Could you take a look?

[benpicco]

So with RIOT-OS/riotdocker#89 merged, can we give Murdock another try?

[benpicco]

You can already squash, so hopefully everything will be green now :)

[kaspar030]

This passed all tests. @benpicco I'll give you the honor! 😉

[benpicco]

Thank you for providing the package & build system integration!

[kaspar030]

Thank you for providing the package & build system integration!

Thanks for the review!