Skip to content

pkg/mbedtls: initial pkg import to use entropy module#15671

Merged
benpicco merged 1 commit intoRIOT-OS:masterfrom
PeterKietzmann:pr_add_pkg_mbedtls
Jan 18, 2022
Merged

pkg/mbedtls: initial pkg import to use entropy module#15671
benpicco merged 1 commit intoRIOT-OS:masterfrom
PeterKietzmann:pr_add_pkg_mbedtls

Conversation

@PeterKietzmann
Copy link
Member

Contribution description

This PR adds mbed TLS as a package to be used for entropy generation, and it is the only module that has been tested in RIOT so far. However, this PR is an enabler for additional mbed TLS features in RIOT.

The PR includes a simple RIOT API to access the entropy generated by mbed TLS. Entropy sources that feed the module are currently ADC Noise (#14324) and the HWRNG. Further information can be found in the package documentation.

Testing procedure

Issues/PRs references

@PeterKietzmann PeterKietzmann added Area: crypto Area: Cryptographic libraries Area: pkg Area: External package ports Area: security Area: Security-related libraries and subsystems Type: new feature The issue requests / The PR implemements a new feature for RIOT labels Dec 17, 2020
@@ -0,0 +1,3344 @@
/**
* \file config.h
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be mistaking, but this files seems similar to what I did with lwipopts.h in lwIP. Does this file has to be exactly this format or can we adapt it to be more RIOT-y? The fact, that this file is 96% comments, makes me think the latter.

I also don't think that you have to be that verbose on documentation, as the documentation (hopefully) lies with mbedtls (which you could just link here).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file is a copy-paste from mbed TLS. For sure I could write a similar RIOT(-y) file, but for further integration of the package, I think it simplifies the development process to have a (almost) copy available. With that, a developer (i) sees immediately all available configuration options (&doc) and (ii) can easily look at diffs to the original file during version bump or the like...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mh from the other point of view, I might be interested as a user, which configuration options of mbedtls are actually used. Scrolling though this 3300 line file to filter out the 142 lines that are actually important seems cumbersome, especially given, that you want to exclude this file from out doxygen build. Furthermore, we might want to make options configurable from the app (like we do e.g. with lwIP...), which seems a bit more complicated when you need to comment out / comment in defines.

I also don't see the maintenance advantage you propose. If you want to see which options are available in mbedtls: go to the mbedtls documentation. If there is a version bump and config options change this will usually lead to failing compilation, giving you exactly the names of the macros that are new (and thus missing).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From #15671 (comment)

Lets give it some more time and if no one opposes, we'll go with your proposal.

Yes, please @RIOT-OS/maintainers, can we have more thoughts and comments on this?

Copy link
Contributor

@aabadie aabadie Dec 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I haven't checked the code, just followed the discussion)
If we can avoid the 3300 lines of configuration code, that would be better. If it's added, there's no reason to skip it from doxygen, or any other checks.
IIUC, the goal of this file is to provide the configuration of the package. Why not link it to the module mechanism of RIOT to define flags (like MBEDTLS_SHA256_ALT) or via a Kconfig file or both (depending on the cases) ?

  • for the module version:
#if IS_USED(MODULE_MBEDTLS_CIPHER_MODE_CBC)
#define MBEDTLS_CIPHER_MODE_CBC
#endif
  • for the Kconfig version (assuming there's a corresponding Kconfig boolean option):
#if IS_ACTIVE(CONFIG_MBEDTLS_ECP_DP_SECP192R1_ENABLED)
#define MBEDTLS_ECP_DP_SECP192R1_ENABLED
#endif

@PeterKietzmann
Copy link
Member Author

To answer @miri64's question in this comment:

And then there is the question why you want to exclude it in the first place? pkg/mbedtls/include/riot_config.h seems very RIOT-specific to me, so I guess it is not an external import. If it is: since you are in a pkg, can't you import it via the pkg mechanism as well?

It is an mbed TLS config file which I slightly adjusted for RIOT, by en-/disabling some options. By enabling additional library features, further changes might be required. To be still mostly "copy-paste-able" I've tried to keep changes at minimum.

Technically I could import & patch it. Practically, as a user, I would like the have the file directly available instead of a patch.

@@ -0,0 +1,73 @@
/*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be its on module so in the future people can use mbedtls without the entropy source (if they so desire)?

So pkg/mbedtls/contrib/mbedtls_entropy_source/.... This way you also might not need to define pseudo-modules.

Again: see lwIP how to implement actual modules within the contrib directory of a package.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I have tried this in my last commit, mind to have another look?

@miri64
Copy link
Member

miri64 commented Dec 17, 2020

It is an mbed TLS config file which I slightly adjusted for RIOT, by en-/disabling some options. By enabling additional library features, further changes might be required. To be still mostly "copy-paste-able" I've tried to keep changes at minimum.

Why do you want to have it "copy-paste-able"? Copy-paste from where? Is configuration so complicated within mbedtls that you need to copy files around to keep track of your configuration? I highly doubt any of this, given the popularity and porting status of the library.

Technically I could import & patch it. Practically, as a user, I would like the have the file directly available instead of a patch.

@PeterKietzmann
Copy link
Member Author

Is configuration so complicated within mbedtls

Well, I guess it strongly depends on the point of view. A look at the amount of configuration options in that file gives you a first impression. Personally (proposing the library import here) I don't have a problem providing a specific configuration file. Still, I could think of other developers seeing this differently. Lets give it some more time and if no one opposes, we'll go with your proposal.

Copy link
Contributor

@aabadie aabadie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went through this PR and have some comment related to the build system and other minor things.
I also think that the riot_config.h should be rewritten and not excluded from the doxygen build (since it might contain important configuration information) and the CI checks.

@@ -0,0 +1,10 @@
PKG_NAME=mbedtls
PKG_URL=https://github.com/ARMmbed/mbedtls.git
PKG_VERSION=2a1d9332d55d1270084232e42df08fdb08129f1b
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this commit hash correspond to a tagged version, please add it in a comment at the end of the line


all:
"$(MAKE)" -C $(PKGDIRBASE)/mbedtls/library/ \
-f $(RIOTPKG)/mbedtls/Makefile.$(PKG_NAME)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makefile.mbedtls only contains a minimal setup, you could just use Makefile.base directly and pass the MODULE variable in the command line:

Suggested change
-f $(RIOTPKG)/mbedtls/Makefile.$(PKG_NAME)
-f $(RIOTBASE)/Makefile.base MODULE=$(PKG_NAME)

And Makefile.mbedtls can be removed.

# include sources if hardware is available
FEATURES_OPTIONAL += periph_hwrng
ifneq (,$(filter periph_hwrng,$(FEATURES_USED)))
DEFAULT_MODULE += mbedtls_entropy_source_hwrng
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indentation is off, it should be 2 spaces inside the ifneq block. Maybe it's a tab ?
Same comment applies in the rest of this file.

Comment on lines +5 to +16
# include sources if hardware is available
FEATURES_OPTIONAL += periph_hwrng
ifneq (,$(filter periph_hwrng,$(FEATURES_USED)))
DEFAULT_MODULE += mbedtls_entropy_source_hwrng
endif

FEATURES_OPTIONAL += periph_adc
ifneq (,$(filter periph_adc,$(FEATURES_USED)))
DEFAULT_MODULE += mbedtls_entropy_source_adc
endif
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the hardware provides both ? What is the entropy source selected in this case ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think those are not mutually exclusive, @PeterKietzmann ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then both are used (if not manually disabled). That is actually the intent of the entropy module, to accumulate multiple entropy sources.

Comment on lines +20 to +21
CFLAGS += -DCONFIG_ENTROPY_SOURCE_ADC_COND=0
CFLAGS += -DCONFIG_ENTROPY_SOURCE_ADC_HEALTH_TEST=1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those 2 constants seem to be unused.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, they configure the ADC noise source which was added in #14324.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be better placed in the Makefile.include?

*
* ```
* FEATURES_PROVIDED += periph_hwrng
* FEATURES_PROVIDED += entropy_source_adc_noise
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* FEATURES_PROVIDED += entropy_source_adc_noise
* FEATURES_PROVIDED += periph_adc

?

*/

/**
* @ingroup pkg_mbedtls
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* @ingroup pkg_mbedtls
* @ingroup pkg_mbedtls_entropy

?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, probably a good idea to spend an own doxygen group now that we moved the entropy support to its own module.

*/

/**
* @ingroup pkg_mbedtls
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* @ingroup pkg_mbedtls
* @ingroup pkg_mbedtls_entropy

?

extern "C" {
#endif

#if defined (MBEDTLS_SHA256_ALT)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#if defined (MBEDTLS_SHA256_ALT)
#if IS_ACTIVE(MBEDTLS_SHA256_ALT)

Comment on lines +11 to +21
ifneq (,$(filter mbedtls_entropy_source_adc,$(USEMODULE)))
ifndef CONFIG_KCONFIG_USEMODULE_ENTROPY_SOURCE_ADC_NOISE
CFLAGS += -DCONFIG_ENTROPY_SOURCE_ADC_HMIN=65536
endif
endif
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't you put this in the package Makefile.include ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid mis-configuration of this entropy source, we set an invalid default value. Hence, a user must evaluate and configure the module according to the hardware in use and the deployment. This line prevents failures during testing.

@PeterKietzmann
Copy link
Member Author

@aabadie many thanks for your review. I have left some answers inline and will work on the other comments. Regarding the configuration file there seems to be a favor for the RIOT-style option. Lets give it some time anyway, and I will provide an alternative during the next couple of days if no one objects.

@PeterKietzmann PeterKietzmann force-pushed the pr_add_pkg_mbedtls branch 3 times, most recently from d11ef29 to f7a7105 Compare December 18, 2020 15:11
@PeterKietzmann
Copy link
Member Author

@aabadie I have addressed your comments (despite the config header file as described above)

@fjmolinas
Copy link
Contributor

ping @aabadie

@MrKevinWeiss MrKevinWeiss added this to the Release 2021.07 milestone Jun 21, 2021
@github-actions github-actions bot added the Area: doc Area: Documentation label Jul 5, 2021
@github-actions github-actions bot removed Area: timers Area: timer subsystems Platform: AVR Platform: This PR/issue effects AVR-based platforms Area: build system Area: Build system Area: examples Area: Example Applications Platform: ARM Platform: This PR/issue effects ARM-based platforms Area: arduino API Area: Arduino wrapper API Area: LoRa Area: LoRa radio support labels Jul 8, 2021
@PeterKietzmann
Copy link
Member Author

Added Kconfig options, enabled config header for doxygen again, tested on native and nrf52840dk, cleaned up minor stuff...

@aabadie, mind to give it an other look to remove the change request?

@PeterKietzmann
Copy link
Member Author

Btw: all tests now pass

@fjmolinas
Copy link
Contributor

@leandrolanzieri is this one ready to go?

Comment on lines +20 to +21
CFLAGS += -DCONFIG_ENTROPY_SOURCE_ADC_COND=0
CFLAGS += -DCONFIG_ENTROPY_SOURCE_ADC_HEALTH_TEST=1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be better placed in the Makefile.include?

Comment on lines +80 to +83
#if defined(MODULE_MBEDTLS_ENTROPY_SOURCE_ADC)
entropy_source_adc_init();
#endif
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this init be needed for additional sources? Would it make sense to add some interface to initialize the selected sources?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It depends on the source, but seems likely that most sources have some kind of an init function. Technically, we could add .init function to entropy_source_mbedtls_riot_t but I would like to delay that, since other sources (e.g. PUF) might have very different requirements from ADC sampling or HWRNG readout.

Comment on lines +84 to +91
for (unsigned i = 1; i < NUM_ENTROPY_FUNCS; i++) {
ret = mbedtls_entropy_add_source(ctx, entropy_funcs[i].func, NULL,
1, entropy_funcs[i].strong);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's something wrong with the ret usage. For example, if something goes wrong with the first entropy source (ret != 0) but works with the last, the whole function return code will be 0. I don't think this is expected.

Maybe it's better to check the return value in the for loop and return with an error if any of source cannot be added ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The loop adds entropy sources to a list which will be polled during entropy gathering. The error case occurs when a maximum number of sources have been added. Hence, the case of failure followed by success in not true. Nevertheless, we can save some redundant calls by returning on the first error.

@PeterKietzmann
Copy link
Member Author

PeterKietzmann commented Nov 23, 2021

@aabadie , @leandrolanzieri your latest comments should be addressed in this commit aef6196

@@ -0,0 +1,10 @@
PKG_NAME=mbedtls
PKG_URL=https://github.com/ARMmbed/mbedtls.git
PKG_VERSION=v2.26.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use v3.0.0 here? (or at least v2.27.0)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

v2.26.0 was up to date when I opened this PR. I worked and tested the most with this version. Sticking with it seemed better to me than "blindly" bumping the version.

Copy link
Contributor

@benpicco benpicco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please squash!

@fjmolinas
Copy link
Contributor

@PeterKietzmann the test fails on samr-21xpro (and a couple of other BOARDs I tested locally, some report a "failed", others hang), is that expected? The PR only mentioned testing on native, is it supposed to work on hardware?

@PeterKietzmann
Copy link
Member Author

is it supposed to work on hardware

Yes it is. I ran the test on different platforms, back then. Cannot remember any high-ups. Just wondering: who has tested this PR before merging? Anyway, I will look into this as soon as I can!

@PeterKietzmann
Copy link
Member Author

some report a "failed", others hang),

@fjmolinas can you name these boards so I have a starting point?

@fjmolinas
Copy link
Contributor

some report a "failed", others hang),

@fjmolinas can you name these boards so I have a starting point?

Nightlies are failing on samr21-xpro, I got a hang on nucleo-l152re. I'll run the test on all the BOARDs I have to give a complete list, will do on this PR and master.

@fjmolinas
Copy link
Contributor

I ran the tests on the PR and on master.

Results (ignore failed compilations, there is some kind of raise condition with pkg downloads...):

master
### nucleo-g071rb/failuresummary

Failures during compilation:
- [tests/pkg_mbedtls](nucleo-g071rb/tests/pkg_mbedtls/compilation.failed)

### nucleo-f207zg/failuresummary


### nucleo-f334r8/failuresummary


### nucleo-g474re/failuresummary

Failures during compilation:
- [tests/pkg_mbedtls](nucleo-g474re/tests/pkg_mbedtls/compilation.failed)

### nrf52dk/failuresummary


### frdm-k64f/failuresummary


### nucleo-l152re/failuresummary

Failures during test:
- [tests/pkg_mbedtls](nucleo-l152re/tests/pkg_mbedtls/test.failed)

### arduino-uno/failuresummary


### nucleo-f767zi/failuresummary

Failures during compilation:
- [tests/pkg_mbedtls](nucleo-f767zi/tests/pkg_mbedtls/compilation.failed)

### samr21-xpro/failuresummary

Failures during test:
- [tests/pkg_mbedtls](samr21-xpro/tests/pkg_mbedtls/test.failed)

### p-nucleo-wb55/failuresummary


### nucleo-l496zg/failuresummary


### nucleo-l452re/failuresummary


### frdm-kw41z/failuresummary

Failures during test:
- [tests/pkg_mbedtls](frdm-kw41z/tests/pkg_mbedtls/test.failed)

### nucleo-l433rc/failuresummary

Failures during compilation:
- [tests/pkg_mbedtls](nucleo-l433rc/tests/pkg_mbedtls/compilation.failed)

### i-nucleo-lrwan1/failuresummary


### openmote-b/failuresummary


### nucleo-wl55jc/failuresummary


### nucleo-l4r5zi/failuresummary


### z1/failuresummary


### nucleo-f091rc/failuresummary

Failures during test:
- [tests/pkg_mbedtls](nucleo-f091rc/tests/pkg_mbedtls/test.failed)

### arduino-zero/failuresummary

Failures during test:
- [tests/pkg_mbedtls](arduino-zero/tests/pkg_mbedtls/test.failed)

### nucleo-l552ze-q/failuresummary


### nucleo-f303k8/failuresummary


### nucleo-f103rb/failuresummary


### b-l072z-lrwan1/failuresummary


### atmega256rfr2-xpro/failuresummary

Failures during test:
- [tests/pkg_mbedtls](atmega256rfr2-xpro/tests/pkg_mbedtls/test.failed)

### nucleo-f303re/failuresummary


### slstk3402a/failuresummary


### nucleo-f030r8/failuresummary

Failures during test:
- [tests/pkg_mbedtls](nucleo-f030r8/tests/pkg_mbedtls/test.failed)

### nucleo-l073rz/failuresummary

pr

### nucleo-g071rb/failuresummary

Failures during test:
- [tests/pkg_mbedtls](nucleo-g071rb/tests/pkg_mbedtls/test.failed)

### nucleo-f207zg/failuresummary

Failures during compilation:
- [tests/pkg_mbedtls](nucleo-f207zg/tests/pkg_mbedtls/compilation.failed)

### nucleo-f334r8/failuresummary


### nucleo-g474re/failuresummary

Failures during compilation:
- [tests/pkg_mbedtls](nucleo-g474re/tests/pkg_mbedtls/compilation.failed)

### nrf52dk/failuresummary


### frdm-k64f/failuresummary


### nucleo-l152re/failuresummary

Failures during test:
- [tests/pkg_mbedtls](nucleo-l152re/tests/pkg_mbedtls/test.failed)

### arduino-uno/failuresummary


### nucleo-f767zi/failuresummary

Failures during compilation:
- [tests/pkg_mbedtls](nucleo-f767zi/tests/pkg_mbedtls/compilation.failed)

### samr21-xpro/failuresummary

Failures during test:
- [tests/pkg_mbedtls](samr21-xpro/tests/pkg_mbedtls/test.failed)

### p-nucleo-wb55/failuresummary


### nucleo-l496zg/failuresummary

Failures during compilation:
- [tests/pkg_mbedtls](nucleo-l496zg/tests/pkg_mbedtls/compilation.failed)

### nucleo-l452re/failuresummary


### frdm-kw41z/failuresummary

Failures during test:
- [tests/pkg_mbedtls](frdm-kw41z/tests/pkg_mbedtls/test.failed)

### nucleo-l433rc/failuresummary


### i-nucleo-lrwan1/failuresummary


### openmote-b/failuresummary


### nucleo-wl55jc/failuresummary


### nucleo-l4r5zi/failuresummary


### z1/failuresummary


### nucleo-f091rc/failuresummary

Failures during compilation:
- [tests/pkg_mbedtls](nucleo-f091rc/tests/pkg_mbedtls/compilation.failed)

### arduino-zero/failuresummary

Failures during compilation:
- [tests/pkg_mbedtls](arduino-zero/tests/pkg_mbedtls/compilation.failed)

### nucleo-l552ze-q/failuresummary


### nucleo-f303k8/failuresummary


### nucleo-f103rb/failuresummary


### b-l072z-lrwan1/failuresummary


### atmega256rfr2-xpro/failuresummary

Failures during test:
- [tests/pkg_mbedtls](atmega256rfr2-xpro/tests/pkg_mbedtls/test.failed)

### nucleo-f303re/failuresummary


### slstk3402a/failuresummary


### nucleo-f030r8/failuresummary

Failures during test:
- [tests/pkg_mbedtls](nucleo-f030r8/tests/pkg_mbedtls/test.failed)

### nucleo-l073rz/failuresummary

@PeterKietzmann
Copy link
Member Author

Just a quick heads-up:

  • The failure on samr21-xpro is actually a good thing. It has insufficient entropy sources and configures only a fallback ADC line to gather noise using the adc_noise module. This detects failures during the runtime tests which lets the (mbedtls) entropy module fail. I will improve the test to better reflect this behavior.
  • Hanging on nucleo-l152re is due to a missing ADC initialization before use. Some ADC drivers block on sampling before initialization. Turns our, I forgot to call the initialization function in auto_init. Will add it.
  • Lets see if these changes will already let the other boards pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: crypto Area: Cryptographic libraries Area: doc Area: Documentation Area: Kconfig Area: Kconfig integration Area: pkg Area: External package ports Area: security Area: Security-related libraries and subsystems Area: sys Area: System Area: tests Area: tests and testing framework CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Type: new feature The issue requests / The PR implemements a new feature for RIOT

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants