Enable more configurability for OSCORE credential storage

[lucien-zuercher]

Is your feature request related to a problem? Please describe.

For the OSCORE integration on server side, there seems to be only the option to
configure the credentials into server via the coap_new_oscore_conf() function.
This credential is in text format and is converted from text -> oscore_conf_t -> oscore_ctx_t.
For embedded devices, there is optimisation potential to save on ram (multiple conversions) and
storage (storing text vs. optimized credentials storage options).
On servers with more resources, the current solution would expect that libcoap stores and manages
the OSCORE credentials currently as linked list. For a large number of connections, other storage formats
might be better suited. Therefor I would like to propose some alternative approaches / improvements.

Describe the solution you would like

The solution would ideally provide following:

1. Prevent the need to store a text format
2. Enable customizing the credentials storage depending on the requirements and/or enabling
   optimization to the available hardware.
   • storage and retrieval optimisation, aka. linked list vs array vs hashmap etc.
   • size optimisation, by storing duplicate information etc.
3. Prevent the need of conversion

Describe alternatives you have considered

1. Provide getter / setter functions for attributes of coap_oscore_conf_t

(+) Non intrusive and minimal change
(+) Allows direct providing of information without convertion text data
(-) Does not allow customized / optimized OSCORE credential storage
(-) Requires the conversion from oscore_conf_t -> oscore_ctx_t

2. Provide getter / setters for an interface directly to oscore_ctx_t

(+) Non intrusive and minimal internal code change
(+) Enables direct configuration of credentials, alternative option for text convertions
(-) Does not allow customized / optimized OSCORE credential storage

3. Provide a callback to be called in oscore_find_context for each request / response.
   This callback would need to provide a subset or all options for
   coap_oscore_conf in combination with (1) or oscore_ctx_t in combination with (2)

(+) Allow the application to optimize the OSCORE credential storage
(+) Depending of how the callback signature looks like, removes the need of data convertion
(+/-) Could mean potential code optimisations
(-) Depending on approach, most likely more intrusive

4. Add an option to skip internal OSCORE to handle it on application level. I am aware, compile time disabling
   is possible, but if a shared library is used, it has benefits if the OSCORE check can be disabled on runtime

(+) Allow custom OSCORE handling and therefore no credential storage dependency on libcoap
(-) Requires handling the OSCORE decoding / encoding in handler, which requires the application to have a custom OSCORE library

Additional context

Generally in my opinion, i see following ranking:
(3) would be the best option but also brings the most work with.
(4) is non intrusive and easy implemented, allowing the application to choose. So the decision is up to the application / integration
(2) possible, but would not solve the customizing of the security storage, and therefore brings its issue on embedded and scalable systems due to storage etc.
(1) seems like a bandage, and not really an option

I am happy to expand on the needs, options and discuss further API options in case (3) is taken into consideration.

Thank you

[mrdeep1]

Certainly pre-complied coap_oscore_conf_t and / or oscore_ctx_t can be stored to save on RAM / configuration compile time.

The concept of (3) works for me - the returned oscore_ctx_tand oscore_recipient_ctx_t are what is used in the other code.

The linked list of oscore_ctx_tI think has to remain, but the recipient_id array could be hashed for speed lookup.

[BitFis]

Thank you for the fast response.

By checking the code, are following assumptions correct?

1. oscore_ctx_t is part of the public API, since the oscore/ files do not contain the postfix _internal.h?
2. Would the find function be passed to libcoap or directly to oscore? Whats your opinion about this question.
3. Since a oscore_find_context function already exists, would you suggest to follow the same structure here, just allow it to be configurable on runtime?

With those questions answered, I would happily provide an MR to propose a solution.

[mrdeep1]

Apologies for the delays in responding.

1. oscore_ctx_t is part of the public API, since the oscore/ files do not contain the postfix _internal.h?

Unfortunately no, all the include/oscore/ files are (currently) internal as structures may change. oscore_ctx_t could easily be exposed as an opaque structure with some set/get functions.

For both the server and client, there is an oscore_ctx_t chain which really define the local (its called sender and has things like sender_context) OSCORE characteristics, then oscore_recipient_ctx_t which is currently linked and defines the remote OSCORE information, but could be hashed (based on rcpkey_id, oscore_r2 and ctxkey_id). However, oscore_find_context() would need to be reworked (certainly optimizations can be just done to the current code) to handle this.

Things are more complicated as there is oscore_association_t which matches tokens used in responses to determine which oscore_ctx_t and recipient_ctx (and other stuff) are in use.

2. Would the find function be passed to libcoap or directly to oscore? Whats your opinion about this question.

The application would optionally define the call back function that is invoked if defined by oscore_find_context(). This call back function could be anywhere.

3. Since a oscore_find_context function already exists, would you suggest to follow the same structure here

I would still call oscore_find_context(), and then if call back defined, use it, else follow the original logic. For this call back, it probably will need the same parameters passed in.

[BitFis]

Going somewhat deeper, I see following connections:

1. oscore_association_t does not need to be touched, since this seems to be anyway pure runtime, so no need to persist and manage memory from the application side.
2. There are multiple approaches to the oscore_find_context(), I agree it is probably the easiest to keep as is. Thought, given oscore_ctx_t is hidden, and my assumption is, this will need to stay that way, it seems a bit complex to provide it via the application in the response. But I can see the clear separation with oscore_recipient_ctx_t as an option. here a few thoughts on in my opinion possible approaches:

2.1 Moving the oscore_ctx_t::next; handling to a functions configured by the application. Iteration over the available oscore context occures only in a few spots, so the memory could be managed by the application and next could be used as customized context.
(+) very flexible, allow application to optimize storage
(-) memory handling gets more complex, responsibility of memory is moved to application, probably not preferred
2.2 Expand the coap3/coap_oscore.h with getter and setter to create and delete oscore_ctx_t values and oscore_recipient_ctx_t. for the oscore_find_context() callback, allows a lazy loading of stored oscore credentials via those create/delete oscore_ctx_t functions. So the lookup function can be extended to manipulate / inject credentials if it is required.
(+) libcoap keeps handling memory
(+) application can inject oscore credentials when they are required, instead of preloading all of them
(-) not sure yet ...

I will be opening an MR as PoC for proposal 2.2 in the coming days, please give me an early heads up, should it be going into a wrong direction.

Thanks

[mrdeep1]

2.2 looks a reasonable direction to go in.

There is already coap_new_oscore_recipient() (and coap_delete_oscore_recipient()), but this only adds / deletes from the primary oscore_ctx_t, so may need to be extended if there is multiple oscore_ctx_t.

#1331 was some work that was done on something similar, but was never merged.

[BitFis]

Brilliant, thanks for the additional information, it seems the oscore_context_t lifecycle management also came up there.

For a clean solution, I see two approaches:

1. Your proposal of defining the oscore create/delete handler and functions, allowing to manage it on application side, as you hinted here OSCORE: Added user-defined callback for finding oscore context #1331 (comment)
2. We let libcoap handle the oscore contexts but remove them from internal libcoap when not used anymore. This could be solved via the oscore_association_t lifecycle. Enforcing to keep the oscore contexts can be triggered however.

I currently I would tend to 2, since this minimizes what needs to be implemented by the application.
And also ensures responsibility of the oscore_context_t memory management to stay inside libcoap and therefore simplifying lifecycle.

Current approach is to use a ref counter to the oscore_recipient_ctx_t, which keeps track of the
number oscore_association_t and if it has been created externally (coap_new_oscore_recipient).
If the ref hits 0, remove the oscore_recipient_ctx_t and if no more oscore_recipient_ctx_t attached to the oscore_context_t cleanup the oscore_context_t aswell. Ensuring libcoap only keeps the oscore_context_t in ram which is currently used by active sessions. Open for discussion to allow force removing an oscore_context_t, but would guess could be resolved via force removing the session instead.

Currently tending to functions as following, but can be better discussed in the MR I am working on:

• void coap_oscore_set_find_func(coap_context_t *, coap_oscore_find_func_t func, void *app_data);
• coap_oscore_ctx_t* coap_oscore_[new|delete](...)

And with the access to the oscore_context_t reference, following setters (getters I am currently checking - just notes):

   // set id and calculates sender_key
  coap_oscore_add_sender(oscore_ctx_t, kid, piv  /* sn */)
  // remove sender from chain
  coap_oscore_remove_sender(oscore_ctx_t, kid)

   // set id and calculates sender_key
  coap_oscore_add_recipient(oscore_ctx_t, kid, piv /* sn */)
  // remove from chain by decrementing ref -> ensuring no issues with oscore_association
  coap_oscore_remove_recipient(oscore_ctx_t, kid)

  // retrieve first in chain, enable loop over available context
  coap_oscore_ctx_t* coap_oscore_context_get(coap_context_t);
  // retrieve next in chain
  coap_oscore_ctx_t* coap_oscore_context_get_next(coap_oscore_ctx_t*);

  // configuration for oscore
  coap_oscore_set_common( // calculates CommonIV
     coap_oscore_ctx_t*,
     coap_bin_const_t ms,
     coap_bin_const_t salt,
     coap_bin_const_t context_id
  );
  coap_oscore_[set|get]_alg(...);
  coap_oscore_[set|get]_hkdf(...);
  coap_oscore_[set|get]_mode(...);
  // use bit flags to set these configs
  coap_oscore_[set|get]_flags(coap_oscore_ctx_t, rfc8613_b_1_2 | rfc8613_b_2);
  coap_oscore_[set|get]_ssn_freq(coap_oscore_ctx_t, uint32_t);
  coap_oscore_[set|get]_replay_windows_size(uint32_t);
  coap_oscore_set_save_seq_num(func, app_data); // update seq num function

For the oscore_find_context I currently tend to follow a minimial approach:

oscore_recipient_ctx_t* oscore_find_context(
  const coap_context_t *c_context,
  const coap_bin_const_t rcpkey_id,
  const coap_bin_const_t ctxkey_id) 
// dropping oscore_b2 and ctxkey_id as pointer, since if enabled, handle outside? No need to handle in application function
// returning oscore_recipient_ctx_t, since oscore_recipient_ctx_t::osc_ctx exists and needs to be set, so also, easier?

[mrdeep1]

Stating the obvious - we need to continue to support the "old" way of doing things in terms of the configuration and not deleting coap_oscore_ctx_t or recipient IDs (but the oscore_recipient_ctx_t should go away when no longer in use) to handle any starting up OSCORE sessions.

I need to double check the code, but when rfc8613_b_1_2 or rfc8613_b_2 are in use during the OSCORE setup / validation of both ends, oscore_association_t may not be there. oscore_association_t is mainly keyed off the response PDU token.

The server is going to need maintain / retrieve the coap_oscore_ctx_t along with recipient IDs when a new OSCORE session starts up in order to build the oscore_recipient_ctx_t for the ongoing session (which is not immediate with rfc8613_b_2). I guess the application will build the coap_oscore_ctx_t / recipient ID in the coap_oscore_find_func_t function. It is likely that one coap_oscore_ctx_t is associated with multiple oscore_recipient_ctx_t.

I agree the more that libcoap can do the management, then the less likely the application has the potential to cause confusion.

[BitFis]

Side question, while going for the implementation of coap_oscore_find_func_t, is following assumption true?

1. coap_oscore_ctx_t 1 --- n oscore_recipient_ctx_t relationship, as you hinted. But oscore_recipient_ctx_t belongs only to one coap_oscore_ctx_t?
2. coap_oscore_ctx_t can be uniquely idenfied by: ctxkey_id, there can NOT be a multiple coap_oscore_ctx_t with different ctxkey_id, while oscore_recipient_ctx_t is uniquely identified by ctxkey_id + rcpkey_id?

[mrdeep1]

1. is correct. However, I think for 2. that ctxkey_id + rcpkey_id has to be unique unique (and only be associated with one coap_oscore_ctx_t), but multiple coap_oscore_ctx_t can have the same ctxkey_id, each with a different set of rcpkey_id, unique for that specific ctxkey_id. Different coap_oscore_ctx_t could, for example, have different master_secret, but the same ctxkey_id.

If you can locate oscore_recipient_ctx_t, then there is a pointer to oscore_ctx_t in the structure.