Revamping Ferveo Ciphertexts (breaking changes ⚠️ )

[cygnusv]

Type of PR:

• Bugfix
• Feature
• Documentation
• Refactor
• Other

Required reviews:

• 1
• 2
• 3

What this does:
Some of the changes introduced by this PR, all of them related with how ciphertexts are produced and used:

• Actually use the AAD input of the AEAD API of Chacha20Poly1305
  • Closes Use AAD as input of Chacha20Poly1305.encrypt() #146
• Decouple the bulk of symmetric ciphertext from validation; instead just use the hash. This allows to have a small (and also constant!) decryption request.
  • Closes Instead of the symmetric ciphertext, pass its hash when creating/checking the auth_tag #147
  • Implies changes in nucypher-core and nucypher (TBD)
• Refactor Ciphertext implementation.
  • Fixes Remove unused construct_tag_hash and Ciphertext.check() #144
  • Remove unused & incorrect ciphertext length method (see Serialize Ciphertext G1 and G2 points in compressed form #145 )

Apart from this, bumps MSRV to 1.67.0, and associated clippy changes.

Why it's needed:
@jMyles and @KPrasch identified an important limitation of current ciphertexts and TDec requests construction. Essentially, the TDec request included the full ciphertext, which implies a big overhead even for relatively small media files (e.g. short audio clips). The main goal of this PR was to decouple the bulk of symmetric ciphertext from the validation procedure, using a hash instead. This enables very small TDec requests ( see #147). In the nucypher side, will allow to close nucypher/nucypher#3176 via nucypher/nucypher#3194

Notes for reviewers:

What should reviewers focus on?
Is there a particular commit/function/section of your PR that requires more attention from reviewers?

[codecov-commenter]

Codecov Report

Merging #149 (4337c3c) into main (03b4e35) will increase coverage by 0.43%.
The diff coverage is 86.44%.

@@            Coverage Diff             @@
##             main     #149      +/-   ##
==========================================
+ Coverage   77.96%   78.40%   +0.43%     
==========================================
  Files          23       23              
  Lines        4979     4969      -10     
==========================================
+ Hits         3882     3896      +14     
+ Misses       1097     1073      -24     

Files Changed	Coverage Δ
ferveo-common/src/lib.rs	0.00% <0.00%> (ø)
ferveo/src/bindings_python.rs	52.22% <0.00%> (+0.23%)	⬆️
ferveo/src/bindings_wasm.rs	0.00% <0.00%> (ø)
tpke/src/ciphertext.rs	100.00% <100.00%> (+10.58%)	⬆️
tpke/src/context.rs	98.03% <100.00%> (-0.15%)	⬇️
tpke/src/decryption.rs	69.92% <100.00%> (ø)

[piotr-roslaniec]

The error on CI is unrelated to this PR. In order to fix it, bump MSRV to 1.67.
Edit: Removed previous workaround.

[piotr-roslaniec]

LGTM 🎉

(I realize this is WIP, please feel free to re-request a review anytime)

[derekpierre]

@cygnusv, so Ciphertext comprises of:

    #[serde_as(as = "serialization::SerdeAs")]
    pub commitment: E::G1Affine,
    // W
    #[serde_as(as = "serialization::SerdeAs")]
    pub auth_tag: E::G2Affine,
    // V
    #[serde(with = "serde_bytes")]
    pub ciphertext: Vec<u8>,

So from a dem_ciphertext and kem_ciphertext perspective, what is the kem_ciphertext here? i.e. what's the ciphertext data I would put in the ThresholdDecryptionRequest to send to Ursulas?

[cygnusv]

@cygnusv, so Ciphertext comprises of:

    #[serde_as(as = "serialization::SerdeAs")]
    pub commitment: E::G1Affine,
    // W
    #[serde_as(as = "serialization::SerdeAs")]
    pub auth_tag: E::G2Affine,
    // V
    #[serde(with = "serde_bytes")]
    pub ciphertext: Vec<u8>,

So from a dem_ciphertext and kem_ciphertext perspective, what is the kem_ciphertext here? i.e. what's the ciphertext data I would put in the ThresholdDecryptionRequest to send to Ursulas?

Yeah, good question. For the request you only need the commitment, the auth_tag, and the hash of the symmetric ciphertext.

BTW, I'm not sure the term Kem/Dem is technically correct once we introduce the hash of the Dem, but I don't have a better suggestion for the moment.

[derekpierre]

Yeah, good question. For the request you only need the commitment, the auth_tag, and the hash of the symmetric ciphertext.

BTW, I'm not sure the term Kem/Dem is technically correct once we introduce the hash of the Dem, but I don't have a better suggestion for the moment.

@cygnusv, 👍 thanks for the clarification. Could we do a quick chat about this whenever you are around?

[derekpierre]

🎸

[derekpierre]

@cygnusv, 👍 thanks for the clarification. Could we do a quick chat about this whenever you are around?

Chatted with @cygnusv , and there will be a follow-up PR to hide some of the Ferveo-specific math details behind higher-level objects.

[cygnusv]

@cygnusv, 👍 thanks for the clarification. Could we do a quick chat about this whenever you are around?

Chatted with @cygnusv , and there will be a follow-up PR to hide some of the Ferveo-specific math details behind higher-level objects.

Added new issue #154 to track this

[jMyles]

@jMyles and @KPrasch identified an important limitation of current ciphertexts and TDec requests construction.

Credit to Fiddle Kuba ( @kubic71 ) as well.