Peras staging ground #1581
Draft
Peras staging ground #1581
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
amesgen
requested review from
dnadales,
fraser-iohk,
geo2a,
jasagredo and
nfrisby
as code owners
1 task
2 tasks
amesgen
force-pushed
the
peras-staging
branch
3 times, most recently
from
4c69d1c to
1390449
Compare
- Avoid orphans - We actually can't check that the open states are consistent directly as we would need to statefully get that info from the SUT.
It makes sense to use this without using an entire PerasCertDB, so decouple these. It might be nice to rename PerasCertDB-bench, but doesn't seem like a priority.
The diff is actually quite small; it is recommended to review with sth like https://github.com/Wilfred/difftastic
- Add `PerasRoundLength` - introduce the `PerasEnabled` datatype to track values are only used when Peras is enabled - HFC: translate between Peras rounds and slots
amesgen
force-pushed
the
peras-staging
branch
from
5ba50d1 to
36d8f01
Compare
In the presence of EBBs, block numbers can be very misleading, eg the tip block number of a shorter chain can have a higher block number than that of a longer one. To avoid test failures due to this peculiar behavior, we do not look at block numbers at all for the `WeightedSelectView`, and instead measure the length of the fragment (relative to its anchor). Concretely, this change fixes test failures in the ChainDB q-s-m test when testing with eg `k=5` instead of `k=2` (as different candidates can then actually contain *multiple* EBBs). When EBBs are not used (which has been the case on mainnet for >5 years), this change has no semantic impact.
Since the Peras boost per certificate will likely become a protocol parameter, we proactively avoid exposing the current hardcoded value, replacing it with an instantiation of (currently trivial) the PerasCfg builder. In the special cases where it's interesting to vary the boost dynamically (ChainDB q-s-m), validated Peras certs now contain randomly generated boost weights.
Extends the ChainDB model with generator state to support carrying gap blocks in state machine tests. This increases the chances of generating and adding (possibly out-of-order) branching sequences of blocks. This, in turn increases the chances of observing the event where the chain selection logic switches from a longer to a shorter (but heavier) chain containing a boosted block.
This commit increases the generation frequencies of both the
'genAddBlock' and 'genAddPerasCert' constructions to help producing
denser chains of blocks. This way, some of the events that were
harder to trigger (especially TagSwitchedToShorterChain) are much
more common now:
* Before:
Tags (5784 in total):
39.83% TagGetIsValidJust
29.72% TagChainSelReprocessKeptSelection
27.92% TagGetIsValidNothing
2.42% TagChainSelReprocessChangedSelection
0.10% TagSwitchedToShorterChain
* After:
Tags (5202 in total):
38.66% TagGetIsValidJust
27.87% TagChainSelReprocessKeptSelection
26.43% TagGetIsValidNothing
5.71% TagChainSelReprocessChangedSelection
1.33% TagSwitchedToShorterChain
After analysing the effect of varying the security parameter (`k`) of the ChainDB state machine tests (currently hardcoded with 2), we have observed a tension between: 1) generating enough tests exercising the new Peras behavior where the chain selection mechanism switches to a shorter but heavier chain (cert boost is derived from k and must be large enough to overcome the weight of a longer chain), and 2) generating enough tests exercising the ImmutableDB logic (the chain must have at least k blocks) Here are some empirical results: k -> P(switch to shorter chain), P(generate a chain with >= k blocks) k=2 -> ~1.3%, ~40% k=3 -> ~1.9%, ~20% k=4 -> ~2.4%, ~9% k=5 -> ~2.5%, ~3% k=10 -> ~3%, ~0.05% We believe that the sweet spot between both desiderata appears to be around `k=2` and `k=4`. This commit introduces a random generator for `k` using a geometric distribution to bias the randomly generated `k`s to be relatively small, while still allowing larger ones to appear from time to time. Under the current parameters, roughly 75% of the tests use `k<=4`; ``` Security Parameter (k) (10000 in total): 50.82% 2 23.83% 3 12.62% 4 6.69% 5 3.08% 6 1.54% 7 0.74% 8 0.37% 9 0.16% 10 0.06% 11 0.05% 12 0.02% 13 0.01% 14 0.01% 17 ``` Yielding the following distributions for 1) and 2), respectively: ``` Tags (5161 in total): 39.35% TagGetIsValidJust 29.22% TagChainSelReprocessKeptSelection 25.91% TagGetIsValidNothing 3.88% TagChainSelReprocessChangedSelection 1.65% TagSwitchedToShorterChain <- HERE ``` ``` Chain length >= k (10000 in total): 73.25% False 26.75% True <- HERE ```
Brings in cardano-base and propagates a set of `CardanoFeatureFlag`s from the top-level `RunNodeArgs` down to the `NodeKernelArgs`. This is currently needed by an upcoming PR to the GSM to distinguish whether having an established PerasCertDiffusion connection with a given peer is necessary or not when trying to decide if such peer is idling.
Bumps the external ouroboros-network source-repository-package to the updated peras-staging/pr-5202, which incorporates the changes from: IntersectMBO/ouroboros-network#5202 In addition, it tweak call sites of `nodeToNodeProtocols` to match its updated signature, passing down the enabled feature flags.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Peras staging ground, only used for running CI