[WIP] Experimental ObjectDiffusion v2 (just inbound side changes) for efficient vote retrieval #1698
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tbagrel1
changed the title
tbagrel1
force-pushed
the
peras/experimental-object-diffusion-inbound-v2
branch
2 times, most recently
from
9de5d64 to
443cfc8
Compare
Co-authored-by: nbacquey <nbacquey@users.noreply.github.com>
Co-authored-by: nbacquey <nbacquey@users.noreply.github.com>
Co-authored-by: nbacquey <nbacquey@users.noreply.github.com>
tbagrel1
force-pushed
the
peras/experimental-object-diffusion-inbound-v2
branch
from
96b7016 to
8bd6007
Compare
tbagrel1
changed the base branch from
main-pr/object-diffusion
to
peras-staging
Co-authored-by: nbacquey <nbacquey@users.noreply.github.com>
nbacquey
force-pushed
the
peras/experimental-object-diffusion-inbound-v2
branch
from
d28fe49 to
f794d6f
Compare
Co-authored-by: nbacquey <nbacquey@users.noreply.github.com>
| - the inbound peer that submitted the object to pool might have acked it already at the moment the object is rejected by the pool, but the rejection indicates that the outbound peer which sent us the object is adversarial, and we should disconnect from it anyway. So there is no harm done by having acked the object to the adversarial outbound peer, as we won't want to re-download this object from it again (or any other object whatsoever). | ||
| - any other inbound peer that has this ID available from its outbound peer won't be able to ack it because this ID isn't in **their** `dpsObjectsOwtPool`, and is not in the pool either, so we will be able to download it from these other peers until we find a valid one. | ||
|
|
||
| As in TxSubmission, acknowledgement is done by indicating to the outbound peer the length of the (longest) prefix of the oustanding FIFO that we no longer care about (i.e. for which all IDs are eligible to acknowledgment by the definition above). The field `dpsOutstandingFifo` on the inbound peer is supposed to mirror exactly the state of the FIFO of the outbound peer, bar eventual discrepancies due to in-flight information. |
There was a problem hiding this comment.
I probably missed something along the way, but what happens when a peer receives all but the first object it requests from an outbound peer? Can that object be re-requested from the same peer, or do we rely on some other peer sending it successfully and thus allowing us to move ahead in the outstanding FIFO?
| When making decisions, we first divide the peers in two groups: | ||
|
|
||
| - Those who are currently executing a decision, i.e., those for which the (previous) decision in the decision channel verifies `pdStatus == DecisionBeingActedUpon`. These are further called _frozen peers_. | ||
| - Those who are not currently executing a decision, i.e., those for which the (previous) decision in the decision channel verifies `pdStatus == DecisionUnread || pdStatus == DecisionCompleted`. The former are the ones who didn't have time to read the previous decision yet, so it makes sense to recompute a more up-to-date decision for them. The latter are the ones who have completed executing the previous decision, so it also makes sense to compute a new decision for them. These two categories of peers are further called _active peers_. |
There was a problem hiding this comment.
The former are the ones who didn't have time to read the previous decision yet, so it makes sense to recompute a more up-to-date decision for them.
Could this somehow lead to starvation? I.e. the peer never leaving the DecisionUnread state because the decision thread keeps recomputing a new one.
| - They are not already in the pool | ||
| - They are available from the peer, and won't be acked by the peer on its next request for IDs according to the partial decision computed by `computeAck` at the previous step (NOTE: theoretically, this second condition is redundant with other constraints and invariants of the current implementation). | ||
|
|
||
| Then we "reverse" this mapping to obtain a map of object IDs to the set of active peers that have the corresponding interesting objects available (according to the criteria above), further called _potential providers_. |
There was a problem hiding this comment.
Is there any notion of quality of service that one could use to prioritize certain potential providers over others at this point?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Co-authored-by: nbacquey nbacquey@users.noreply.github.com
Description
Please include a meaningful description of the PR and link the relevant issues
this PR might resolve.
Also note that: