Skip to content

feat: sequential feed#250

Draft
IgorShadurin wants to merge 3 commits intomasterfrom
feat/sequential-feed
Draft

feat: sequential feed#250
IgorShadurin wants to merge 3 commits intomasterfrom
feat/sequential-feed

Conversation

@IgorShadurin
Copy link
Contributor

@IgorShadurin IgorShadurin commented Jul 27, 2023

BREAKING CHANGE!

Implemented ability to use not only Epoch feed, but also Sequence feed.

Epoch vs. Sequence Feed types: speed comparison

The performance of the two types of feeds, Epoch and Sequence, in Bee was tested under two conditions: using a local Bee node and a remote FDS Bee node. The same script was used to test both feed types to ensure consistent conditions. The script performed the following tasks:

  1. Created a new FDP instance using either the Epoch or Sequence feed type.
  2. Generated two random identifiers and created two pods with them.
  3. Generated two random directories and created these directories in each pod.
  4. Uploaded a randomly generated file to each pod.
  5. Downloaded file from each pod.
  6. Read the root directory of each pod in recursive mode.

Here's a comparison of the execution times:

Local Bee Node Test:

  • Epoch: 10.6 seconds
  • Sequence: 12.6 seconds

Remote FDS Bee Node Test (https://bee-1.fairdatasociety.org/):

  • Epoch: 5 minutes 11 seconds (311 seconds)
  • Sequential: 2 minutes 36 seconds (156 seconds)

The Sequence feed took 18.9% longer than the Epoch feed when tested on the local Bee node. However, when tested on the remote FDS Bee node, the Sequence feed executed in just 50.2% of the time it took the Epoch feed, achieving a 49.8% improvement in execution time.

Despite the speed increase, it's worth clarifying with the Bee team about the possibility of parallel data uploading in Sequential feed for use in FairOS. If they've fixed this, it will work not only in fdp-storage but also in FairOS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant