Skip to content
This repository was archived by the owner on Nov 27, 2024. It is now read-only.

Conversation

@gdwoolbert3
Copy link
Owner

@gdwoolbert3 gdwoolbert3 commented Mar 15, 2024

The purpose of this PR is to improve the performance and memory usage. More specifically, this PR makes the following implementation changes:

  • All async flushes are now run in a Task, making them asynchronous not just to a caller (when applicable), but to the partition process. This allows the partition to process inserts faster after a flush condition is met.
  • Buffered items are now stored in a private ETS table instead of the partition state. This decreases the memory footprint of each partition and, more importantly, allows for a task to flush the partition without having to copy it's contents.

@gdwoolbert3 gdwoolbert3 changed the title Rewrite partition module to use ets and tasks Use ets and tasks to improve performance and memory usage Mar 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants