-
Notifications
You must be signed in to change notification settings - Fork 64
Coordination lock api #723
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
slampy97
wants to merge
42
commits into
ydb-platform:main
Choose a base branch
from
slampy97:coordination-lock
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
42 commits
Select commit
Hold shift + click to select a range
2d5bfdf
first step
slampy97 96c0ff1
also driver
slampy97 a4a292e
working coord client plus not working lock/ conenction problemsg
slampy97 d9de575
add wrappers, simple test is working, but problem to got structure back
slampy97 7701a05
several fixes plus wrappers
slampy97 82669fc
delete path from pr
slampy97 9fd7073
fix add gitmodules
slampy97 44a20a6
add alter node
slampy97 d749eb0
fix
slampy97 a5a36f6
fix config
slampy97 53ebed9
erase wrappers
slampy97 678598b
fix public wrappers
slampy97 1a6e166
fix last drivers coordination client inclusion
slampy97 aa8eaf9
fix tox -e style
slampy97 bd96246
tox -e black-format
slampy97 c3324cf
fix review remarks
slampy97 41323e6
fix review remarks
slampy97 65edbea
fix flake8 mistakes
slampy97 d2d25d4
fix review remarks plus styles checks
slampy97 9850b78
working async client, session_init + ping pong + lock release/acquire
slampy97 3f55d20
working async client, session_init + ping pong + lock release/acquire…
slampy97 89fe18d
some fix / redundant logic
slampy97 31e4fb6
fix lock logic
slampy97 194942f
refactor logic lock + reconnecor + stream - > lock should be resosibl…
slampy97 a81db7e
add describe -> start making crud lock object
slampy97 8defd59
crud lock object + acquire and release.
slampy97 53d547b
Merge branch 'coordination-service-impementation' into coordination-lock
slampy97 7d78387
add style checking
slampy97 4f66aad
fix linter
slampy97 45fc213
Add public wrappers for describe lock result
slampy97 e7a4357
rewrite stream with wrapper plus sync client
slampy97 c16e51e
first working version of sync and asunc client with lock api
slampy97 ae7f76c
refactor + reformat
slampy97 44a73fe
refactor + reformat
slampy97 82ea901
fixing format
slampy97 6f70b24
add NodeConfig as it was before
slampy97 134982a
simplify lock classes
slampy97 29d74b8
simplify lock classes
slampy97 4742860
fix common_utils.py + simplify stream.py
slampy97 202e4b5
simplify stream.py + correct lock.py
slampy97 b1a1849
fix lock delete method
slampy97 633f2d4
fix acquire method
slampy97 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,13 +1,19 @@ | ||
| import asyncio | ||
| import threading | ||
| import time | ||
|
|
||
| import pytest | ||
|
|
||
| import ydb | ||
| from ydb import aio | ||
| from ydb import aio, StatusCode, logger | ||
|
|
||
| from ydb.coordination import ( | ||
| NodeConfig, | ||
| ConsistencyMode, | ||
| RateLimiterCountersMode, | ||
| CoordinationClient, | ||
| CreateSemaphoreResult, | ||
| DescribeLockResult, | ||
| ) | ||
|
|
||
|
|
||
|
|
@@ -93,3 +99,170 @@ async def test_coordination_node_lifecycle_async(self, aio_connection): | |
|
|
||
| with pytest.raises(ydb.SchemeError): | ||
| await client.describe_node(node_path) | ||
|
|
||
| async def test_coordination_lock_full_lifecycle(self, aio_connection): | ||
| client = aio.CoordinationClient(aio_connection) | ||
|
|
||
| node_path = "/local/test_lock_full_lifecycle" | ||
|
|
||
| try: | ||
| await client.delete_node(node_path) | ||
| except ydb.SchemeError: | ||
| pass | ||
|
|
||
| await client.create_node( | ||
| node_path, | ||
| NodeConfig( | ||
| session_grace_period_millis=1000, | ||
| attach_consistency_mode=ConsistencyMode.STRICT, | ||
| read_consistency_mode=ConsistencyMode.STRICT, | ||
| rate_limiter_counters_mode=RateLimiterCountersMode.UNSET, | ||
| self_check_period_millis=0, | ||
| ), | ||
| ) | ||
|
|
||
| lock = client.lock("test_lock", node_path) | ||
|
|
||
| create_resp: CreateSemaphoreResult = await lock.create(init_limit=1, init_data=b"init-data") | ||
| assert create_resp.status == StatusCode.SUCCESS | ||
|
|
||
| describe_resp: DescribeLockResult = await lock.describe() | ||
| assert describe_resp.status == StatusCode.SUCCESS | ||
| assert describe_resp.name == "test_lock" | ||
| assert describe_resp.data == b"init-data" | ||
| assert describe_resp.count == 0 | ||
| assert describe_resp.ephemeral is False | ||
| assert list(describe_resp.owners) == [] | ||
| assert list(describe_resp.waiters) == [] | ||
|
|
||
| update_resp = await lock.update(new_data=b"updated-data") | ||
| assert update_resp.status == StatusCode.SUCCESS | ||
|
|
||
| describe_resp2: DescribeLockResult = await lock.describe() | ||
| assert describe_resp2.status == StatusCode.SUCCESS | ||
| assert describe_resp2.name == "test_lock" | ||
| assert describe_resp2.data == b"updated-data" | ||
| assert describe_resp2.count == 0 | ||
| assert describe_resp2.ephemeral is False | ||
| assert list(describe_resp2.owners) == [] | ||
| assert list(describe_resp2.waiters) == [] | ||
|
|
||
| lock2_started = asyncio.Event() | ||
| lock2_acquired = asyncio.Event() | ||
|
|
||
| async def second_lock_task(): | ||
| lock2_started.set() | ||
| async with client.lock("test_lock", node_path): | ||
| lock2_acquired.set() | ||
| await asyncio.sleep(0.5) | ||
|
|
||
| async with client.lock("test_lock", node_path) as lock1: | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. очень сложно это все читать. давай будем пользоваться методами acquire и release, а также не будем делать этот километровый дескрайб |
||
|
|
||
| resp: DescribeLockResult = await lock1.describe() | ||
| assert resp.status == StatusCode.SUCCESS | ||
| assert resp.name == "test_lock" | ||
| assert resp.data == b"updated-data" | ||
| assert resp.count == 1 | ||
| assert resp.ephemeral is False | ||
| assert len(list(resp.owners)) == 1 | ||
| assert list(resp.waiters) == [] | ||
|
|
||
| t2 = asyncio.create_task(second_lock_task()) | ||
| await lock2_started.wait() | ||
|
|
||
| await asyncio.sleep(0.5) | ||
|
|
||
| await asyncio.wait_for(lock2_acquired.wait(), timeout=5) | ||
| await asyncio.wait_for(t2, timeout=5) | ||
|
|
||
| async with client.lock("test_lock", node_path) as lock3: | ||
|
|
||
| resp3: DescribeLockResult = await lock3.describe() | ||
| assert resp3.status == StatusCode.SUCCESS | ||
| assert resp3.count == 1 | ||
|
|
||
| delete_resp = await lock.delete() | ||
| assert delete_resp.status == StatusCode.SUCCESS | ||
|
|
||
| describe_after_delete: DescribeLockResult = await lock.describe() | ||
| assert describe_after_delete.status == StatusCode.NOT_FOUND | ||
|
|
||
| def test_coordination_lock_full_lifecycle_sync(self, driver_sync): | ||
| client = CoordinationClient(driver_sync) | ||
| node_path = "/local/test_lock_full_lifecycle" | ||
|
|
||
| try: | ||
| client.delete_node(node_path) | ||
| except ydb.SchemeError: | ||
| pass | ||
|
|
||
| client.create_node( | ||
| node_path, | ||
| NodeConfig( | ||
| session_grace_period_millis=1000, | ||
| attach_consistency_mode=ConsistencyMode.STRICT, | ||
| read_consistency_mode=ConsistencyMode.STRICT, | ||
| rate_limiter_counters_mode=RateLimiterCountersMode.UNSET, | ||
| self_check_period_millis=0, | ||
| ), | ||
| ) | ||
|
|
||
| lock = client.lock("test_lock", node_path) | ||
|
|
||
| create_resp: CreateSemaphoreResult = lock.create(init_limit=1, init_data=b"init-data") | ||
| assert create_resp.status == StatusCode.SUCCESS | ||
|
|
||
| describe_resp: DescribeLockResult = lock.describe() | ||
| assert describe_resp.status == StatusCode.SUCCESS | ||
| assert describe_resp.data == b"init-data" | ||
|
|
||
| update_resp = lock.update(new_data=b"updated-data") | ||
| assert update_resp.status == StatusCode.SUCCESS | ||
| assert lock.describe().data == b"updated-data" | ||
|
|
||
| lock2_ready = threading.Event() | ||
| lock2_acquired = threading.Event() | ||
| thread_exc = {"err": None} | ||
|
|
||
| def second_lock_task(): | ||
| try: | ||
| lock2_ready.set() | ||
| with client.lock("test_lock", node_path): | ||
| lock2_acquired.set() | ||
| logger.info("Second thread acquired lock") | ||
| except Exception as e: | ||
| logger.exception("second_lock_task failed") | ||
| thread_exc["err"] = e | ||
|
|
||
| t2 = threading.Thread(target=second_lock_task) | ||
|
|
||
| with client.lock("test_lock", node_path) as lock1: | ||
| resp = lock1.describe() | ||
| assert resp.status == StatusCode.SUCCESS | ||
| assert resp.count == 1 | ||
|
|
||
| t2.start() | ||
| started = lock2_ready.wait(timeout=2.0) | ||
| assert started, "Second thread did not signal readiness to acquire lock" | ||
|
|
||
| acquired = lock2_acquired.wait(timeout=10.0) | ||
| t2.join(timeout=5.0) | ||
|
|
||
| if not acquired: | ||
| if thread_exc["err"]: | ||
| raise AssertionError(f"Second thread raised exception: {thread_exc['err']!r}") from thread_exc["err"] | ||
| else: | ||
| raise AssertionError("Second thread did not acquire the lock in time. Check logs for details.") | ||
|
|
||
| assert not t2.is_alive(), "Second thread did not finish after acquiring lock" | ||
|
|
||
| with client.lock("test_lock", node_path) as lock3: | ||
| resp3: DescribeLockResult = lock3.describe() | ||
| assert resp3.status == StatusCode.SUCCESS | ||
| assert resp3.count == 1 | ||
|
|
||
| delete_resp = lock.delete() | ||
| assert delete_resp.status == StatusCode.SUCCESS | ||
| time.sleep(0.1) | ||
| describe_after_delete: DescribeLockResult = lock.describe() | ||
| assert describe_after_delete.status == StatusCode.NOT_FOUND | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
зачем это пользователю? в случае ошибки мы получим эксепшн, в случае успеха - нет