Skip to content

Fix flaky test_multiple_clients_subscription in CI #1754

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 8, 2025
Merged

Fix flaky test_multiple_clients_subscription in CI #1754

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
54ba135
fix: increase timeouts for cross-node operations in test_multiple_cli…
sanity Aug 7, 2025
7f206a8
Merge branch 'main' into fix-flaky-test-timeouts
sanity Aug 8, 2025
3ee60d6
Merge branch 'main' into fix-flaky-test-timeouts
sanity Aug 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 13 additions & 3 deletions crates/core/tests/operations.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -792,18 +792,27 @@ async fn test_multiple_clients_subscription() -> TestResult {
// Add delay to allow contract to propagate from Node A to Node B/C
tracing::info!("Waiting 5 seconds for contract to propagate across nodes...");
tokio::time::sleep(Duration::from_secs(5)).await;

tracing::info!(
"Client 3: Sending GET request for contract {} to Node B",
contract_key
);
let get_start = std::time::Instant::now();
make_get(&mut client_api_node_b, contract_key, true, false).await?;

// Wait for get response on third client
// Note: Contract propagation from Node A to Node B can take 5-10s locally, longer in CI
loop {
let resp =
tokio::time::timeout(Duration::from_secs(30), client_api_node_b.recv()).await;
tokio::time::timeout(Duration::from_secs(60), client_api_node_b.recv()).await;
match resp {
Ok(Ok(HostResponse::ContractResponse(ContractResponse::GetResponse {
key,
contract: Some(_),
state: _,
}))) => {
let elapsed = get_start.elapsed();
tracing::info!("Client 3: Received GET response after {:?}", elapsed);
assert_eq!(
key, contract_key,
"Contract key mismatch in GET response for client 3"
Expand All @@ -820,7 +829,8 @@ async fn test_multiple_clients_subscription() -> TestResult {
bail!("Client 3: Error receiving get response: {}", e);
}
Err(_) => {
bail!("Client 3: Timeout waiting for get response");
let elapsed = get_start.elapsed();
bail!("Client 3: Timeout waiting for get response after {:?}. Contract may not have propagated from Node A to Node B", elapsed);
}
}
}
Expand All @@ -831,7 +841,7 @@ async fn test_multiple_clients_subscription() -> TestResult {
// Wait for subscribe response
loop {
let resp =
tokio::time::timeout(Duration::from_secs(30), client_api_node_b.recv()).await;
tokio::time::timeout(Duration::from_secs(60), client_api_node_b.recv()).await;
match resp {
Ok(Ok(HostResponse::ContractResponse(ContractResponse::SubscribeResponse {
key,
Expand Down
Loading