Skip to content

Commit 3e0a1e2

Browse files
QqwyOpsBotPrime
authored andcommitted
Clarify why we're using a 1-thread multi_threaded Tokio runtime in the documentation of its construction
1 parent e6cb764 commit 3e0a1e2

File tree

1 file changed

+18
-4
lines changed

1 file changed

+18
-4
lines changed

libs/opsqueue_python/src/common.rs

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -443,10 +443,24 @@ pub async fn check_signals_in_background() -> FatalPythonException {
443443

444444
/// Sets up a Tokio runtime to use for a client.
445445
///
446-
/// Rather than the current-thread scheduler,
447-
/// we use a (single extra!) background thread,
448-
/// allowing us to keep (GIL-less) tasks alive in the background
449-
/// even when returning back to Python
446+
/// Note that we very intentionally use the multi-threaded scheduler
447+
/// but with a single thread.
448+
///
449+
/// We **cannot** use the current_thread scheduler,
450+
/// since that would result in the Python task scheduler (e.g. `asyncio`)
451+
/// and the Rust task scheduler (`Tokio`) to run on the same thread.
452+
/// This seems to work fine, until you end up with a Python future
453+
/// that depends on a Rust future completing or vice-versa:
454+
/// Since the task schedulers each have their own task queues,
455+
/// work co-operatively, and know nothing of each-other,
456+
/// they will not (nor can they) yield to the other.
457+
/// The result: deadlocks!
458+
///
459+
/// Therefore, we run the Tokio scheduler on a separate thread.
460+
/// Since switching between the 'Python scheduler thread' and the
461+
/// 'Tokio scheduler thread' is preemptive,
462+
/// the same problem now no longer occurs:
463+
/// Both schedulers are able to make forward progress (even on a 1-CPU machine).
450464
pub fn start_runtime() -> Arc<tokio::runtime::Runtime> {
451465
let runtime = tokio::runtime::Builder::new_multi_thread()
452466
.worker_threads(1)

0 commit comments

Comments
 (0)