I'd like to understand the threading model for async/await with tokio in Rust. In particular I'd like to know when async/await will cause code to be executed on different threads.
What I've observed is that running async/await/join code in an async fn main annotated with #[tokio::main] (which creates a multi-threaded runtime) executes all the code in the same thread. I don't see additional threads being used until I start executing code via tokio::spawn.
Some relevant docs.
Tasks are the unit of execution managed by the scheduler. Spawning the task submits it to the Tokio scheduler, which then ensures that the task executes when it has work to do. The spawned task may be executed on the same thread as where it was spawned, or it may execute on a different runtime thread. The task can also be moved between threads after being spawned. (emphasis mine)
This says that a single tokio task that is spawned can be moved between threads during its execution. When would this happen?
There's also the doc for block_on which is what the #[tokio::main] runs the function body in
This runs the given future on the current thread, blocking until it is complete, and yielding its resolved result. Any tasks or timers which the future spawns internally will be executed on the runtime.
This can be read to imply that unless spawning new tasks or timers, the code in the given future will all run on the same thread. This is what I've observed.
Is it that async rust code will progress futures concurrently (but not in parallel) across .await and join calls and only schedule on different threads when tokio::spawn is used?
The docs seem to indicate otherwise, and if that's the case, how is the decision made to move execution to a new thread?
I don't see additional threads being used until I start executing code via tokio::spawn.
That is because they don't, a future is not a unit of scheduling, only the task is.
This says that a single tokio task that is spawned can be moved between threads during its execution. When would this happen?
This requires rewinding a bit: the multi-threaded runtime means tokio has multiple schedulers, one per thread. Each of these threads has a local queue, plus an additional global shared queue for the entire runtime.
By default if a task is spawned from async code, the task is enqueued on the queue of the current scheduler. If the local queue is full or the task is created from outside the runtime (e.g. via Runtime::block_on), then it is instead added to the shared queue.
When a scheduler has no task running, it checks its local queue.
If its local queue is empty or no task is ready, then it tries to steal a task from a sibling. That is when a task can be moved between threads.