I have a type that implements a certain functionality I'd like to put behind a trait (because I'd like to use automock in order to test the usages of that module in related dependencies). When I try to pass the object wrapped in an Arc to an axum handler without generics (to enforce the trait bound) - everything compiles well. However, when I introduce the trait bound into the story (on the callers of the trait), I get the ominous future returned by ... is not Send. Peppering Send + Sync + 'static all over the place did not do anything to improve the situation and I really can't wrap my head around why this is happening.
The full code is in the gist below, however, for posterity I will try to summarize it here:
use mockall::automock;
use tokio::sync::Mutex;
pub struct JobManager {
jobs: Mutex<HashMap<Job, JobState>>,
}
#[derive(Clone, Error, Debug, PartialEq)]
pub enum JobError {
...
}
#[automock]
pub trait GetOrCreate: Send + Sync + 'static {
async fn get_or_create(&self, job: Job) -> Result<JobState, JobError>;
}
impl GetOrCreate for JobManager {
async fn get_or_create(&self, job: Job) -> Result<JobState, JobError> {
let mut hs = self.jobs.lock().await;
match hs.get(&job).cloned() {
Some(j) => Ok(j),
None => {
... Ok or Err ...
}
}
}
}
impl JobManager {
pub fn new() -> Self {
JobManager {
jobs: Mutex::new(HashMap::new()),
}
}
}
#[derive(Clone, Debug, PartialEq)]
pub enum JobState {
Created,
InProgress,
Done(Result<u64, String>),
}
#[derive(Hash, Eq, PartialEq, Debug, Clone)]
pub struct Job {
pub nonce_group: u8,
pub challenge: [u8; 8],
pub difficulty: [u8; 32],
pub miner: [u8; 32],
}
....
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
...
let job_manager = JobManager::new();
let router = router(Arc::new(job_manager));
let listener = tokio::net::TcpListener::bind(args.bind_address)
.await
.unwrap();
...
}
fn router<T: GetOrCreate + Send + Sync + 'static>(job_manager: Arc<T>) -> Router {
Router::new()
.route(
"/job/.....",
get(get_job),
)
.with_state(job_manager)
.....
}
#[serde_as]
#[derive(Deserialize)]
struct HexStr<const COUNT: usize>(#[serde_as(as = "serde_with::hex::Hex")] [u8; COUNT]);
impl<const COUNT: usize> HexStr<COUNT> {
fn as_array(self) -> [u8; COUNT] {
self.0
}
}
async fn get_job<T: GetOrCreate + Send + Sync + 'static>(
State(manager): State<Arc<T>>,
Path((no, ch, diff, m)): Path<(
u8,
HexStr<8>,
HexStr<32>,
HexStr<32>,
)>,
) -> Result<job_manager::JobState, job_manager::JobError> {
manager
.get_or_create(job_manager::Job {
no,
ch: ch.as_array(),
diff: diff.as_array(),
m: m.as_array(),
})
.await
}
The compile error is as follows:
error: future cannot be sent between threads safely
--> src/main.rs:138:17
|
138 | get(get_job),
| ^^^^^^^ future returned by `get_job` is not `Send`
|
= help: within `impl Future<Output = Result<job_manager::JobState, job_manager::JobError>>`, the trait `Send` is not implemented for `impl Future<Output = Result<job_manager::JobState, job_manager::JobError>>`, which is required by `fn(State<Arc<_>>, axum::extract::Path<(u8, HexStr<8>, HexStr<32>, HexStr<32>)>) -> impl Future<Output = Result<job_manager::JobState, job_manager::JobError>> {get_job::<_>}: Handler<_, _>`
note: future is not `Send` as it awaits another future which is not `Send`
--> src/main.rs:196:5
|
196 | / manager
197 | | .get_or_create(job_manager::Job {
198 | | nonce_group,
199 | | challenge: challenge.as_array().clone(),
200 | | difficulty: difficulty.as_array().clone(),
201 | | miner: miner.as_array().clone(),
202 | | })
| |__________^ await occurs here on type `impl Future<Output = Result<job_manager::JobState, job_manager::JobError>>`, which is not `Send`
note: required by a bound in `axum::routing::get`
--> /home/acud/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.7.5/src/routing/method_routing.rs:385:1
|
385 | top_level_handler_fn!(get, GET);
| ^^^^^^^^^^^^^^^^^^^^^^---^^^^^^
| | |
| | required by a bound in this function
| required by this bound in `get`
= note: this error originates in the macro `top_level_handler_fn` (in Nightly builds, run with -Z macro-backtrace for more info)
help: `Send` can be made part of the associated future's guarantees for all implementations of `GetOrCreate::get_or_create`
|
41 - async fn get_or_create(&self, job: Job) -> Result<JobState, JobError>;
41 + fn get_or_create(&self, job: Job) -> impl std::future::Future<Output = Result<JobState, JobError>> + Send;
|
Once router signature change in the following manner, the code compiles:
- fn router<T: GetOrCreate + Send + Sync + 'static>(job_manager: Arc<T>) -> Router {
+ fn router(job_manager: Arc<JobManager>) -> Router {
Full example (does not compile): https://gist.github.com/acud/f32d653dedd63b52069cc50d2de8801f
Does compile: https://gist.github.com/acud/51019803a46832fb2a916470169ad0d9
The problem is this trait:
pub trait GetOrCreate: Send + Sync + 'static {
async fn get_or_create(&self, job: Job) -> Result<JobState, JobError>;
}
Because this is a trait, and the underlying future implementations can differ, the compiler cannot "see through" the function to determine any auto traits that the future might implement. It has to take the trait itself as the final authority about what the function returns, and the Send bound is not specified here.
To indicate that implementations must return a future implementation that is Send, desugar the async fn syntax into a fn that returns impl Future, and then add the Send bound:
pub trait GetOrCreate: Send + Sync + 'static {
fn get_or_create(
&self,
job: Job,
) -> impl Future<Output = Result<JobState, JobError>> + Send;
}
Because async fn is sugar, you can continue to use async fn in the trait implementations.
Once RTN lands, you'll be able to leave GetOrCreate as it's defined, and move the Send requirement to a bound or router and get_job, if you so desire. This will allow your trait to be implemented regardless of whether the returned future is Send, instead applying that constraint only where it's actually needed.
(Note that you do not need many of the Send + Sync + 'static bounds that you added trying to resolve the issue. I would remove as many as the compiler will allow you to.)