Search code examples
ruststaticmutablepyo3lazy-static

Store an object in a static Box and then retrieve as mutable?


This is a PyO3 situation. So the Python calls a long-running task. Using ZeroMQ messaging (inter-process messaging*) it is possible for the user to order the task to be suspended. At that point I want to store the "handling framework" object in a static box so that, possibly, the user might later click "Resume", and by so doing, back in Rust, on a second call, the framework object can be retrieved from this static box and put to work again from where it left off. The great advantage of this would be that the state of the framework could be preserved.

I'm inspired to attempt this by the fact that, somewhat to my amazement, static structures in Rust memory appear to be preserved between Python calls to PyO3 functions.

This is how I'm "stowing" the framework on detecting a "suspend" command:

static SUSPENDED_FRAMEWORK_OPTION: OnceLock<Box<HandlingFramework>> = OnceLock::new();
...
let mut handling_framework = HandlingFramework::new(op_type, index_name, dir_root_path_str, current_app_version, current_index_version, tx_zero_client);
handling_framework.init()?; 
let (job_suspended_state, total_word_count, total_ldoc_count) = handling_framework.index_docs()?;
if std::matches!(job_suspended_state, JobSuspendedState::Suspended) {
    let _ = SUSPENDED_FRAMEWORK_OPTION.set(Box::new(handling_framework))
}

... and this is how I'm attempting to retrieve the framework on a later PyO3 call:

if op_type == "RESUME".to_string(){
    let option_for_framework = SUSPENDED_FRAMEWORK_OPTION.get();
    let framework_box_ref = option_for_framework.unwrap();
    info!("text_doc_hit_objs_for_processing remaining {}", framework_box_ref.text_doc_hit_objs_for_processing.len());
    info!("framework_box_ref type {}", str_type_of(&framework_box_ref));

gives framework_box_ref type &alloc::boxed::Box<populate_index::handling_framework::HandlingFramework>

This works in the sense that I am able to get read-access to the various objects in the retrieved framework.

But to do anything useful (to resume processing) I need to have mutable access to the framework object. Everything I'm trying there is failing, usually along the lines of "cannot move out of a shared reference".

I've tried things like this:

fn unbox<T>(value: Box<T>) -> T {
    *value
}
let mut framework = unbox(*framework_box_ref);

This gives:

error[E0507]: cannot move out of `*framework_box_ref` which is behind a shared reference
   --> src\lib.rs:114:29
    |
114 |         let mut framework = unbox(*framework_box_ref);
    |                                   ^^^^^^^^^^^^^^^^^^ move occurs because `*framework_box_ref` has type `Box<HandlingFramework>`, which does not implement the `Copy` trait

Can anyone suggest how this mutable variable might be obtained? At the moment HandlingFramework doesn't implement Copy or Clone... I've tried to derive these but there are too many internal fields which don't implement one or both. I can potentially implement one or both "manually" if required, but for the moment I'm hopeful of getting mutable access without that...


* actually this isn't inter-process messaging but inter-thread messaging: the Rust code appears to run in the same thread as the Python worker thread which calls it. So another Python thread is used to send the message...


Solution

  • There is no way to write to the value in a OnceLock without some kind of interior mutability, at which point you should just use that instead of OnceLock. Note that you can't move from a reference anyway, not even a &mut; to do that you need something to swap in so that the referent is still valid.

    It's also not clear exactly why you need a Box at all.

    Consider instead using a Mutex<Option<_>>. You can use Option::take to "steal" the contents of the Option, leaving None behind.

    static SUSPENDED_FRAMEWORK_OPTION: Mutex<Option<HandlingFramework>> = Mutex::new(None);
    
    // ...
    
    if std::matches!(job_suspended_state, JobSuspendedState::Suspended) {
        *SUSPENDED_FRAMEWORK_OPTION.lock().unwrap() = Some(handling_framework);
    }
    

    Later you can take out the value like this:

    let mut framework = SUSPENDED_FRAMEWORK_OPTION
        .lock()
        .unwrap()
        .take()
        .expect("SUSPENDED_FRAMEWORK_OPTION was None");
    

    However, I would strongly discourage use of global state like this. It would be better to stash this value on a Python object that represents the state of the framework, and then you can get it from there instead of needing a global.