Deserialize json array into existing Vec

Question

I have some function, here called request, that returns a JSON string with some data. For example, an array with variable size. I can deserialize the JSON string into a Vec just fine with serde_json, but I have to call the request many times. I would like instead to reuse the Vec already created, instead of dropping and creating a new Vec every time deserialization happens.

Is there a way to deserialize a JSON string into an existing Vec? I had a look at the visitor and at this example, but I am not sure which I would have to implement. Either a custom visitor or a deserializer that wraps the serde_json::Deserializer and has a reference to a Vec, or perhaps something else entirely.

Here is some code with an example. Thanks.

use serde_json;

fn request(i: usize) -> String {
    let mut v = vec![0; i];

    for n in 1..i {
        v[n] = n
    }

    serde_json::to_string(&v).unwrap()
}

fn main() {
    let mut bigvec = serde_json::from_str::<Vec<i32>>(&request(100)).unwrap();
    println!("{:?}", &bigvec);

    // - now we will rerun the request many times and we would like to reuse our vec

    let many_iterations = 10;
    for n in 1..many_iterations {
        bigvec.clear();

        let resp = request(n); // we receive a new request with a vec with different length

        // - we would like something like the below, which does not exist
        serde_json::from_str_into::<Vec<i32>>(&resp, &mut bigvec);
    }
}

Playground

Answer 1

There is a hidden¹ deserialize_in_place method on the Deserialize trait that can support this. By default it would simply create a new value and assign it to the existing one, but Vec's impl does exactly what you want.

So you just need to call it manually with the serde-json deserializer:

use serde::Deserialize;
use serde_json::Deserializer;

let mut deserializer = Deserializer::from_str(&resp);
Deserialize::deserialize_in_place(&mut deserializer, &mut bigvec).unwrap();

^{1. This is not a private API. The documentation says "This method is stable and an official public API, but hidden from the documentation because it is almost never what newbies are looking for."}

---

However, using #[derive(Deserialize)] will NOT implement deserialize_in_place unless you enable the deserialize_in_place feature of serde_derive. The default implementation of Deserialize::deserialize_in_place simply calls to Deserialize::deserialize. This means that calling deserialize_in_place will not always perform in-place deserialization.

#[derive(Deserialize)]
struct Foo {
    a: Vec<i32>,
}

let mut foo = Foo { a: Vec::new() };
for data in example_data {
    // If the `deserialize_in_place` feature is not enabled, this call will result
    // in a new `Vec` being created every call.
    let mut deserializer = Deserializer::from_str(&resp);
    Foo::deserialize_in_place(&mut deserializer, &mut foo).unwrap();
}