I have some data being stored in redis cache which will be read by my application in Rust. The data is being stored by python. Whenever I am storing a string or an array, it stores it in a weird form which I was not able to read into Rust. Vice versa, I want to write from Rust and be able to read it in python.
Using django shell:
In [0]: cache.set("test","abc")
In [1]: cache.get("test")
Out[1]:'abc'
Using redis-cli:
127.0.0.1:6379> GET :1:test
"\x80\x04\x95\a\x00\x00\x00\x00\x00\x00\x00\x8c\x03abc\x94."
Output from Rust:
Err(Invalid UTF-8)
Rust code read data using redis-rs library:
let client = redis::Client::open("redis://127.0.0.1:6379")?;
let mut con = client.get_connection()?;
let q:Result<String, redis::RedisError> = con.get(":1:test");
println!("{:?}",q);
I want to be able to read a string or array into Rust as it was written in Python and vice-versa. Also, data in one key will only be ever written by either Rust or Python, not both.
This question is not a duplicate of this as that deals specifically for accent encoding, however, I want to solve my problem for arrays as well. Moreover, the value being set in redis by django for a string is not simply the UTF encoding for the string.
Ah, the joys of trying to throw data across environments. The thing you're being bitten by right now is called Pickle and is the default serializer of django-redis. What a serializer does in this case (in python) is the transformation of your data between python and redis so you can store it, regardless of the type, but more importantly so you can retrieve it with the type it came in.
Obviously, if you had infinite time and effort, you could rewrite pickle in rust and you'd then be able to read this format. I'm pretty sure you have neither, and depending on the data you're storing, you might not even want to do so.
Instead, what I'm going to suggest is to change the serializer from pickle to json. The description of what to change in the config is located at https://django-redis-cache.readthedocs.io/en/latest/advanced_configuration.html#pluggable-serializers , and in particular, I'm pretty sure the class name you want to use is django_redis.serializers.JSONSerializer.
This comes with drawbacks. In particular, there will be some object types you will no longer be able to store on the python side, but if you do really intend to read data on the rust side, this should not concern you.
Sven Marnach mentioned in one of the comments that the serde-pickle crate exists. I have not used it myself, but it does look promising and might save you a ton of interop work if it does function.
To read stuff, now that every key is going to be json, you'll be decoding types with either serde or miniserde. This should be pretty straightforward; do bear in mind that you will not get native types out of this; instead, you'll get members of the serde::Value enum (Boolean, Number, Object, etc), which you will then have to filter through.
Edit your question to indicate what you are trying to store, and I'll happily expand on how to do this on here!