Search code examples
node.jsredisioredis

Storing string data with compressed (buffer) data together on Redis


Is there any chance to store Buffer data with a string data together on Redis.

Pseudo code:

// Data compression
var user = { name: "Xyz" }
var userString = JSON.stringify(user)
var userBuffer = new Buffer(userString, "utf8")
var compressed = zlib.gzip(userBuffer)
// Adding data onto Redis
var data = { id: 1, compressed: compressed }
var dataString = JSON.stringify(data)
redis.set("test", dataString)

Although it seems impossible I wanna ask.

Is there any chance to store compressed data with string together?

Edit:

After storing compressed data with uncompressed data in the same key, I cannot decompress the user data above.

Pseudo code:

var dataString = redis.get("test")
var data = JSON.parse(dataString)
console.log(data) 
// writes { id:1, compressed: { type: Buffer, data: [31, 139, 8...] } }

var compressed = data.compressed
var user = zlib.gunzip(compressed)
// user would be undefined here

Solution

  • const zlib     = require('zlib');
    const redis    = require('redis').createClient();
    var user       = { name: "Xyz" }
    var userString = JSON.stringify(user)
    var userBuffer = new Buffer(userString, "utf8")
    var compressed = zlib.gzipSync(userBuffer)
    var data       = { id: 1, compressed: compressed }
    var dataString = JSON.stringify(data)
    
    console.log('Compressed', dataString);
    redis.set('mykey', dataString, err => {
      if (err) throw err;
      redis.get('mykey', (err, dataString) => {
        if (err) throw err;
    
        var data         = JSON.parse(dataString)
        var buffer       = Buffer.from(data.compressed);
        var uncompressed = zlib.gunzipSync(buffer);
    
        console.log('Uncompressed', JSON.parse(uncompressed));
        redis.end(false);
      })
    });
    

    The key is to convert the string-representation of the Buffer back to a proper instance before decompressing it (using Buffer.from()).

    However, the JSON-representation of a Buffer isn't very concise, so you probably should look into an alternative way of storing the data in Redis (one where you don't have to stringify the buffer). Perhaps using a hash.