Search code examples
c++lz4

LZ4 not compressing strings correctly


Recently, I made an attempt to integrate loss less data compression to my game engine for loading assets; but this simple compression example does not seem to work correctly. Any suggestions ? Here is my code :

 const char *srcData = "Hi ! This is a really really really long test string !";
 const int dstBufferSize = LZ4_compressBound(sizeof(srcData));
 char *dstData = new char[dstBufferSize];
 int bytesPassed = LZ4_compress_default(srcData, dstData,  
                                       sizeof(srcData), 
                                       dstBufferSize); // compress data
 BOOST_LOG_TRIVIAL(info) << dstData << std::endl; // print compressed data

 delete[] dstData;

This is the output. Obviously, you can see it's wrong (part of the string is missing) :

[2016-02-24 15:56:47.986366] [0x00000b0c] [info]    @Hi !═══════════════²²²²À▀WÏÇ0

EDIT When decompressing data, only the 'Hi' part is appearing : the rest are random characters/ no characters

EDIT 2 After Simon's suggestion, I changed the code; but after decompressing the code; i only get Hi ! (nothing after it); Here is the updated code :

 const char *srcData = "Hi ! This is a really really really long test string !";
  const int dstBufferSize = LZ4_compressBound(strlen(srcData) + 1);
  char *dstData = new char[dstBufferSize];
  int bytesPassed = LZ4_compress_default(srcData, dstData,  
                                       sizeof(srcData), 
                                       dstBufferSize);
  BOOST_LOG_TRIVIAL(info) << dstData << std::endl;

  std::ofstream fWriter("test.bin", std::ofstream::binary);
  fWriter << dstData;
  fWriter.close();

  char* decStr = new char[strlen(srcData) + 1];

  LZ4_decompress_fast(dstData, decStr, strlen(srcData) + 1);

  std::cout << decStr << std::endl; // only Hi appearing

  delete[] dstData;

Solution

  • You are using sizeof(srcData) which will give you the size of the pointer and not of the data it points to.

    You should use strlen(srcData)+1 instead (+1 for the \0).

    Or use std::string and std::string::size() (Also with +1 for the null terminator).