Search code examples
c#excelserializationtabular

Is there a format which can store tabular data in lesser size than excel?


I have already tried out keeping in text file(Obviosuly it takes more space as it is not customized to hold tables as an excel is). I also tried keeping serialized data(c#, just an fyi). There wasn't any benefit in either cases.


Solution

  • Since most 'tabular' data will be text, you'll get large gains in compressing this before saving (using GZip and BsonWriter as an example):

    public static byte[] Compress(object entity)
    {
      using (var stream = new MemoryStream())
      {
        using (var zipStream = new GZipStream(stream, CompressionLevel.Optimal))
        {
          using (var writer = new BsonWriter(zipStream))
          {
            var serializer = new JsonSerializer();
            serializer.Serialize(writer, entity);
          }
        }
    
        return stream.ToArray();
      }
    }
    

    ** Update **

    Protobuf is far superior to both XmlSerializer and BinaryFormatter in terms of serialized entity size and speed. Try using this to serialize your entities before compressing:

    public static byte[] Compress(object entity)
    {
      using (var stream = new MemoryStream())
      {
        using (var zipStream = new GZipStream(stream, CompressionLevel.Optimal))
        {
          Serializer.Serialize(stream, entity);  
        }
    
        return stream.ToArray();
      }
    }