Search code examples
c++hashcryptographycrypto++

How can I hash bigger text file than 16kB?

I'm using Crypto++ library for hashing. How can I hash bigger file than 16kB (string size)? I want hash files like 1Mb at least.

The error I am getting is when the text is bigger then 16kB:

error C2026: string too big, trailing characters truncated

Here is the code I am using:

std::string hash;
CryptoPP::SHA512 sha;

CryptoPP::StringSource ss(source, true,
                          new CryptoPP::HashFilter(sha,
                              new CryptoPP::HexEncoder(
                                  new CryptoPP::StringSink(hash))));

cout << "SHA-512 hash: " << hash << endl;
Solution

  • When I'm using Crypto++ library for hash how can I hash bigger file than 16kB (string size)? I want hash files like 1Mb at least.

    You should do either one of two things. First, use a FileSource rather than a StringSource. Something like:

    string s1, s2, s3, s4;
SHA1 sha1; SHA224 sha224; SHA256 sha256; SHA512 sha512;

HashFilter f1(sha1, new HexEncoder(new StringSink(s1)));
HashFilter f2(sha224, new HexEncoder(new StringSink(s2)));
HashFilter f3(sha256, new HexEncoder(new StringSink(s3)));
HashFilter f4(sha512, new HexEncoder(new StringSink(s4)));

ChannelSwitch cs;
cs.AddDefaultRoute(f1);
cs.AddDefaultRoute(f2);
cs.AddDefaultRoute(f3);
cs.AddDefaultRoute(f4);

FileSource fs("filename.xxx", true /*pumpAll*/, new Redirector(cs));

cout << Message: " << message << endl;
cout << "SHA-1: " << s1 << endl;
cout << "SHA-224: " << s2 << endl;
cout << "SHA-256: " << s3 << endl;
cout << "SHA-512: " << s4 << endl;

    In general, you can swap in any Crypto++ Source and things will just work.

    Second, you can map a view of the file into your process and then repeatedly call Put in a loop. When you are finished consuming all the data in the file, call MessageEnd and then unmap the file. You call Put and MessageEnd on on the ChannelSwitch above because its the BufferedTransformation of interest.

    On Windows, you map the view of a file with MapViewOfFile. On Linux, you use mmap(2).

    When you map a view of a file, you will effectively have a in-memory byte array backed by disk. In this case, you would use an ArraySource instead of StringSource or FileSource.