Search code examples
javascripthtmlhashmd5web-worker

HTML5 Web Workers for huge md5 sum


I'm already using the web workers scenario to get md5 sum from local files, before upload - and it's working well for small files.

But for my app purposes I need to get the hash from a file with 5gb, and all md5 implementations that I found need to allocate all content in one string - what results in an allocation size overflow.

It's possible to get the md5 js hash with memory pagination?


Solution

  • What you are looking for is a algorithm for md5 that supports streaming. There are streaming md5 algorithms that you may want to look into. For example, here is one:

    Note that even though the example I give is an NPM package, it does not have any external dependencies. You should be able to use it in the browser.