SHA256 of data stream

Question

I am having a use case where i am downloading the file from URL using GET request. Is it possible to calculate SHA256 of file stream without saving to disk or holding an entire object in memory?

Answer 1

This can be accomplished with a MessageDigest and Sink.fold.

First we need a function to create an empty digest and a function to update a digest with a ByteBuffer:

import java.security.MessageDigest
import java.nio.ByteBuffer

def emptySHA256Digest : MessageDigest = MessageDigest getInstance "SHA-256" 

val updateDigest : (MessageDigest, ByteBuffer) => MessageDigest = 
  (messageDigest, byteBuffer) => {
    messageDigest update byteBuffer
    messageDigest
  }

These two functions can then be used within a fold which is applied to the entity of an HttpResponse to update the digest with all ByteString values in the entity:

import akka.http.scaladsl.model.HttpResponse

val responseBodyToDigest : HttpResponse => Future[MessageDigest] = 
  (_ : HttpResponse)
    .entity
    .dataBytes
    .map(_.asByteBuffer)
    .runFold(emptySHA256Digest)(updateDigest)