CMSSignerDigestMismatchException: when trying to digitally sign document using deferred signature

I am trying to digitially sign a PDF document using Digital signature using Java itext library.

Here are my implementation details:

I first retrieved the certificate chain with which to sign.

        CertificateFactory cf = CertificateFactory.getInstance("X509");
        Certificate myCert = cf.generateCertificate(new ByteArrayInputStream(Hex.decode(certHexStr)));
        Certificate[] chain = new Certificate[] { myCert };

Then I added empty signature field

PdfStamper stamper = PdfStamper.createSignature(reader, os, '\0', null, true);
PdfSignatureAppearance appearance = stamper.getSignatureAppearance();
String fieldName = appearance.getFieldName();
document.setSigField(fieldName);
logger.info("Singnature fieldName: " + fieldName);
appearance.setCertificate(clientCertChain[0]);

if (document.getSignReq() != null) {
    SignRequest signReq = document.getSignReq();
    if (StringUtils.isNotBlank(signReq.getReason()))
        appearance.setReason(signReq.getReason());

    if (StringUtils.isNotBlank(signReq.getLocation()))
        appearance.setLocation(signReq.getLocation());
    int pageNumber = signReq.getPageNumber() == 0 ? 1 : signReq.getPageNumber();
    Rectangle rect = new Rectangle(signReq.getLeftX(), signReq.getLeftY(), signReq.getRightX(),
            signReq.getRightY());
    logger.info(rect.toString());
    appearance.setVisibleSignature(rect, pageNumber, fieldName);

Then I created the hash of the document

// PrivateKeySignature signature = new PrivateKeySignature(pk, "SHA256",
// "SunMSCAPI");
String hashAlgorithm = "SHA256"; // "SHA-256"
BouncyCastleDigest digest = new BouncyCastleDigest();
// saving the hash for using later
byte hash[] = DigestAlgorithms.digest(rg, digest.getMessageDigest(hashAlgorithm));

// logger.debug("pre-hash: " + Arrays.toString(hash));
String encodedhHash = Base64.getEncoder().encodeToString(hash);
document.setDigest(encodedhHash);

I then signed the digest using a remote client

PdfPKCS7 sgn = new PdfPKCS7(null, certs, Constants.hashAlgorithm, null, bcDigest, false);

I then merged the signature

byte[] signedEncoded = Base64.getDecoder().decode(document.getSignedEncoded());

logger.info("Signature fieldname: " + document.getSigField());
AcroFields af = reader.getAcroFields();

PdfDictionary v = af.getSignatureDictionary(document.getSigField());
if (v == null)
    throw new DocumentException("No field");
if (!af.signatureCoversWholeDocument(document.getSigField()))
    throw new DocumentException("Not the last signature");
PdfArray b = v.getAsArray(PdfName.BYTERANGE);
long[] gaps = b.asLongArray();
if (b.size() != 4 || gaps[0] != 0)
    throw new DocumentException("Single exclusion space supported");
RandomAccessSource readerSource = reader.getSafeFile().createSourceView();

int spaceAvailable = (int) (gaps[2] - gaps[1]) - 2;
if ((spaceAvailable & 1) != 0)
    throw new DocumentException("Gap is not a multiple of 2");
spaceAvailable /= 2;
if (spaceAvailable < signedContent.length)
    throw new DocumentException("Not enough space");
StreamUtil.CopyBytes(readerSource, 0, gaps[1] + 1, outs);
ByteBuffer bb = new ByteBuffer(spaceAvailable * 2);
for (byte bi : signedContent) {
    bb.appendHex(bi);
}
int remain = (spaceAvailable - signedContent.length) * 2;
for (int k = 0; k < remain; ++k) {
    bb.append((byte) 48);
}
bb.writeTo(outs);
StreamUtil.CopyBytes(readerSource, gaps[2] - 1, gaps[3] + 1, outs);
reader.close();
bb.close();
outs.close();

But when I try to verify the signature, it says,

CMSSignerDigestMismatchException: message-digest attribute value does not match calculated value - when trying to digitally sign document using deferred signature.

Also in pdf viewer, it says the pdf has been modified after signature

I am using Java 11 with bouncycastle 1.70 and itext 7.0.4

Solution

As evident in https://www.rfc-editor.org/rfc/rfc8017#section-9.2, the digest is to be padded with bytevalue (0x)30 31 30 0d 06 09 60 86 48 01 65 03 04 02 01 05 00 04 20 (we are using SHA256).

        byte[] sha256bytes = new byte[] { 0x30, 0x31, 0x30, 0x0d, 0x06, 0x09, 0x60, (byte) 0x86, 0x48, 0x01, 0x65, 0x03,
                0x04, 0x02, 0x01, 0x05, 0x00, 0x04, 0x20 };
        return ArrayUtils.addAll(sha256bytes, digest);

It should be the job of the remote signing server. But sending digest with the prefix fixed the issue.